Intra-frame prediction mode decoding method and device

Abstract

The present application discloses a decoding method and device for an intra-frame prediction mode. It relates to the field of video or image compression technology based on artificial intelligence (AI), and specifically to the field of video compression technology based on neural networks. The method includes an encoding method and a decoding method, and the encoding method includes: obtaining a probability vector of the current image block according to at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding images of the current image block and a prediction mode probability model; encoding the target intra-frame prediction mode into a bitstream according to the target intra-frame prediction mode and probability vector of the current image block. The decoding method includes: obtaining the probability vector of the current image block using the same method as the encoding end; determining the target intra-frame prediction mode according to the bitstream and the probability vector. The method in the embodiment of the present application can improve accuracy, save storage overhead, and reduce computational complexity when encoding and decoding the target intra-frame prediction mode of the current image block.

Description

Technical Field

[0001] The present application relates to the field of video or image compression technology based on artificial intelligence (AI), and in particular to a decoding method and device for an intra-frame prediction mode. Background Art

[0002] Video coding (video encoding and decoding) is widely used in digital video applications such as broadcast digital television, video transmission over the Internet and mobile networks, real-time conversation applications such as video chat and video conferencing, DVD and Blu-ray Discs, video content acquisition and editing systems, and security applications in camcorders.

[0003] Even in the case of short films, large amounts of video data are required to describe them, which can create difficulties when the data is to be sent or otherwise transmitted across networks with limited bandwidth capacity. Therefore, video data is often compressed before being transmitted across modern telecommunications networks. The size of the video can also be an issue when storing the video on a storage device, as memory resources may be limited. Video compression devices typically use software and / or hardware on the source side to encode the video data prior to transmission or storage, thereby reducing the amount of data required to represent the digital video image. The compressed data is then received by a video decompression device on the destination side. With limited network resources and the growing demand for higher video quality, there is a need for improved compression and decompression techniques that can increase compression rates with little to no impact on image quality.

[0004] In recent years, applying deep learning to the field of image and video encoding and decoding has gradually become a trend.

[0005] Among them, the decoding method of the intra-frame prediction mode of HEVC based on neural network introduces the neural network model into the intra-frame prediction process. During encoding, the syntax elements related to the intra-frame prediction mode of the current image block are obtained with the help of the neural network model, and the obtained syntax elements are encoded; during decoding, the intra-frame prediction mode of the current image block is determined according to the syntax elements decoded in the bitstream and the neural network model.

[0006] However, the prior art has problems such as limited accuracy, large storage overhead, and high computational complexity when encoding and decoding the intra-frame prediction mode of the current image block. Summary of the invention

[0007] The present application provides a method and device for decoding an intra-frame prediction mode based on a neural network, which can improve the accuracy of encoding and decoding the intra-frame prediction mode of the current image block, save storage space, reduce computational complexity, and thus improve encoding and decoding performance.

[0008] Specific embodiments are outlined in the attached independent claims, further embodiments are outlined in the dependent claims.

[0009] In a first aspect, the present application provides a method for encoding an intra-frame prediction mode, the method comprising: obtaining at least two of a reconstructed block, a prediction block and a residual block of surrounding image blocks of a current image block; obtaining a probability vector of the current image block based on at least two of the reconstructed block, the prediction block and the residual block of the surrounding image blocks and a prediction mode probability model, wherein multiple elements in the probability vector correspond one-to-one to multiple intra-frame prediction modes, and any element in the probability vector is used to characterize the probability of using an intra-frame prediction mode corresponding to any element when predicting the current image block; determining a target intra-frame prediction mode for the current image block; and encoding the target intra-frame prediction mode into a bitstream based on the target intra-frame prediction mode and the probability vector.

[0010] It can be seen that in the embodiment of the present application, at least two of the reconstructed blocks, prediction blocks and residual blocks of the three surrounding image blocks of the current image block are obtained. Since the other one can be calculated from two of the reconstructed blocks, prediction blocks and residual blocks of each surrounding image block, the embodiment of the present application can make full use of the relevant information of the surrounding image blocks, and then the probability vector of the current image block generated according to at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks is more accurate, thereby improving the accuracy of the data encoded in the bitstream and improving the encoding performance.

[0011] In one possible design, the above-mentioned obtaining of the probability vector of the current image block based on at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks, the current image block and the prediction mode probability model includes: when the target intra-frame prediction mode is a non-matrix weighted intra-frame prediction MIP mode, obtaining the probability vector of the current image block based on at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks, the current image block and the prediction mode probability model.

[0012] It can be seen that when the target intra-frame prediction mode is a non-MIP mode, the method in the embodiment of the present application is used to encode the intra-frame prediction mode of the current image block. By obtaining at least two of the reconstructed blocks, prediction blocks and residual blocks of the three surrounding image blocks of the current image block, the embodiment of the present application can make full use of the relevant information of the surrounding image blocks, and then the probability vector of the current image block generated according to at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks is more accurate, thereby improving the accuracy of the data encoded in the bitstream and improving the encoding performance when the target intra-frame prediction mode is a non-MIP mode.

[0013] In one possible design, the above-mentioned obtaining the probability vector of the current image block based on at least two of the reconstructed block, the prediction block and the residual block of the surrounding image blocks and the prediction mode probability model includes: splicing or cascading at least two of the reconstructed block, the prediction block and the residual block of the surrounding image blocks to obtain a first data block; and obtaining the probability vector of the current image block based on the first data block and the prediction mode probability model.

[0014] It should be understood that the prediction mode probability model in the embodiment of the present application can be a neural network model, and second data blocks of different sizes correspond to different neural network models.

[0015] It can be seen that in the embodiment of the present application, since at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks can fully characterize the relevant information of the surrounding image blocks, the first data block is obtained by splicing or cascading at least two of the reconstructed blocks, prediction blocks and residual blocks of the three surrounding image blocks. Therefore, the first data block can fully reflect the relevant information of the current image block, and then the probability vector of the current image block obtained according to the first data block is more accurate, thereby improving the accuracy of the data encoded in the bitstream and improving the encoding performance.

[0016] In a possible design, obtaining the probability vector of the current image block according to the first data block and the prediction mode probability model includes: when the size of the first data block is the target size, inputting the first data block into the prediction mode probability model for processing to obtain the probability vector of the current image block, or;

[0017] When the size of the first data block is not the target size, a size transformation operation is performed on the first data block to obtain a second data block, the size of the second data block is equal to the target size, the size transformation operation includes at least one of scaling and transposition, the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; the second data block is input into a prediction mode probability model for processing to obtain a probability vector of the current image block.

[0018] In a possible design, when the size of the first data block is not the target size, the size transformation operation is performed on the first data block to obtain the second data block, which may include four methods: (1) when the size of the first data block is not the target size, and the ratio of the first side length to the second side length of the first data block is equal to the ratio of the first side length to the second side length of the target size, the first data block is scaled proportionally in both horizontal and vertical directions to obtain the second data block; (2) when the size of the first data block is not the target size, and the ratio of the first side length to the second side length of the first data block is equal to the ratio of the second side length to the first side length of the target size, the first data block is transposed and scaled proportionally in both horizontal and vertical directions to obtain the second data block; (3) when the size of the first data block is not the target size, and the first variable length and the second side length of the first data block are respectively equal to the second side length and the first side length of the target size, the first data block is transposed to obtain the second data block; (4) when the size of the first data block is not the target size, the first data block is scaled to obtain the second data block.

[0019] It should be understood that the target sizes used in the above four methods in the embodiments of the present application may be different, and the target size may be one or more. At the same time, the number of target sizes is less than the number of first data block sizes; the above first side length and the second side length may be width and height, respectively, or height and width, respectively; in addition, in the above second method, there is no order of precedence for the transposition and scaling operations performed on the first data block. The second data block obtained in the two methods of first transposing and then scaling and first scaling and then transposing is the same.

[0020] It can be seen that in the above embodiment, since the first data block has various sizes, the second data block is obtained by performing a size transformation operation on the first data block, which can effectively reduce the number of types of second data block sizes, thereby reducing the number of neural network models corresponding to second data blocks of different sizes, thereby saving storage space for the neural network model and improving encoding performance. In addition, for the first data block with a simpler texture and a larger size, after being reduced to a second data block of a smaller size through a size transformation operation, the computational complexity of subsequently calculating the probability vector using the neural network model can be effectively reduced, thereby improving encoding efficiency.

[0021] In a possible design, obtaining the probability vector of the current image block according to the first data block and the prediction mode probability model includes: when the size of the first data block is the target size, inputting the first data block into the prediction mode probability model for processing to obtain the probability vector of the current image block, or;

[0022] When the size of the first data block is not the target size, a size transformation operation is performed on the first data block according to a size relationship between a first side length and a second side length of the first data block, and / or a horizontal gradient and a vertical gradient of the current image block to obtain a second data block, the size of the second data block being the target size, the size transformation operation including at least one of scaling and transposition, the scaling including horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; the second data block is input into a prediction mode probability model for processing to obtain a probability vector of the current image block.

[0023] It can be seen that in the embodiment of the present application, the process of determining the second data block through the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block, fully utilizes the current image block's own information, enhances the correlation between the second data block and the current image block, and thus improves the accuracy of the probability vector subsequently obtained based on the second data block, thereby improving the accuracy of the data in the code stream and improving the encoding performance.

[0024] In a possible design, the size transformation operation includes scaling, and the size transformation operation is performed on the first data block according to the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, including: when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, the first side length and the second side length of the second data block are 4M / N, 4 respectively; when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, scaling the first data block to obtain the second data block. The first data block is scaled to obtain a second data block, and the first side length and the second side length of the second data block are 8M / N, 8 respectively; when the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, the first data block is scaled to obtain the second data block, and the first side length and the second side length of the second data block are 4, 4N / M respectively; when the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, the first data block is scaled to obtain the second data block, and the first side length and the second side length of the second data block are 8, 8N / M respectively; wherein M and N are positive integers.

[0025] It can be seen that in the embodiment of the present application, the process of determining the second data block based on the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block, makes full use of the current image's own information, thereby improving the accuracy of the probability vector subsequently obtained based on the second data block, thereby improving the accuracy of the data in the code stream; at the same time, the number of types of second data block sizes can be reduced through scaling, thereby also reducing the number of neural network models corresponding to second data blocks of different sizes, saving storage space for storing neural network models, and improving encoding performance.

[0026] In a possible design, the size transformation operation includes at least one of scaling and transposition, and the size transformation operation is performed on the first data block according to the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, including: when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, the first side length and the second side length of the second data block are 4M / N, 4 respectively; when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, scaling the first data block to obtain the second data block, the first side length and the second side length of the second data block are 8M / N, 8 respectively;

[0027] When the first side length M of the first data block is less than the second side length N of the first data block, the following operations are performed on the first data block to obtain a second data block: when the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is less than a preset threshold, the first data block is transposed and scaled to obtain a second data block, and the first side length and the second side length of the second data block are 4N / M, 4 respectively; when the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is greater than or equal to a preset threshold, the first data block is transposed and scaled to obtain a second data block, and the first side length and the second side length of the second data block are 8N / M, 8 respectively; wherein M and N are positive integers.

[0028] It can be seen that in the embodiment of the present application, the process of determining the second data block based on the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block, makes full use of the current image's own information to determine the size of the second data block, enhances the correlation between the second data block and the current image block, thereby improving the accuracy of the subsequent probability vector obtained based on the second data block, thereby improving the accuracy of the data in the code stream; at the same time, by performing scaling and transposition operations, the number of types of second data block sizes can be reduced, and the number of neural network models corresponding to second data blocks of different sizes will also be reduced, thereby saving storage space for storing neural network models and improving encoding performance.

[0029] In a possible design, the target intra-frame prediction mode is encoded into the bitstream according to the target intra-frame prediction mode and the probability vector, including: when the size transformation operation does not include transposition, a first reference value is determined according to a first identifier and a probability vector of the target intra-frame prediction mode, and the first reference value is encoded to obtain a bitstream corresponding to the target intra-frame prediction mode; when the size transformation operation includes transposition, and the target intra-frame prediction mode is an angle prediction mode, a second identifier is determined according to a preset constant and the first identifier of the target intra-frame prediction mode, and a second reference value is determined according to the second identifier and the probability vector; the second reference value is encoded to obtain a bitstream corresponding to the target intra-frame prediction mode; when the size transformation operation includes transposition, and the target intra-frame prediction mode is a non-angle prediction mode, a first reference value is determined according to the first identifier and the probability vector of the target intra-frame prediction mode; the first reference value is encoded to obtain a bitstream corresponding to the target intra-frame prediction mode.

[0030] It should be understood that in the embodiment of the present application, the preset constant is equal to the number of intra-frame prediction modes in the versatile video coding VVC technology, that is, 67; the first identifier can be the mode value corresponding to the target intra-frame prediction mode; the angle prediction mode is 65 intra-frame prediction modes with mode values ​​of 2-67 in VVC, and the non-angle prediction mode is a planar prediction mode with a mode value of 0 and a direct current DC prediction mode with a mode value of 1.

[0031] It can be seen that in an embodiment of the present application, when the size transformation operation includes transposition and the target intra-frame prediction mode is an angle prediction mode, the first identifier of the target intra-frame prediction mode is processed by a preset constant to obtain a second identifier, and then the second reference value is determined according to the second identifier, ensuring that when the first data block is transposed, the corresponding correct second reference value is encoded in the bitstream, thereby ensuring the correctness of the encoding process.

[0032] In a second aspect, the present application provides an intra-frame prediction mode encoding method, the method comprising: obtaining surrounding image blocks of a current image block; splicing or cascading the surrounding image blocks to obtain a first data block; performing a size transformation operation on the first data block to obtain a second data block, the size transformation operation comprising at least one of scaling and transposition, the scaling comprising horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; inputting the second data block into a prediction mode probability model for processing to obtain a probability vector of the current image block, wherein multiple elements in the probability vector correspond one-to-one to multiple intra-frame prediction modes, and any element in the probability vector is used to characterize the probability of using an intra-frame prediction mode corresponding to any element when predicting the current image block; determining a target intra-frame prediction mode for the current image block; and encoding the target intra-frame prediction mode into a bitstream according to the target intra-frame prediction mode and the probability vector.

[0033] It should be understood that the prediction mode probability model in the embodiment of the present application can be a neural network model, and second data blocks of different sizes correspond to different neural network models.

[0034] It can be seen that in the embodiment of the present application, by scaling the first data block to obtain a second data block of smaller size, the computational complexity of subsequently using the second data block to participate in the probability vector calculation can be effectively reduced, thereby improving the encoding efficiency; at the same time, through the size transformation operation, the number of second data block size types can be reduced, thereby reducing the number of neural network models corresponding to second data blocks of different sizes, thereby saving storage space for the neural network model and improving encoding performance.

[0035] In one possible design, the above-mentioned resizing operation is performed on the first data block to obtain the second data block, including: when the target intra-frame prediction mode is a non-matrix weighted intra-frame prediction MIP mode, and the size of the first data block is not the target size, performing a resizing operation on the first data block to obtain a second data block, and the size of the second data block is equal to the target size.

[0036] It should be understood that the target size can be set according to the actual scenario, and the number of target sizes is smaller than the number of first data block sizes.

[0037] It can be seen that in the embodiment of the present application, since there are multiple sizes of the first data block, the second data block is obtained by performing a size transformation operation on the first data block, which can effectively reduce the number of types of second data block sizes, thereby reducing the number of neural network models corresponding to second data blocks of different sizes when the target intra-frame prediction mode is a non-matrix weighted intra-frame prediction MIP mode, thereby saving storage space for the neural network model and improving encoding performance; at the same time, by scaling the first data block to obtain a smaller second data block, when the target intra-frame prediction mode is a non-matrix weighted intra-frame prediction MIP mode, the computational complexity of subsequently using the second data block to participate in the probability vector calculation can be effectively reduced, thereby improving encoding efficiency.

[0038] In one possible design, the above-mentioned resizing operation is performed on the first data block to obtain the second data block, including: when the target intra-frame prediction mode is a non-matrix weighted intra-frame prediction MIP mode, and the size of the first data block is not the target size, according to the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and vertical gradient of the current image block, performing a resizing operation on the first data block to obtain the second data block, and the size of the second data block is equal to the target size.

[0039] It should be understood that the first side length and the second side length mentioned above may be the width and the height respectively, or the height and the width respectively.

[0040] It can be seen that in the embodiment of the present application, the process of determining the second data block through the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block, fully utilizes the current image block's own information, enhances the correlation between the second data block and the current image block, and thus improves the accuracy of the probability vector subsequently obtained based on the second data block, thereby improving the accuracy of the data in the code stream and improving the encoding performance.

[0041] In one possible design, when the target intra prediction mode is a non-matrix weighted intra prediction MIP mode and the size of the first data block is equal to the target size, the first data block is input into the prediction mode probability model for processing to obtain a probability vector of the current image block.

[0042] It can be seen that in an embodiment of the present application, when the size of the first data block is equal to the set target size, no size transformation operation is performed, and the first data block is directly used to participate in the generation of the probability vector, which simplifies the process of encoding the target intra-frame prediction mode and improves the encoding efficiency.

[0043] In a possible design, the size transformation operation includes scaling, and the size transformation operation is performed on the first data block according to the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, including: when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, the first side length and the second side length of the second data block are 4M / N, 4 respectively; when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, scaling the first data block to obtain the second data block. The first data block is scaled to obtain a second data block, and the first side length and the second side length of the second data block are 8M / N, 8 respectively; when the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, the first data block is scaled to obtain the second data block, and the first side length and the second side length of the second data block are 4, 4N / M respectively; when the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, the first data block is scaled to obtain the second data block, and the first side length and the second side length of the second data block are 8, 8N / M respectively; wherein M and N are positive integers.

[0044] It can be seen that in the embodiment of the present application, the process of determining the second data block based on the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block, makes full use of the current image's own information, thereby improving the accuracy of the probability vector subsequently obtained based on the second data block, thereby improving the accuracy of the data in the code stream; at the same time, the number of types of second data block sizes can be reduced through scaling, thereby also reducing the number of neural network models corresponding to second data blocks of different sizes, saving storage space for storing neural network models, and improving encoding performance.

[0045] In a possible design, the size transformation operation includes at least one of scaling and transposition; the size transformation operation is performed on the first data block according to the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, including: when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, the first side length and the second side length of the second data block are 4M / N, 4 respectively; when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, scaling the first data block to obtain the second data block, the first side length and the second side length of the second data block are 8M / N, 8 respectively;

[0046] When the first side length M of the first data block is less than the second side length N of the first data block, the following operations are performed on the first data block to obtain a second data block: when the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is less than a preset threshold, the first data block is transposed and scaled to obtain a second data block, and the first side length and the second side length of the second data block are 4N / M, 4 respectively; when the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is greater than or equal to a preset threshold, the first data block is transposed and scaled to obtain a second data block, and the first side length and the second side length of the second data block are 8N / M, 8 respectively; wherein M and N are positive integers.

[0047] It can be seen that in the embodiment of the present application, the process of determining the second data block based on the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block, makes full use of the current image's own information to determine the size of the second data block, enhances the correlation between the second data block and the current image block, thereby improving the accuracy of the subsequent probability vector obtained based on the second data block, thereby improving the accuracy of the data in the code stream; at the same time, by performing scaling and transposition operations, the number of types of second data block sizes can be reduced, and the number of neural network models corresponding to second data blocks of different sizes will also be reduced, thereby saving storage space for storing neural network models and improving encoding performance.

[0048] In a possible design, the target intra-frame prediction mode is encoded into the bitstream according to the target intra-frame prediction mode and the probability vector, including: when the size transformation operation does not include transposition, a first reference value is determined according to a first identifier and a probability vector of the target intra-frame prediction mode; the first reference value is encoded to obtain a bitstream corresponding to the target intra-frame prediction mode; when the size transformation operation includes transposition, and the target intra-frame prediction mode is an angle prediction mode, a second identifier is determined according to a preset constant and the first identifier of the target intra-frame prediction mode, and a second reference value is determined according to the second identifier and the probability vector; the second reference value is encoded to obtain a bitstream corresponding to the target intra-frame prediction mode; when the size transformation operation includes transposition, and the target intra-frame prediction mode is a non-angle prediction mode, a first reference value is determined according to the first identifier and the probability vector of the target intra-frame prediction mode; the first reference value is encoded to obtain a bitstream corresponding to the target intra-frame prediction mode.

[0049] It should be understood that in the embodiment of the present application, the preset constant is equal to the number of intra-frame prediction modes in the versatile video coding VVC technology, that is, 67; the first identifier can be the mode value corresponding to the target intra-frame prediction mode; the angle prediction mode is 65 intra-frame prediction modes with mode values ​​of 2-67 in VVC, and the non-angle prediction mode is a planar prediction mode with a mode value of 0 and a direct current DC prediction mode with a mode value of 1.

[0050] It can be seen that in an embodiment of the present application, when the size transformation operation includes transposition and the target intra-frame prediction mode is an angle prediction mode, the first identifier of the target intra-frame prediction mode is processed by a preset constant to obtain a second identifier, and then the second reference value is determined according to the second identifier, ensuring that when the first data block is transposed, the corresponding correct second reference value is encoded in the bitstream, thereby ensuring the correctness of the encoding process.

[0051] In a third aspect, the present application provides a method for decoding an intra-frame prediction mode, the method comprising: obtaining a code stream corresponding to a current image block, and at least two of a reconstructed block, a prediction block, and a residual block of surrounding image blocks of the current image block; obtaining a probability vector of the current image block based on at least two of the reconstructed block, the prediction block, and the residual block of the surrounding image blocks and a prediction mode probability model, wherein multiple elements in the probability vector correspond one-to-one to multiple intra-frame prediction modes, and any element in the probability vector is used to characterize the probability of using an intra-frame prediction mode corresponding to any element when predicting the current image block; determining a target intra-frame prediction mode of the current image block based on the code stream and the probability vector, and determining a prediction block of the current image block based on the target intra-frame prediction mode.

[0052] It can be seen that in the embodiment of the present application, at least two of the reconstructed blocks, prediction blocks and residual blocks of the three surrounding image blocks of the current image block are obtained. Since the other one can be calculated from two of the reconstructed blocks, prediction blocks and residual blocks of each surrounding image block, the embodiment of the present application can make full use of the relevant information of the surrounding image blocks, and then the probability vector of the current image block generated according to at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks is more accurate, and then the target intra-frame prediction mode of the current image block can be accurately determined according to the probability vector and the code stream, thereby improving the decoding performance.

[0053] In one possible design, the above-mentioned obtaining the probability vector of the current image block based on at least two of the reconstructed block, the prediction block and the residual block of the surrounding image blocks and the prediction mode probability model includes: splicing or cascading at least two of the reconstructed block, the prediction block and the residual block of the surrounding image blocks to obtain a first data block; and obtaining the probability vector of the current image block based on the first data block and the prediction mode probability model.

[0054] It should be understood that the prediction mode probability model in the embodiment of the present application can be a neural network model, and second data blocks of different sizes correspond to different neural network models.

[0055] It can be seen that in the embodiment of the present application, since at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks can fully characterize the relevant information of the surrounding image blocks, the first data block is obtained by splicing or cascading at least two of the reconstructed blocks, prediction blocks and residual blocks of the three surrounding image blocks, and thus the first data block can fully reflect the relevant information of the current image block, and the probability vector of the current image block obtained according to the first data block is also more accurate, and then the target intra-frame prediction mode of the current image block can be accurately determined according to the probability vector and the code stream, thereby improving the decoding performance.

[0056] In one possible design, the above-mentioned method of obtaining the probability vector of the current image block based on the first data block and the prediction mode probability model includes: when the size of the first data block is the target size, inputting the first data block into the prediction mode probability model for processing to obtain the probability vector of the current image block, or; when the size of the first data block is not the target size, performing a size transformation operation on the first data block to obtain a second data block, the size of the second data block is equal to the target size, the size transformation operation includes at least one of scaling and transposition, the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; inputting the second data block into the prediction mode probability model for processing to obtain the probability vector of the current image block.

[0057] It should be understood that the specific implementation process of each step in the embodiment of the present application is the same as the corresponding step in the first aspect, and will not be repeated here.

[0058] It can be seen that in the embodiment of the present application, since there are multiple sizes of the first data block, the second data block is obtained by performing a size transformation operation on the first data block, which can effectively reduce the number of types of second data block sizes, thereby reducing the number of neural network models corresponding to second data blocks of different sizes, thereby saving storage space for the neural network model and improving decoding performance. In addition, for the first data block with a simpler texture and a larger size, after being reduced to a second data block of a smaller size through a size transformation operation, the computational complexity of subsequently calculating the probability vector using the neural network model can be effectively reduced, thereby improving decoding efficiency.

[0059] In a possible design, obtaining the probability vector of the current image block according to the first data block and the prediction mode probability model includes: when the size of the first data block is the target size, inputting the first data block into the prediction mode probability model for processing to obtain the probability vector of the current image block, or;

[0060] When the size of the first data block is not the target size, a size transformation operation is performed on the first data block according to a size relationship between a first side length and a second side length of the first data block, and / or a horizontal gradient and a vertical gradient of the current image block to obtain a second data block, wherein the first side length and the second side length are lengths of two mutually perpendicular sides in the first data block, and the size of the second data block is the target size, and the size transformation operation includes at least one of scaling and transposition, and the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; and the second data block is input into a prediction mode probability model for processing to obtain a probability vector of the current image block.

[0061] It can be seen that in the embodiment of the present application, the process of determining the second data block through the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block, fully utilizes the current image block's own information, enhances the correlation between the second data block and the current image block, and thus improves the accuracy of the probability vector subsequently obtained based on the second data block, so that the target intra-frame prediction mode of the current image block can be accurately determined based on the probability vector and the code stream, thereby improving the decoding performance.

[0062] In a possible design, the size transformation operation includes scaling, and the size transformation operation is performed on the first data block according to the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, including: when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, the first side length and the second side length of the second data block are 4M / N, 4 respectively; when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, scaling the first data block to obtain the second data block. The first data block is scaled to obtain a second data block, and the first side length and the second side length of the second data block are 8M / N, 8 respectively; when the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, the first data block is scaled to obtain the second data block, and the first side length and the second side length of the second data block are 4, 4N / M respectively; when the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, the first data block is scaled to obtain the second data block, and the first side length and the second side length of the second data block are 8, 8N / M respectively; wherein M and N are positive integers.

[0063] It can be seen that in the embodiment of the present application, the process of determining the second data block based on the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block, makes full use of the current image's own information, thereby improving the accuracy of the probability vector subsequently obtained based on the second data block, so that the target intra-frame prediction mode of the current image block subsequently determined based on the probability vector and the code stream is more accurate; at the same time, the number of types of second data block sizes can be reduced through scaling, thereby also reducing the number of neural network models corresponding to second data blocks of different sizes, saving storage space for storing neural network models, and improving decoding performance.

[0064] In a possible design, the size transformation operation includes at least one of scaling and transposition, and the size transformation operation is performed on the first data block according to the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, including: when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, the first side length and the second side length of the second data block are 4M / N, 4 respectively; when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, scaling the first data block to obtain the second data block, the first side length and the second side length of the second data block are 8M / N, 8 respectively;

[0065] When the first side length M of the first data block is less than the second side length N of the first data block, the following operations are performed on the first data block to obtain a second data block: when the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is less than a preset threshold, the first data block is transposed and scaled to obtain a second data block, and the first side length and the second side length of the second data block are 4N / M, 4 respectively; when the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is greater than or equal to a preset threshold, the first data block is transposed and scaled to obtain a second data block, and the first side length and the second side length of the second data block are 8N / M, 8 respectively; wherein M and N are positive integers.

[0066] It can be seen that in the above embodiment, the process of determining the second data block based on the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block, makes full use of the current image's own information to determine the size of the second data block, enhances the correlation between the second data block and the current image block, thereby improving the accuracy of the subsequent probability vector obtained based on the second data block, thereby improving the accuracy of the data in the code stream; at the same time, by performing scaling and transposition operations, the number of types of second data block sizes can be reduced, and the number of neural network models corresponding to second data blocks of different sizes will also be reduced, thereby saving storage space for storing neural network models and improving encoding performance.

[0067] In one possible design, the above-mentioned determining the target intra-frame prediction mode of the current image block according to the bitstream and the probability vector includes: decoding the bitstream to obtain a target reference value; when the size transformation operation does not include transposition, determining the target intra-frame prediction mode according to the target reference value and the probability vector; when the size transformation operation includes transposition, determining the first identifier according to the target reference value and the probability vector; when the intra-frame prediction mode corresponding to the first identifier is an angle prediction mode, determining the target intra-frame prediction mode according to a preset constant and the first identifier; when the intra-frame prediction mode corresponding to the first identifier is a non-angle prediction mode, determining the target intra-frame prediction mode according to the target reference value and the probability vector.

[0068] It can be seen that in an embodiment of the present application, when the size transformation operation performed on the first data block includes transposition, and the intra-frame prediction mode corresponding to the above-mentioned first identifier is an angle prediction mode, it indicates that the above-mentioned first identifier has been transposed during the encoding process. At this time, the first identifier is transposed according to a preset constant to obtain a second identifier. The intra-frame prediction mode corresponding to the second identifier is the target intra-frame prediction mode of the current image block. The above method can ensure that after the first data block is transposed, the correct target intra-frame prediction mode is obtained during the decoding process, and then the real prediction block of the current image block is obtained according to the target intra-frame prediction mode.

[0069] In a fourth aspect, the present application provides a method for decoding an intra-frame prediction mode, the method comprising: obtaining a code stream corresponding to a current image block, and surrounding image blocks of the current image block; splicing or cascading the surrounding image blocks to obtain a first data block; performing a size transformation operation on the first data block to obtain a second data block, the size transformation operation comprising at least one of scaling and transposition, the scaling comprising horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; inputting the second data block into a prediction mode probability model for processing to obtain a probability vector of the current image block, wherein multiple elements in the probability vector correspond one-to-one to multiple intra-frame prediction modes, and any element in the probability vector is used to characterize the probability of using an intra-frame prediction mode corresponding to any element when predicting the current image block; determining a target intra-frame prediction mode of the current image block according to the code stream and the probability vector, and determining a prediction block of the current image block according to the target intra-frame prediction mode.

[0070] It can be seen that in the embodiment of the present application, for a first data block with a simpler texture and a larger size, after being reduced to a second data block of a smaller size through a size transformation operation, the computational complexity of subsequent calculation of the probability vector using the neural network model can be effectively reduced, thereby improving the decoding efficiency; in addition, since there are multiple sizes of the first data block, by performing a size transformation operation on the first data block to obtain the second data block, the number of types of second data block sizes can be effectively reduced, thereby reducing the number of neural network models corresponding to second data blocks of different sizes, thereby saving storage space for the neural network model and improving decoding performance.

[0071] In one possible design, the above-mentioned resizing operation on the first data block to obtain the second data block includes: when the size of the first data block is not the target size, performing a resizing operation on the first data block to obtain the second data block, and the size of the second data block is equal to the target size.

[0072] It can be seen that in the embodiment of the present application, the target size can be set according to the actual application scenario, and changing the size of the first data block to the target size through the size transformation operation can effectively meet the actual scenario requirements; at the same time, since the number of target size types is smaller than the number of first data block size types, the size transformation operation can effectively reduce the number of second data block size types and the number of neural network models corresponding to second data blocks of different sizes, thereby saving storage space of the neural network model and improving decoding performance.

[0073] In one possible design, the above-mentioned resizing operation is performed on the first data block to obtain the second data block, including: when the size of the first data block is not the target size, according to the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and vertical gradient of the current image block, the first data block is resized to obtain the second data block, and the size of the second data block is equal to the target size.

[0074] It can be seen that in the embodiment of the present application, the process of determining the second data block through the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block, fully utilizes the current image block's own information, enhances the correlation between the second data block and the current image block, and thus improves the accuracy of the probability vector subsequently obtained based on the second data block, so that the target intra-frame prediction mode of the current image block can be accurately determined based on the probability vector and the code stream, thereby improving the decoding performance.

[0075] In a possible design, the method further includes: when the size of the first data block is equal to the target size, inputting the first data block into the prediction mode probability model for processing to obtain a probability vector of the current image block.

[0076] It can be seen that in an embodiment of the present application, when the size of the first data block is equal to the set target size, no size transformation operation is performed, and the first data block is directly used to participate in the generation of the probability vector, which simplifies the generation process of the probability vector of the current image block and improves the decoding efficiency.

[0077] In a possible design, the size transformation operation includes scaling, and the size transformation operation is performed on the first data block according to the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, including: when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, the first side length and the second side length of the second data block are 4M / N, 4 respectively; when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, scaling the first data block to obtain the second data block. The first data block is scaled to obtain a second data block, and the first side length and the second side length of the second data block are 8M / N, 8 respectively; when the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, the first data block is scaled to obtain the second data block, and the first side length and the second side length of the second data block are 4, 4N / M respectively; when the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, the first data block is scaled to obtain the second data block, and the first side length and the second side length of the second data block are 8, 8N / M respectively; wherein M and N are positive integers.

[0078] It should be understood that the specific process and beneficial effects of scaling the first data block to obtain the second data block are the same as the corresponding process in the third aspect, and will not be repeated here.

[0079] In a possible design, the size transformation operation includes at least one of scaling and transposition, and the size transformation operation is performed on the first data block according to the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, including: when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, the first side length and the second side length of the second data block are 4M / N, 4 respectively; when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, scaling the first data block to obtain the second data block, the first side length and the second side length of the second data block are 8M / N, 8 respectively;

[0080] When the first side length M of the first data block is less than the second side length N of the first data block, the following operations are performed on the first data block to obtain a second data block: when the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is less than a preset threshold, the first data block is transposed and scaled to obtain a second data block, and the first side length and the second side length of the second data block are 4N / M, 4 respectively; when the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is greater than or equal to a preset threshold, the first data block is transposed and scaled to obtain a second data block, and the first side length and the second side length of the second data block are 8N / M, 8 respectively; wherein M and N are positive integers.

[0081] It should be understood that the specific process and beneficial effects of scaling the first data block to obtain the second data block are the same as the corresponding process in the third aspect, and will not be repeated here.

[0082] In one possible design, the above-mentioned determining the target intra-frame prediction mode of the current image block according to the bitstream and the probability vector includes: decoding the bitstream to obtain a target reference value; when the size transformation operation does not include transposition, determining the target intra-frame prediction mode according to the target reference value and the probability vector; when the size transformation operation includes transposition, determining the first identifier according to the target reference value and the probability vector; when the intra-frame prediction mode corresponding to the first identifier is an angle prediction mode, determining the target intra-frame prediction mode according to a preset constant and the first identifier; when the intra-frame prediction mode corresponding to the first identifier is a non-angle prediction mode, determining the target intra-frame prediction mode according to the target reference value and the probability vector.

[0083] It should be understood that the above process and beneficial effects of determining the target intra-frame prediction mode of the current image block based on the bit stream and the probability vector are the same as the corresponding process in the third aspect, and will not be repeated here.

[0084] In a fifth aspect, the present application provides a coding device, and its beneficial effects can be found in the description of the first aspect, which will not be repeated here. The coding device has the function of implementing the behavior in the method example of the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In a possible design, the coding device includes: an acquisition unit, which is used to acquire at least two of the reconstruction blocks, prediction blocks and residual blocks of the surrounding image blocks of the current image block; a processing unit, which is used to obtain the probability vector of the current image block according to the reconstruction blocks, prediction blocks and residual blocks of the surrounding image blocks and the prediction mode probability model, wherein multiple elements in the probability vector correspond to multiple intra-frame prediction modes one by one, and any element in the probability vector is used to characterize the probability of using the intra-frame prediction mode corresponding to any element when predicting the current image block; a determination unit, which is used to determine the target intra-frame prediction mode of the current image block; and a coding unit, which is used to encode the target intra-frame prediction mode into the bitstream according to the target intra-frame prediction mode and the probability vector. These modules can perform the corresponding functions in the method example of the first aspect, and refer to the detailed description in the method example for details, which will not be repeated here.

[0085] In a sixth aspect, the present application provides a coding device, and its beneficial effects can be found in the description of the second aspect, which will not be repeated here. The coding device has the function of implementing the behavior in the method example of the second aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the encoding device includes: an acquisition unit for acquiring surrounding image blocks of the current image block; a processing unit for splicing or cascading the surrounding image blocks to obtain a first data block; performing a size transformation operation on the first data block to obtain a second data block, the size transformation operation includes at least one of scaling and transposition, and the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; and for inputting the second data block into a prediction mode probability model for processing to obtain a probability vector of the current image block, wherein multiple elements in the probability vector correspond to multiple intra-frame prediction modes one by one, and any element in the probability vector is used to characterize the probability of using an intra-frame prediction mode corresponding to any element when predicting the current image block; a determination unit for determining a target intra-frame prediction mode of the current image block; and an encoding unit for encoding the target intra-frame prediction mode into a bitstream according to the target intra-frame prediction mode and the probability vector. These modules can perform the corresponding functions in the above-mentioned second aspect method example, and refer to the detailed description in the method example for details, which will not be repeated here.

[0086] In the seventh aspect, the present application provides a decoding device, and the beneficial effects can be found in the description of the third aspect, which will not be repeated here. The decoding device has the function of implementing the behavior in the method example of the third aspect. The function can be implemented by hardware, or by hardware executing the corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the decoding device includes: an acquisition unit, which is used to acquire the code stream corresponding to the current image block, and at least two of the reconstruction blocks, prediction blocks and residual blocks of the surrounding image blocks of the current image block; a processing unit, which is used to obtain the probability vector of the current image block according to at least two of the reconstruction blocks, prediction blocks and residual blocks of the surrounding image blocks and the prediction mode probability model, and multiple elements in the probability vector correspond to multiple intra-frame prediction modes one by one, and any element in the probability vector is used to characterize the probability of using the intra-frame prediction mode corresponding to any element when predicting the current image block; a decoding unit, which is used to determine the target intra-frame prediction mode of the current image block according to the code stream and the probability vector, and determine the prediction block of the current image block according to the target intra-frame prediction mode. These modules can perform the corresponding functions in the method example of the third aspect above, and refer to the detailed description in the method example for details, which will not be repeated here.

[0087] In an eighth aspect, the present application provides a decoding device, and the beneficial effects can be found in the description of the fourth aspect, which will not be repeated here. The decoding device has the function of implementing the behavior in the method example of the fourth aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In a possible design, the decoding device includes: an acquisition unit, which is used to acquire a code stream corresponding to the current image block, and surrounding image blocks of the current image block; a processing unit, which is used to splice or cascade the surrounding image blocks to obtain a first data block; a size transformation operation is performed on the first data block to obtain a second data block, and the size transformation operation includes at least one of scaling and transposition, and the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; and a second data block is input into a prediction mode probability model for processing to obtain a probability vector of the current image block, wherein multiple elements in the probability vector correspond to multiple intra-frame prediction modes one by one, and any element in the probability vector is used to characterize the probability of using an intra-frame prediction mode corresponding to any element when predicting the current image block; a decoding unit, which is used to determine the target intra-frame prediction mode of the current image block according to the code stream and the probability vector, and determine the prediction block of the current image block according to the target intra-frame prediction mode. These modules can perform the corresponding functions in the above-mentioned fourth aspect method example, and refer to the detailed description in the method example for details, which will not be repeated here.

[0088] The method described in the first aspect of the present application can be performed by the device described in the fifth aspect of the present application. Other features and implementations of the method described in the first aspect of the present application directly depend on the functionality and implementation of the device described in the fifth aspect of the present application.

[0089] The method described in the second aspect of the present application can be performed by the device described in the sixth aspect of the present application. Other features and implementations of the method described in the second aspect of the present application directly depend on the functionality and implementation of the device described in the sixth aspect of the present application.

[0090] The method described in the third aspect of the present application can be performed by the device described in the seventh aspect of the present application. Other features and implementations of the method described in the third aspect of the present application directly depend on the functionality and implementation of the device described in the seventh aspect of the present application.

[0091] The method described in the fourth aspect of the present application can be performed by the device described in the eighth aspect of the present application. Other features and implementations of the method described in the fourth aspect of the present application directly depend on the functionality and implementation of the device described in the eighth aspect of the present application.

[0092] In a ninth aspect, the present application provides an encoder (20) comprising a processing circuit, which can be used to execute the method in all or any possible embodiments of the first and second aspects.

[0093] In a tenth aspect, the present application provides a decoder (30) comprising a processing circuit, which can be used to execute the method in all or any possible embodiments of the third aspect and the fourth aspect.

[0094] In the eleventh aspect, the present application provides an encoder, characterized in that it includes: one or more processors; and a non-transitory computer-readable storage medium, which is coupled to the processor and stores a program executed by the processor, wherein when the program is executed by the processor, the encoder executes the method in all or any possible embodiment of the first aspect or the second aspect mentioned above.

[0095] In the twelfth aspect, the present application provides a decoder comprising: one or more processors; a non-transitory computer-readable storage medium, which is coupled to the processor and stores a program executed by the processor, wherein the program, when executed by the processor, causes the decoder to execute the method in all or any possible embodiment of the third aspect or the fourth aspect above.

[0096] In a thirteenth aspect, the present application provides a non-transitory computer-readable storage medium, comprising a program code, which, when executed by a computer device, executes the method in all or any possible embodiments of the above-mentioned first aspect, second aspect, third aspect and fourth aspect.

[0097] In a fourteenth aspect, the present application provides a non-transitory computer-readable storage medium, comprising a bit stream encoded according to a method in all or any possible embodiment of the first aspect or the second aspect above.

[0098] In a fifteenth aspect, the present application provides a computer program product, comprising a program code, which, when running on a computer or a processor, executes the methods in all or any possible embodiments of the above-mentioned first aspect, second aspect, third aspect and fourth aspect.

[0099] One or more embodiments will be described in detail in the drawings and the following description. Other features, objects, and advantages will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] The following is an introduction to the drawings used in the embodiments of the present application.

[0101] Figure 1A A block diagram of an example of a video decoding system for implementing an embodiment of the present application, wherein the system uses a neural network to encode or decode a video image;

[0102] Figure 1B A block diagram of another example of a video decoding system for implementing an embodiment of the present application, wherein the video encoder and / or the video decoder uses a neural network to encode or decode a video image;

[0103] Figure 2 A block diagram of an example of a video encoder for implementing an embodiment of the present application, wherein the video encoder 20 uses a neural network to encode video images;

[0104] Figure 3 A block diagram of an example of a video decoder for implementing an embodiment of the present application, wherein the video decoder 30 uses a neural network to decode video images;

[0105] Figure 4 is a schematic block diagram of a video decoding device for implementing an embodiment of the present application;

[0106] Figure 5 is a schematic block diagram of a video decoding device for implementing an embodiment of the present application;

[0107] Figure 6 A schematic diagram of a neural network structure for obtaining a probability vector of a current image block in an embodiment of the application;

[0108] Figure 7 A flowchart of a method for encoding a target intra-frame prediction mode of a current image block in an embodiment of the application;

[0109] Figure 8 A flowchart of another method for encoding a target intra-frame prediction mode of a current image block in an embodiment of the application;

[0110] Fig. 9 is a schematic diagram of multiple intra-frame prediction modes in an embodiment of the present application;

[0111] Fig.10 A schematic diagram of a splicing method of a current image block in an embodiment of the present application;

[0112] Figure 11-1 to Figure 11-4 Schematic diagram of four position relationships of the current image block in the current frame in an embodiment of the present application;

[0113] Fig.12 This is a flowchart of an encoder encoding relevant syntax elements of an intra-frame prediction mode of a current image block in an embodiment of the present application;

[0114] Fig.13 This is a flowchart of a method for decoding a target intra-frame prediction mode of a current image block in an embodiment of the present application;

[0115] Fig.14 This is a flowchart of another method for decoding a target intra-frame prediction mode of a current image block in an embodiment of the present application;

[0116] Fig.15 is a schematic block diagram of an encoding device in an embodiment of the present application;

[0117] Fig.16 It is a schematic block diagram of a decoding device in an embodiment of the present application. DETAILED DESCRIPTION

[0118] The embodiments of the present application provide an AI-based video image compression technology, and in particular, provide a coding and decoding technology based on a neural network intra-frame prediction mode to improve a traditional hybrid video coding and decoding system.

[0119] Video coding generally refers to processing a sequence of images that form a video or video sequence. In the field of video coding, the terms "picture", "frame" or "image" can be used as synonyms. Video coding (or commonly referred to as coding) includes two parts: video encoding and video decoding. Video coding is performed on the source side and generally includes processing (e.g., compressing) the original video image to reduce the amount of data required to represent the video image (thereby more efficiently storing and / or transmitting). Video decoding is performed on the destination side and generally includes inverse processing relative to the encoder to reconstruct the video image. The "encoding" of the video image (or commonly referred to as the image) involved in the embodiment should be understood as the "encoding" or "decoding" of the video image or video sequence. The encoding part and the decoding part are also collectively referred to as codec (encoding and decoding, CODEC).

[0120] In the case of lossless video coding, the original video image can be reconstructed, that is, the reconstructed video image has the same quality as the original video image (assuming no transmission loss or other data loss during storage or transmission). In the case of lossy video coding, further compression is performed by quantization, etc. to reduce the amount of data required to represent the video image, but the decoder side cannot fully reconstruct the video image, that is, the quality of the reconstructed video image is lower or worse than the quality of the original video image.

[0121] Several video coding standards belong to the category of "lossy hybrid video codecs" (i.e., combining spatial and temporal prediction in the pixel domain with 2D transform coding in the transform domain for applying quantization). Each image in a video sequence is usually divided into a set of non-overlapping blocks, which are usually encoded at the block level. In other words, the encoder usually processes, i.e., encodes the video at the block (video block) level, for example, by generating a prediction block through spatial (intra-frame) prediction and temporal (inter-frame) prediction; subtracting the prediction block from the current block (currently processed / to be processed block) to obtain a residual block; transforming the residual block in the transform domain and quantizing the residual block to reduce the amount of data to be transmitted (compressed), while the decoder side applies the inverse processing part relative to the encoder to the encoded or compressed block to reconstruct the current block for representation. In addition, the encoder needs to repeat the processing steps of the decoder so that the encoder and the decoder generate the same predictions (e.g., intra-frame predictions and inter-frame predictions) and / or reconstructed pixels for processing, i.e., encoding subsequent blocks.

[0122] In the following embodiment of the decoding system 10, the encoder 20 and the decoder 30 are based on Figures 1A to 3 Give a description.

[0123] Figure 1A1 is a schematic block diagram of an exemplary decoding system 10, such as a video decoding system 10 (or simply a decoding system 10) that can utilize the techniques of the present application. The video encoder 20 (or simply encoder 20) and the video decoder 30 (or simply decoder 30) in the video decoding system 10 represent devices that can be used to perform various techniques according to various examples described in the present application.

[0124] like Figure 1A As shown, the decoding system 10 includes a source device 12, which is used to provide encoded image data 21 such as encoded images to a destination device 14 for decoding the encoded image data 21.

[0125] The source device 12 includes an encoder 20 , and may additionally or optionally include an image source 16 , a preprocessor (or a preprocessing unit) 18 such as an image preprocessor, and a communication interface (or a communication unit) 22 .

[0126] The image source 16 may include or may be any type of image capture device for capturing real-world images, etc., and / or any type of image generation device, such as a computer graphics processor for generating computer-animated images or any type of device for acquiring and / or providing real-world images, computer-generated images (e.g., screen content, virtual reality (VR) images, and / or any combination thereof (e.g., augmented reality (AR) images). The image source may be any type of memory or storage that stores any of the above images.

[0127] In order to distinguish the processing performed by the pre-processor (or pre-processing unit) 18 , the image (or image data) 17 may also be referred to as a raw image (or raw image data) 17 .

[0128] The preprocessor 18 is used to receive (raw) image data 17 and preprocess the image data 17 to obtain a preprocessed image (or preprocessed image data) 19. For example, the preprocessing performed by the preprocessor 18 may include trimming, color format conversion (e.g., from RGB to YCbCr), color adjustment, or denoising. It is understood that the preprocessing unit 18 may be an optional component.

[0129] The video encoder (or encoder) 20 is used to receive the pre-processed image data 19 and provide the encoded image data 21 (hereinafter referred to as Figure 2 etc. for further description).

[0130] The communication interface 22 in the source device 12 can be used to receive the encoded image data 21 and send the encoded image data 21 (or any other processed version) to another device such as the destination device 14 or any other device through the communication channel 13 for storage or direct reconstruction.

[0131] The destination device 14 includes a decoder 30 and, in addition or alternatively, may include a communication interface (or communication unit) 28 , a post-processor (or post-processing unit) 32 , and a display device 34 .

[0132] The communication interface 28 in the destination device 14 is used to receive the encoded image data 21 (or any other processed version) directly from the source device 12 or from any other source device such as a storage device, for example, the storage device is a encoded image data storage device, and provide the encoded image data 21 to the decoder 30.

[0133] The communication interface 22 and the communication interface 28 can be used to send or receive encoded image data (or encoded data) 21 through a direct communication link between the source device 12 and the destination device 14, such as a direct wired or wireless connection, or through any type of network, such as a wired network, a wireless network or any combination thereof, any type of private network and public network or any type of combination thereof.

[0134] For example, the communication interface 22 may be used to encapsulate the encoded image data 21 into a suitable format such as a message, and / or process the encoded image data using any type of transmission coding or processing for transmission over a communication link or network.

[0135] The communication interface 28 corresponds to the communication interface 22 , for example, and can be used to receive transmission data and process the transmission data using any type of corresponding transmission decoding or processing and / or decapsulation to obtain the encoded image data 21 .

[0136] Both the communication interface 22 and the communication interface 28 can be configured as follows Figure 1A A unidirectional communication interface, or a bidirectional communication interface, indicated by an arrow pointing from the source device 12 to the corresponding communication channel 13 of the destination device 14, and can be used to send and receive messages, etc. to establish a connection, confirm and exchange any other information related to the communication link and / or data transmission such as encoded image data transmission, etc.

[0137] The video decoder (or decoder) 30 is used to receive the encoded image data 21 and provide decoded image data (or decoded image data) 31 (hereinafter referred to as Figure 3 etc. for further description).

[0138] The post-processor 32 is used to post-process the decoded image data 31 (also called reconstructed image data) such as the decoded image to obtain the post-processed image data 33 such as the post-processed image. The post-processing performed by the post-processing unit 32 may include, for example, color format conversion (e.g., from YCbCr to RGB), color adjustment, cropping or resampling, or any other processing for generating the decoded image data 31 for display by the display device 34 or the like.

[0139] The display device 34 is used to receive the post-processed image data 33 to display the image to a user or viewer, etc. The display device 34 can be or include any type of display for displaying the reconstructed image, such as an integrated or external display screen or display. For example, the display screen can include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display screen.

[0140] The decoding system 10 also includes a training engine 25, which is used to train the encoder 20 (especially the intra-frame prediction unit 254 in the encoder 20) or the decoder 30 (especially the intra-frame prediction unit 344 in the decoder 30) to process the first data block or the second data block obtained by splicing or concatenating the surrounding image blocks of the current image block to generate a probability vector of the current image block.

[0141] although Figure 1A The source device 12 and the destination device 14 are shown as independent devices, but the device embodiment may also include the source device 12 and the destination device 14 or the functions of the source device 12 and the destination device 14 at the same time, that is, the source device 12 or the corresponding function and the destination device 14 or the corresponding function at the same time. In these embodiments, the source device 12 or the corresponding function and the destination device 14 or the corresponding function can be implemented using the same hardware and / or software or by separate hardware and / or software or any combination thereof.

[0142] According to the description, Figure 1A The presence and (exact) division of different units or functions in the source device 12 and / or the destination device 14 shown may vary depending on the actual device and application, which will be obvious to the skilled person.

[0143] The encoder 20 (eg, video encoder 20) or the decoder 30 (eg, video decoder 30) or both may be configured as follows: Figure 1BThe processing circuitry shown may be implemented, for example, one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, video encoding dedicated processors, or any combination thereof. The encoder 20 may be implemented by processing circuitry 46 to include reference Figure 2 The various modules discussed in encoder 20 and / or any other encoder system or subsystem described herein. Decoder 30 may be implemented by processing circuit 46 to include reference Figure 3 The processing circuit 46 may be used to perform the various operations discussed below. Figure 5 As shown, if part of the technology is implemented in software, the device can store the instructions of the software in a suitable non-transitory computer-readable storage medium and use one or more processors to execute the instructions in hardware to perform the technology of the present invention. One of the video encoder 20 and the video decoder 30 can be integrated into a single device as part of a combined codec (encoder / decoder, CODEC), such as Figure 1B shown.

[0144] Source device 12 and destination device 14 may include any of a variety of devices, including any type of handheld or fixed device, such as a notebook or laptop computer, a mobile phone, a smart phone, a tablet or tablet computer, a camera, a desktop computer, a set-top box, a television, a display device, a digital media player, a video game console, a video streaming device (e.g., a content service server or a content distribution server), a broadcast receiving device, a broadcast transmitting device, etc., and may not use or use any type of operating system. In some cases, source device 12 and destination device 14 may be equipped with components for wireless communication. Therefore, source device 12 and destination device 14 may be wireless communication devices.

[0145] In some cases, Figure 1AThe video decoding system 10 shown is merely exemplary, and the techniques provided herein may be applicable to video encoding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between an encoding device and a decoding device. In other examples, data is retrieved from a local memory, sent over a network, and so on. A video encoding device may encode data and store the data in a memory, and / or a video decoding device may retrieve data from a memory and decode the data. In some examples, encoding and decoding are performed by devices that do not communicate with each other but only encode data to a memory and / or retrieve and decode data from a memory.

[0146] Figure 1B According to an exemplary embodiment, Figure 2 The video encoder 20 and / or Figure 3 30. The video decoding system 40 may include an imaging device 41, a video encoder 20, a video decoder 30 (and / or a video codec implemented by a processing circuit 46), an antenna 42, one or more processors 43, one or more memory storages 44, and / or a display device 45.

[0147] like Figure 1B As shown, imaging device 41, antenna 42, processing circuit 46, video encoder 20, video decoder 30, processor 43, memory storage 44 and / or display device 45 can communicate with each other. In different examples, video decoding system 40 can include only video encoder 20 or only video decoder 30.

[0148] In some instances, antenna 42 can be used to transmit or receive a coded bit stream of video data. In addition, in some instances, display device 45 can be used to present video data. Processing circuit 46 can include application-specific integrated circuit (ASIC) logic, graphics processor, general processor, etc. Video decoding system 40 can also include optional processor 43, which can similarly include application-specific integrated circuit (ASIC) logic, graphics processor, general processor, etc. In addition, memory storage 44 can be any type of memory, such as volatile memory (e.g., static random access memory (SRAM), dynamic random access memory (DRAM), etc.) or non-volatile memory (e.g., flash memory, etc.). In a non-limiting example, memory storage 44 can be implemented by cache memory. In other instances, processing circuit 46 can include memory (e.g., cache, etc.) for implementing image buffer, etc.

[0149] In some examples, video encoder 20 implemented by logic circuits may include an image buffer (e.g., implemented by processing circuit 46 or memory storage 44) and a graphics processing unit (e.g., implemented by processing circuit 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include video encoder 20 implemented by processing circuit 46 to implement reference Figure 2 The various modules discussed in connection with the encoder and / or any other encoder system or subsystem described herein. Logic circuits may be used to perform the various operations discussed herein.

[0150] In some examples, video decoder 30 may be implemented in a similar manner by processing circuitry 46 to implement reference Figure 3 The various modules discussed in connection with the video decoder 30 of the present invention and / or any other decoder system or subsystem described herein. In some examples, the video decoder 30 implemented by logic circuitry may include an image buffer (implemented by processing circuitry 46 or memory storage 44) and a graphics processing unit (implemented, for example, by processing circuitry 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include the video decoder 30 implemented by processing circuitry 46 to implement the video decoder 30 of the present invention. Figure 3 and / or the various modules discussed herein for any other decoder system or subsystem described herein.

[0151] In some examples, antenna 42 may be used to receive an encoded bitstream of video data. As discussed, the encoded bitstream may include data related to the encoded video frames discussed herein, indicators, index values, mode selection data, etc., such as data related to the encoded partitions (e.g., transform coefficients or quantized transform coefficients, (as discussed) optional indicators, and / or data defining the encoded partitions). Video decoding system 40 may also include video decoder 30 coupled to antenna 42 and used to decode the encoded bitstream. Display device 45 is used to present the video frames.

[0152] It should be understood that for the examples described with reference to the video encoder 20 in the embodiments of the present application, the video decoder 30 can be used to perform the reverse process. With respect to the signaling syntax elements, the video decoder 30 can be used to receive and parse such syntax elements and decode the related video data accordingly. In some examples, the video encoder 20 can entropy encode the syntax elements into an encoded video bitstream. In such examples, the video decoder 30 can parse such syntax elements and decode the related video data accordingly.

[0153] For ease of description, the embodiments of the present invention are described with reference to the Versatile video coding (VVC) reference software or the High-Efficiency Video Coding (HEVC) developed by the Joint Collaboration Team on Video Coding (JCT-VC) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Motion Picture Experts Group (MPEG). Those skilled in the art will appreciate that the embodiments of the present invention are not limited to HEVC or VVC.

[0154] Encoders and encoding methods

[0155] Figure 2 FIG. 2 is a schematic block diagram of an example of a video encoder 20 for implementing the technology of the present application. Figure 2In the example of , the video encoder 20 includes an input terminal (or input interface) 201, a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a loop filter 220, a decoded picture buffer (DPB) 230, a mode selection unit 260, an entropy coding unit 270, and an output terminal (or output interface) 272. The mode selection unit 260 may include an inter-frame prediction unit 244, an intra-frame prediction unit 254, and a segmentation unit 262. The inter-frame prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). Figure 2 The illustrated video encoder 20 may also be referred to as a hybrid video encoder or a hybrid video codec-based video encoder.

[0156] See also Figure 2 The intra-frame prediction module includes a trained target model (also called a neural network), which is used to process a first data block or a second data block obtained by splicing or concatenating surrounding image blocks of the current image block to generate a probability vector of the current image block.

[0157] The residual calculation unit 204, the transform processing unit 206, the quantization unit 208 and the mode selection unit 260 constitute the forward signal path of the encoder 20, while the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoded picture buffer (DPB) 230, the inter-frame prediction unit 244 and the intra-frame prediction unit 254 constitute the backward signal path of the encoder, wherein the backward signal path of the encoder 20 corresponds to the signal path of the decoder (see Figure 3 The inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the loop filter 220, the decoded image buffer 230, the inter-frame prediction unit 244 and the intra-frame prediction unit 254 also constitute a “built-in decoder” of the video encoder 20.

[0158] Images and image segmentation (images and blocks)

[0159] The encoder 20 may be configured to receive, via an input 201 or the like, an image (or image data) 17, e.g., an image in a sequence of images forming a video or a video sequence. The received image or image data may also be a pre-processed image (or pre-processed image data) 19. For simplicity, the following description uses image 17. Image 17 may also be referred to as a current image or an image to be encoded (particularly when distinguishing the current image from other images in video encoding, such as previously encoded and / or decoded images in the same video sequence, i.e., a video sequence also including the current image).

[0160] A (digital) image is or can be considered as a two-dimensional array or matrix of pixels with intensity values. The pixels in the array can also be called pixels (pixel or pel) (short for picture elements). The number of pixels in the array or image in the horizontal and vertical directions (or axes) determines the size and / or resolution of the image. In order to represent color, three color components are usually used, that is, the image can be represented as or include three pixel arrays. In the RBG format or color space, the image includes corresponding red, green and blue pixel arrays. However, in video coding, each pixel is usually represented in a brightness / chrominance format or color space, such as YCbCr, including a brightness component indicated by Y (sometimes also represented by L) and two chrominance components represented by Cb and Cr. The brightness (luma) component Y represents the brightness or grayscale level intensity (for example, the two are the same in grayscale images), while the two chrominance (abbreviated as chroma) components Cb and Cr represent the chrominance or color information components. Accordingly, an image in YCbCr format includes a luminance pixel array of luminance pixel values ​​(Y) and two chrominance pixel arrays of chrominance values ​​(Cb and Cr). An image in RGB format can be converted or transformed into YCbCr format and vice versa, a process also referred to as color conversion or transformation. If the image is black and white, the image may include only a luminance pixel array. Accordingly, an image may be, for example, a luminance pixel array in a monochrome format or a luminance pixel array and two corresponding chrominance pixel arrays in 4:2:0, 4:2:2 and 4:4:4 color formats.

[0161] In one embodiment, an embodiment of the video encoder 20 may include an image segmentation unit ( Figure 2), for dividing the image 17 into a plurality of (usually non-overlapping) image blocks 203. These blocks may also be referred to as root blocks, macroblocks (H.264 / AVC) or coding tree blocks (CTB), or coding tree units (CTU) in the H.265 / HEVC and VVC standards. The partitioning unit may be used to use the same block size for all images in a video sequence and a corresponding grid of defined block sizes, or to change the block size between images or subsets of images or groups of images, and to partition each image into corresponding blocks.

[0162] In other embodiments, the video encoder may be configured to directly receive a block 203 of the image 17, for example, one, several or all blocks constituting the image 17. The image block 203 may also be referred to as a current image block or an image block to be encoded.

[0163] Like image 17, image block 203 is also or can be considered as a two-dimensional array or matrix composed of pixels with intensity values ​​(pixel values), but image block 203 is smaller than image 17. In other words, block 203 may include one pixel array (e.g., a brightness array in the case of monochrome image 17 or a brightness array or chrominance array in the case of a color image) or three pixel arrays (e.g., one brightness array and two chrominance arrays in the case of a color image 17) or any other number and / or type of arrays according to the color format adopted. The number of pixels in the horizontal and vertical directions (or axes) of block 203 defines the size of block 203. Accordingly, the block can be an M×N (M columns×N rows) pixel array, or an M×N transform coefficient array, etc.

[0164] In one embodiment, Figure 2 The video encoder 20 shown is used to encode the image 17 block by block, for example, encoding and prediction are performed on each block 203.

[0165] In one embodiment, Figure 2 The illustrated video encoder 20 may also be used to partition and / or encode an image using slices (also referred to as video slices), wherein an image may be partitioned or encoded using one or more slices (usually non-overlapping). Each slice may include one or more blocks (e.g., coding tree units CTUs) or one or more groups of blocks (e.g., coding tiles in the H.265 / HEVC / VVC standard and bricks in the VVC standard).

[0166] In one embodiment, Figure 2The video encoder 20 shown can also be used to partition and / or encode an image using slices / coding block groups (also called video coding block groups) and / or coding blocks (also called video coding blocks), wherein an image can be partitioned or encoded using one or more slices / coding block groups (usually non-overlapping), each slice / coding block group may include one or more blocks (e.g., CTU) or one or more coding blocks, etc., wherein each coding block may be in a rectangular shape, etc., and may include one or more complete or partial blocks (e.g., CTU).

[0167] Residual calculation

[0168] The residual calculation unit 204 is used to calculate the residual block 205 (the prediction block 265 is described in detail later) based on the image block 203 and the prediction block 265 in the following manner: for example, the pixel value of the prediction block 265 is subtracted from the pixel value of the image block 203 pixel by pixel (pixel by pixel) to obtain the residual block 205 in the pixel domain.

[0169] Transform

[0170] The transform processing unit 206 is used to perform discrete cosine transform (DCT) or discrete sine transform (DST) on the pixel values ​​of the residual block 205 to obtain transform coefficients 207 in the transform domain. The transform coefficients 207 may also be called transform residual coefficients, representing the residual block 205 in the transform domain.

[0171] The transform processing unit 206 may be used to apply an integerized approximation of the DCT / DST, such as the transform specified for H.265 / HEVC. This integerized approximation is typically scaled by a factor compared to an orthogonal DCT transform. In order to maintain the norm of the residual block processed by the forward transform and the inverse transform, other scaling factors are used as part of the transform process. The scaling factor is typically selected based on certain constraints, such as whether the scaling factor is a power of 2 for the shift operation, the bit depth of the transform coefficients, a trade-off between accuracy and implementation cost, etc. For example, a specific scaling factor is specified for the inverse transform on the encoder 20 side by the inverse transform processing unit 212 (and for the corresponding inverse transform on the decoder 30 side by, for example, the inverse transform processing unit 312), and correspondingly, a corresponding scaling factor may be specified for the forward transform on the encoder 20 side by the transform processing unit 206.

[0172] In one embodiment, the video encoder 20 (correspondingly, the transform processing unit 206) may be used to output transform parameters such as one or more transform types, for example, directly output or output after encoding or compression by the entropy coding unit 270, for example, so that the video decoder 30 may receive and use the transform parameters for decoding.

[0173] Quantification

[0174] The quantization unit 208 is used to quantize the transform coefficient 207 by, for example, scalar quantization or vector quantization to obtain a quantized transform coefficient 209 . The quantized transform coefficient 209 may also be referred to as a quantized residual coefficient 209 .

[0175] The quantization process may reduce the bit depth associated with some or all of the transform coefficients 207. For example, an n-bit transform coefficient may be rounded down to an m-bit transform coefficient during quantization, where n is greater than m. The degree of quantization may be modified by adjusting a quantization parameter (QP). For example, for scalar quantization, different degrees of scaling may be applied to achieve finer or coarser quantization. A smaller quantization step size corresponds to a finer quantization, while a larger quantization step size corresponds to a coarser quantization. A suitable quantization step size may be indicated by a quantization parameter (QP). For example, a quantization parameter may be an index of a predefined set of suitable quantization step sizes. For example, a smaller quantization parameter may correspond to fine quantization (a smaller quantization step size), a larger quantization parameter may correspond to coarse quantization (a larger quantization step size), and vice versa. Quantization may include dividing by a quantization step size, while a corresponding or inverse dequantization performed by an inverse quantization unit 210 or the like may include multiplying by a quantization step size. Embodiments according to some standards, such as HEVC, may be used to determine a quantization step size using a quantization parameter. In general, the quantization step size may be calculated using a fixed-point approximation of an equation containing division according to the quantization parameter. Other scaling factors may be introduced for quantization and dequantization to recover the norm of the residual block that may be modified due to the scaling used in the fixed-point approximation of the equations for the quantization step size and the quantization parameter. In one exemplary implementation, the scaling of the inverse transform and dequantization may be combined. Alternatively, a custom quantization table may be used and indicated from the encoder to the decoder in the bitstream, etc. Quantization is a lossy operation, where the larger the quantization step size, the greater the loss.

[0176] In one embodiment, the video encoder 20 (correspondingly, the quantization unit 208) may be configured to output a quantization parameter (QP), for example, directly output or output after being encoded or compressed by the entropy coding unit 270, for example, so that the video decoder 30 may receive and use the quantization parameter for decoding.

[0177] Dequantization

[0178] The inverse quantization unit 210 is used to perform inverse quantization of the quantization unit 208 on the quantized coefficients to obtain dequantized coefficients 211, for example, performing an inverse quantization scheme of the quantization scheme performed by the quantization unit 208 according to or using the same quantization step size as the quantization unit 208. The dequantized coefficients 211 may also be referred to as dequantized residual coefficients 211, corresponding to the transform coefficients 207, but due to the loss caused by quantization, the dequantized coefficients 211 are usually not completely the same as the transform coefficients.

[0179] Inverse Transform

[0180] The inverse transform processing unit 212 is used to perform an inverse transform of the transform performed by the transform processing unit 206, such as an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST), to obtain a reconstructed residual block 213 (or a corresponding dequantized coefficient 213) in the pixel domain. The reconstructed residual block 213 may also be referred to as a transform block 213.

[0181] reconstruction

[0182] The reconstruction unit 214 (e.g., the summer 214) is used to add the transform block 213 (i.e., the reconstructed residual block 213) to the prediction block 265 to obtain the reconstructed block 215 in the pixel domain, for example, by adding the pixel point values ​​of the reconstructed residual block 213 and the pixel point values ​​of the prediction block 265.

[0183] Filtering

[0184] The loop filter unit 220 (or simply "loop filter" 220) is used to filter the reconstruction block 215 to obtain the filter block 221, or is generally used to filter the reconstructed pixel points to obtain the filtered pixel point value. For example, the loop filter unit is used to smoothly perform pixel conversion or improve video quality. The loop filter unit 220 may include one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as an adaptive loop filter (ALF), a noise suppression filter (NSF), or any combination. For example, the loop filter unit 220 may include a deblocking filter, an SAO filter, and an ALF filter. The order of the filtering process may be a deblocking filter, an SAO filter, and an ALF filter. For another example, a process called luma mapping with chromascaling (LMCS) (i.e., an adaptive in-loop shaper) is added. This process is performed before deblocking. For another example, the deblocking filtering process may also be applied to internal sub-block edges, such as affine sub-block edges, ATMVP sub-block edges, sub-block transform (SBT) edges, and intra sub-partition (ISP) edges. Figure 2 2. Although shown as a loop filter in FIG. 2, in other configurations, the loop filter unit 220 may be implemented as a post-loop filter. The filter block 221 may also be referred to as a filter reconstruction block 221.

[0185] In one embodiment, the video encoder 20 (correspondingly, the loop filter unit 220) may be used to output loop filter parameters (e.g., SAO filter parameters, ALF filter parameters, or LMCS parameters), for example, directly or after being entropy encoded by the entropy encoding unit 270, such that the decoder 30 may receive and use the same or different loop filter parameters for decoding.

[0186] Decoded Image Buffer

[0187] The decoded picture buffer (DPB) 230 may be a reference picture memory that stores reference picture data for use by the video encoder 20 when encoding video data. The DPB 230 may be formed by any of a variety of memory devices, such as a dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of storage devices. The decoded picture buffer 230 may be used to store one or more filter blocks 221. The decoded picture buffer 230 may also be used to store other previous filter blocks of the same current picture or a different picture such as a previous reconstructed picture, such as a previously reconstructed and filtered block 221, and may provide a complete previously reconstructed, i.e., decoded picture (and corresponding reference blocks and pixels) and / or a partially reconstructed current picture (and corresponding reference blocks and pixels), such as for inter-frame prediction. The decoded image buffer 230 may also be used to store one or more unfiltered reconstructed blocks 215, or generally to store unfiltered reconstructed pixels, for example, reconstructed blocks 215 that have not been filtered by the loop filter unit 220, or reconstructed blocks or reconstructed pixels that have not undergone any other processing.

[0188] Mode selection (segmentation and prediction)

[0189] The mode selection unit 260 includes a segmentation unit 262, an inter-frame prediction unit 244, and an intra-frame prediction unit 254, which are used to receive or obtain original image data such as an original block 203 (current block 203 of the current image 17) and reconstructed image data from the decoded image buffer 230 or other buffers (e.g., a column buffer, not shown in the figure), such as filtered and / or unfiltered reconstructed pixels or reconstructed blocks of the same (current) image and / or one or more previously decoded images. The reconstructed image data is used as reference image data required for prediction such as inter-frame prediction or intra-frame prediction to obtain a prediction block 265 or a prediction value 265.

[0190] The mode selection unit 260 can be used to determine or select a partition for the current block prediction mode (including non-partition) and prediction mode (such as intra-frame or inter-frame prediction mode), generate a corresponding prediction block 265, so as to calculate the residual block 205 and reconstruct the reconstruction block 215.

[0191] In one embodiment, the mode selection unit 260 may be used to select a segmentation and prediction mode (e.g., from prediction modes supported or available by the mode selection unit 260) that provides the best match or minimum residual (minimum residual means better compression in transmission or storage), or provides minimum signaling overhead (minimum signaling overhead means better compression in transmission or storage), or considers or balances both of the above. The mode selection unit 260 may be used to determine the segmentation and prediction mode based on rate distortion optimization (RDO), i.e., select the prediction mode that provides minimum rate distortion optimization. The terms "best", "lowest", "optimal", etc. herein do not necessarily mean "best", "lowest", "optimal" in general, but may also refer to situations where termination or selection criteria are met, for example, values ​​exceeding or falling below a threshold or other restrictions may result in a "suboptimal selection" but reduce complexity and processing time.

[0192] In other words, the partitioning unit 262 may be used to partition an image in a video sequence into a sequence of coding tree units (CTUs), and the CTU 203 may be further partitioned into smaller block portions or sub-blocks (again forming blocks), for example, by iteratively using quad-tree partitioning (QT), binary-tree partitioning (BT) or triple-tree partitioning (TT) or any combination thereof, and for performing prediction on each of the block portions or sub-blocks, for example, wherein the mode selection comprises selecting a tree structure of the partition block 203 and selecting a prediction mode to be applied to each of the block portions or sub-blocks.

[0193] The segmentation (eg, performed by segmentation unit 262) and prediction processes (eg, performed by inter-prediction unit 244 and intra-prediction unit 254) performed by video encoder 20 are described in detail below.

[0194] segmentation

[0195] The partitioning unit 262 may partition (or divide) a coding tree unit 203 into smaller parts, such as small blocks in a square or rectangular shape. For an image with three pixel arrays, a CTU consists of N×N luminance pixel blocks and two corresponding chrominance pixel blocks. The maximum allowed size of a luminance block in a CTU is specified as 128×128 in the Versatile Video Coding (VVC) standard under development, but may be specified as a value different from 128×128 in the future, such as 256×256. The CTUs of an image may be concentrated / grouped into slices / coding block groups, coding blocks, or bricks. A coding block covers a rectangular area of ​​an image, and a coding block may be divided into one or more bricks. A brick consists of multiple CTU rows within a coding block. A coding block that is not partitioned into multiple bricks may be called a brick. However, a brick is a true subset of a coding block and is therefore not called a coding block. VVC supports two coding block group modes, namely a raster scan slice / coding block group mode and a rectangular slice mode. In raster scan coded block group mode, a slice / coded block group contains a sequence of coded blocks in a raster scan of the coded blocks of an image. In rectangular slice mode, a slice contains multiple bricks of an image, which together form a rectangular area of ​​the image. The bricks within a rectangular slice are arranged in the order of the brick raster scan of the slice. These smaller blocks (also called sub-blocks) can be further split into smaller parts. This is also called tree partitioning or hierarchical tree partitioning, where a root block at root tree level 0 (hierarchy level 0, depth 0), etc., can be recursively split into two or more blocks at the next lower tree level, such as nodes at tree level 1 (hierarchy level 1, depth 1). These blocks can be split into two or more blocks at the next lower level, such as tree level 2 (hierarchy level 2, depth 2), etc., until the partitioning ends (because the end criteria are met, such as reaching the maximum tree depth or minimum block size). Blocks that are not further split are also called leaf blocks or leaf nodes of the tree. A tree divided into two parts is called a binary tree (BT), a tree divided into three parts is called a ternary tree (TT), and a tree divided into four parts is called a quadtree (QT).

[0196] For example, a coding tree unit (CTU) may be or include a CTB of luminance pixels, two corresponding CTBs of chrominance pixels of an image with three pixel arrays, or a CTB of pixels of a monochrome image, or a CTB of pixels of an image encoded using three independent color planes and syntax structures (for encoding pixels). Accordingly, a coding tree block (CTB) may be an N×N pixel block, where N may be set to a certain value so that the component is divided into CTBs, which is segmentation. A coding unit (coding unit, CU) may be or include a coding block of luminance pixels, two corresponding coding blocks of chrominance pixels of an image with three pixel arrays, or a coding block of pixels of a monochrome image, or a coding block of pixels of an image encoded using three independent color planes and syntax structures (for encoding pixels). Accordingly, a coding block (CB) may be an M×N pixel block, where M and N may be set to a certain value so that the CTB is divided into coding blocks, which is segmentation.

[0197] For example, in an embodiment, according to HEVC, a coding tree unit (CTU) may be divided into multiple CUs using a quadtree structure represented as a coding tree. A decision is made at the leaf-CU level whether to use inter-frame (temporal) prediction or intra-frame (spatial) prediction to encode an image area. Each leaf-CU may be further divided into one, two, or four PUs according to the PU partition type. The same prediction process is used within a PU, and relevant information is transmitted to the decoder in units of PUs. After the prediction process is applied according to the PU partition type to obtain the residual block, the leaf-CU may be partitioned into transform units (TUs) according to other quadtree structures similar to the coding tree for the CU.

[0198] For example, in an embodiment, according to the latest video coding standard currently under development (referred to as Versatile Video Coding (VVC), a combined quadtree of nested multi-type trees (e.g., binary trees and ternary trees) is used to partition a segment structure for partitioning a coding tree unit. In the coding tree structure within the coding tree unit, the CU may be a square or a rectangle. For example, a coding tree unit (CTU) is first partitioned by a quadtree structure. The quadtree leaf nodes are further partitioned by a multi-type tree structure. The multi-type tree structure has four partition types: vertical binary tree partition (SPLIT_BT_VER), horizontal binary tree partition (SPLIT_BT_HOR), vertical ternary tree partition The multi-type tree leaf nodes are called coding units (CUs), unless the CU is too large for the maximum transform length, such segmentation is used for prediction and transform processing without any other segmentation. In most cases, this means that the block sizes of CU, PU and TU in the coding block structure of the quadtree nested multi-type tree are the same. This exception occurs when the maximum supported transform length is less than the width or height of the color components of the CU. VVC has established a unique signaling mechanism for the segmentation information in the coding structure with quadtree nested multi-type trees. In the signaling mechanism, the coding The tree unit (CTU) as the root of the quadtree is first split by the quadtree structure. Then each quadtree leaf node (when large enough) is further split into a multi-type tree structure. In the multi-type tree structure, the first flag (mtt_split_cu_flag) indicates whether the node is further split. When the node is further split, the second flag (mtt_split_cu_vertical_flag) is used to indicate the division direction, and then the third flag (mtt_split_cu_binary_flag) is used to indicate whether the division is a binary tree division or a ternary tree division. According to mtt_split_c The values ​​of u_vertical_flag and mtt_split_cu_binary_flag allow the decoder to derive the multi-type tree splitting mode (MttSplitMode) of the CU based on predefined rules or tables. It should be noted that for certain designs, such as the 64×64 luma block and 32×32 chroma pipeline design in the VVC hardware decoder, TT splitting is not allowed when the width or height of the luma coding block is greater than 64. TT splitting is also not allowed when the width or height of the chroma coding block is greater than 32. The pipeline design divides the image into multiple virtual pipeline data units (VPDUs), each VPDU is defined as a non-overlapping unit in the image. In the hardware decoder, consecutive VPDUs are processed simultaneously in multiple pipeline stages. In most pipeline stages, the VPDU size is roughly proportional to the buffer size, so it is necessary to keep the VPDU small.In most hardware decoders, the VPDU size can be set to the maximum transform block (TB) size. However, in VVC, the ternary tree (TT) and binary tree (BT) partitioning may increase the VPDU size.

[0199] In addition, it should be noted that when a part of the tree node block exceeds the bottom or the right boundary of the image, the tree node block is forcibly divided until all the pixels of each coding CU are located within the image boundary.

[0200] For example, the intra sub-partitions (ISP) tool may divide the luma intra prediction block vertically or horizontally into two or four sub-partitions according to the block size.

[0201] In one example, mode select unit 260 of video encoder 20 may be used to perform any combination of the segmentation techniques described above.

[0202] As described above, the video encoder 20 is operable to determine or select the best or optimal prediction mode from a (predetermined) set of prediction modes. The set of prediction modes may include, for example, an intra prediction mode and / or an inter prediction mode.

[0203] Intra-frame prediction

[0204] The intra prediction mode set may include 35 different intra prediction modes, for example, non-directional modes like DC (or mean) mode and plane mode, or directional modes as defined in HEVC, or may include 67 different intra prediction modes, for example, non-directional modes like DC (or mean) mode and plane mode, or directional modes as defined in VVC. For example, several traditional angle intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non-square blocks defined in VVC. For another example, in order to avoid the division operation of DC prediction, only the longer side is used to calculate the average value of non-square blocks. In addition, the intra prediction result of the plane mode can also be modified using the position-dependent intra prediction combination (PDPC) method.

[0205] The intra prediction unit 254 is configured to generate an intra prediction block 265 using reconstructed pixels of adjacent blocks of the same current image according to an intra prediction mode in the intra prediction mode set.

[0206] The intra-frame prediction unit 254 (or generally the mode selection unit 260) is also used to output intra-frame prediction parameters (in this embodiment of the present application, the intra-frame prediction parameters include information about the target intra-frame prediction mode of the current image block and the probability vector of the current image block) in the form of syntax elements 266 to be sent to the entropy coding unit 270 for inclusion in the encoded image data 21, so that the video decoder 30 can perform operations, such as receiving and using the prediction parameters for decoding.

[0207] Inter prediction

[0208] In a possible implementation, the set of inter-prediction modes depends on the available reference picture (i.e., at least part of the previously decoded picture stored in the DBP 230 as mentioned above) and other inter-prediction parameters, for example, on whether to use the entire reference picture or only a part of the reference picture, such as a search window area around the area of ​​the current block, to search for the best matching reference block, and / or on whether to perform pixel interpolation such as half-pixel, quarter-pixel and / or 1 / 16 interpolation, for example.

[0209] In addition to the above prediction modes, skip mode and / or direct mode may also be employed.

[0210] For example, extended merge prediction, the merge candidate list of this mode consists of the following five candidate types in order: spatial MVP from spatially adjacent CUs, temporal MVP from collocated CUs, history-based MVP from FIFO tables, pairwise average MVP, and zero MV. Decoder side motion vector refinement (DMVR) based on bilateral matching can be used to increase the accuracy of the MV of the merge mode. Merge mode with MVD (MMVD) is derived from the merge mode with motion vector difference. The MMVD flag is sent immediately after the skip flag and merge flag are sent to specify whether the CU uses the MMVD mode. The CU-level adaptive motion vector resolution (AMVR) scheme can be used. AMVR supports encoding the MVD of the CU with different precisions. The MVD of the current CU is adaptively selected according to the prediction mode of the current CU. When the CU is encoded in merge mode, the combined inter / intra prediction (CIIP) mode can be applied to the current CU. The CIIP prediction is obtained by weighted averaging the inter-frame and intra-frame prediction signals. For affine motion compensated prediction, the affine motion field of the block is described by the motion information of 2 control points (4 parameters) or 3 control points (6 parameters) motion vectors. Subblock-based temporal motion vector prediction (SbTMVP) is similar to the temporal motion vector prediction (TMVP) in HEVC, but the motion vector of the sub-CU within the current CU is predicted. Bidirectional optical flow (BDOF), formerly known as BIO, is a simplified version that reduces calculations, especially in terms of the number of multiplications and the size of the multipliers. In the triangle partitioning mode, the CU is evenly divided into two triangular parts in two ways: diagonal partitioning and anti-diagonal partitioning. In addition, the bidirectional prediction mode is extended on the basis of simple averaging to support weighted averaging of two prediction signals.

[0211] The inter-frame prediction unit 244 may include a motion estimation (ME) unit and a motion compensation (MC) unit (both in Figure 2). The motion estimation unit may be configured to receive or acquire the image block 203 (the current image block 203 of the current image 17) and the decoded image 231, or at least one or more previously reconstructed blocks, for example, reconstructed blocks of one or more other / different previously decoded images 231, for motion estimation. For example, a video sequence may include the current image and the previously decoded image 231, or in other words, the current image and the previously decoded image 231 may be part of or form a sequence of images forming the video sequence.

[0212] For example, the encoder 20 may be configured to select a reference block from a plurality of reference blocks of the same or different images in a plurality of other images, and provide the reference image (or reference image index) and / or the offset (spatial offset) between the position (x, y coordinates) of the reference block and the position of the current block as an inter-frame prediction parameter to the motion estimation unit. The offset is also referred to as a motion vector (MV).

[0213] The motion compensation unit is used to obtain, for example, receive, inter-frame prediction parameters, and perform inter-frame prediction based on or using the inter-frame prediction parameters to obtain an inter-frame prediction block 246. The motion compensation performed by the motion compensation unit may include extracting or generating a prediction block based on a motion / block vector determined by motion estimation, and may also include performing interpolation with sub-pixel accuracy. Interpolation filtering can generate pixel points of other pixels from pixel points of known pixels, thereby potentially increasing the number of candidate prediction blocks that can be used to encode the image block. Once a motion vector corresponding to a PU of a current image block is received, the motion compensation unit can locate the prediction block pointed to by the motion vector in one of the reference image lists.

[0214] The motion compensation unit may also generate syntax elements associated with blocks and video slices for use by the video decoder 30 when decoding image blocks of the video slice. In addition, or as an alternative to slices and corresponding syntax elements, coding block groups and / or coding blocks and corresponding syntax elements may be generated or used.

[0215] Entropy Coding

[0216] The entropy coding unit 270 is used to apply an entropy coding algorithm or scheme (e.g., a variable length coding (VLC) scheme, a context adaptive VLC scheme (CALVC), an arithmetic coding scheme, a binarization algorithm, a context adaptive binary arithmetic coding (CABAC), a syntax-based context-adaptive binary arithmetic coding (SBAC), a probability interval partitioning entropy (PIPE) coding, or other entropy coding methods or techniques) to the quantized residual coefficients 209, inter-frame prediction parameters, intra-frame prediction parameters, loop filter parameters, and / or other syntax elements to obtain coded image data 21 that can be output in the form of a coded bit stream 21 through an output terminal 272, so that the video decoder 30, etc. can receive and use the parameters for decoding. The coded bit stream 21 can be transmitted to the video decoder 30, or it can be stored in a memory for later transmission or retrieval by the video decoder 30.

[0217] In an embodiment of the present application, the intra-frame prediction parameters include a probability vector of a target intra-frame prediction mode and a current image block. The entropy encoding unit 270 selects a first reference value or a second reference value corresponding to the target intra-frame prediction mode in the probability vector, and then encodes the first reference value or the second reference value into the bitstream. The first reference value or the second reference value is used to indicate the target intra-frame prediction mode of the current image block.

[0218] Other structural variations of the video encoder 20 may be used to encode the video stream. For example, a non-transform based encoder 20 may directly quantize the residual signal without a transform processing unit 206 for certain blocks or frames. In another implementation, the encoder 20 may have a quantization unit 208 and an inverse quantization unit 210 combined into a single unit.

[0219] Decoder and decoding method

[0220] Figure 3 An exemplary video decoder 30 for implementing the technology of the present application is shown. The video decoder 30 is used to receive the encoded image data 21 (e.g., the encoded bit stream 21) encoded by the encoder 20, for example, to obtain a decoded image 331. The encoded image data or bit stream includes information for decoding the encoded image data, such as data representing image blocks of an encoded video slice (and / or a coding block group or coding block) and related syntax elements.

[0221] exist Figure 3 In the example of FIG. 3 , decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., a summer 314), a loop filter 320, a decoded picture buffer (DBP) 330, a mode application unit 360, an inter-frame prediction unit 344, and an intra-frame prediction unit 354. Inter-frame prediction unit 344 may be or include a motion compensation unit. In some examples, video decoder 30 may perform substantially the same as reference Figure 2 The decoding process of the video encoder 100 is the reverse of the encoding process described above.

[0222] See also Figure 3 The intra-frame prediction module includes a trained target model (also called a neural network), which is used to process a first data block or a second data block obtained by splicing or concatenating surrounding image blocks of the current image block to generate a probability vector of the current image block.

[0223] As described for encoder 20, inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, loop filter 220, decoded picture buffer DPB 230, inter prediction unit 344, and intra prediction unit 354 also constitute a “built-in decoder” of video encoder 20. Accordingly, inverse quantization unit 310 may be functionally identical to inverse quantization unit 110, inverse transform processing unit 312 may be functionally identical to inverse transform processing unit 122, reconstruction unit 314 may be functionally identical to reconstruction unit 214, loop filter 320 may be functionally identical to loop filter 220, and decoded picture buffer 330 may be functionally identical to decoded picture buffer 230. Therefore, the explanation of the corresponding units and functions of video encoder 20 applies accordingly to the corresponding units and functions of video decoder 30.

[0224] Entropy decoding

[0225] The entropy decoding unit 304 is used to parse the bit stream 21 (or generally the encoded image data 21) and perform entropy decoding on the encoded image data 21 to obtain the quantization coefficients 309 and / or the decoded encoding parameters ( Figure 3The entropy decoding unit 304 may be used to provide the inter-frame prediction parameters, intra-frame prediction parameters and / or other syntax elements to the mode application unit 360, and to provide other parameters to other units of the decoder 30. The video decoder 30 may receive syntax elements at the video slice and / or video block level. In addition, or as an alternative to slices and corresponding syntax elements, coding block groups and / or coding blocks and corresponding syntax elements may be received or used.

[0226] In the embodiment of the present application, the entropy decoding unit 304 may be used to provide the intra prediction parameters to the mode application unit 360 including a target reference value for indicating a target intra prediction mode for performing intra prediction on the current image block.

[0227] Dequantization

[0228] The inverse quantization unit 310 may be configured to receive a quantization parameter (QP) (or generally information related to inverse quantization) and a quantization coefficient from the encoded image data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304), and inverse quantize the decoded quantization coefficient 309 based on the quantization parameter to obtain an inverse quantization coefficient 311, which may also be referred to as a transform coefficient 311. The inverse quantization process may include using the quantization parameter calculated by the video encoder 20 for each video block in the video slice to determine a degree of quantization, and also determine a degree of inverse quantization that needs to be performed.

[0229] Inverse Transform

[0230] The inverse transform processing unit 312 may be configured to receive the dequantized coefficients 311, also referred to as transform coefficients 311, and apply a transform to the dequantized coefficients 311 to obtain a reconstructed residual block 213 in the pixel domain. The reconstructed residual block 213 may also be referred to as a transform block 313. The transform may be an inverse transform, such as an inverse DCT, an inverse DST, an inverse integer transform, or a conceptually similar inverse transform process. The inverse transform processing unit 312 may also be configured to receive transform parameters or corresponding information from the encoded image data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304) to determine the transform applied to the dequantized coefficients 311.

[0231] reconstruction

[0232] The reconstruction unit 314 (eg, the summer 314 ) is used to add the reconstructed residual block 313 to the prediction block 365 to obtain the reconstructed block 315 in the pixel domain, for example, by adding the pixel point values ​​of the reconstructed residual block 313 and the pixel point values ​​of the prediction block 365 .

[0233] Filtering

[0234] The loop filter unit 320 (in or after the encoding loop) is used to filter the reconstruction block 315 to obtain a filter block 321, so as to smoothly perform pixel conversion or improve video quality, etc. The loop filter unit 320 may include one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as an adaptive loop filter (ALF), a noise suppression filter (NSF), or any combination. For example, the loop filter unit 220 may include a deblocking filter, an SAO filter, and an ALF filter. The order of the filtering process may be a deblocking filter, an SAO filter, and an ALF filter. For another example, a process called luma mapping with chromascaling (LMCS) (i.e., an adaptive in-loop shaper) is added. This process is performed before deblocking. For another example, the deblocking filtering process may also be applied to internal sub-block edges, such as affine sub-block edges, ATMVP sub-block edges, sub-block transform (SBT) edges, and intra sub-partition (ISP) edges. Figure 3 3. Although shown as a loop filter in FIG. 3, in other configurations, the loop filter unit 320 may be implemented as a post-loop filter.

[0235] Decoded Image Buffer

[0236] The decoded video blocks 321 in one picture are then stored in a decoded picture buffer 330 which stores the decoded pictures 331 as reference pictures for subsequent motion compensation of other pictures and / or for respective output display.

[0237] The decoder 30 is used to output the decoded image 311 through the output terminal 312, etc., to display it to the user or for the user to view it.

[0238] predict

[0239] The inter-frame prediction unit 354 may be functionally identical to the inter-frame prediction unit 244 (particularly the motion compensation unit), and the intra-frame prediction unit 344 may be functionally identical to the intra-frame prediction unit 254, and may determine the division or segmentation and perform prediction based on the segmentation and / or prediction parameters or corresponding information received from the coded image data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304). In the present application, the intra-frame prediction unit 344 may determine the target intra-frame prediction mode for intra-frame prediction of the current image block based on the target reference value obtained in the entropy decoding unit 204 and the probability vector obtained in the neural network model. The mode application unit 360 may be used to perform prediction (intra-frame or inter-frame prediction) of each block according to the reconstructed image, block or corresponding pixel point (filtered or unfiltered), and obtain a prediction block 365.

[0240] When the video slice is encoded as an intra coded (I) slice, the intra prediction unit 354 in the mode application unit 360 is used to generate a prediction block 365 for the image block of the current video slice based on the indicated intra prediction mode and data from the previously decoded block of the current image. When the video image is encoded as an inter coded (i.e., B or P) slice, the inter prediction unit 344 (e.g., motion compensation unit) in the mode application unit 360 is used to generate a prediction block 365 for the video block of the current video slice based on the motion vector and other syntax elements received from the entropy decoding unit 304. For inter prediction, these prediction blocks can be generated from one of the reference images in one of the reference image lists. The video decoder 30 can construct reference frame list 0 and list 1 using a default construction technique based on the reference images stored in the DPB 330. The same or similar processes may be applied to embodiments of coding block groups (e.g., video coding block groups) and / or coding blocks (e.g., video coding blocks) in addition to or as an alternative to slices (e.g., video slices), for example, a video may be encoded using I, P or B coding block groups and / or coding blocks.

[0241] The mode application unit 360 is used to determine prediction information for a video block of a current video slice by parsing motion vectors and other syntax elements, and use the prediction information to generate a prediction block for the current video block being decoded. For example, the mode application unit 360 uses some of the received syntax elements to determine the prediction mode (e.g., intra-frame prediction or inter-frame prediction) for encoding the video block of the video slice, the inter-frame prediction slice type (e.g., B slice, P slice, or GPB slice), the construction information of one or more reference picture lists for the slice, the motion vector for each inter-frame coded video block of the slice, the inter-frame prediction state for each inter-frame coded video block of the slice, and other information to decode the video block in the current video slice. In addition to slices (e.g., video slices) or as an alternative to slices, the same or similar process can be applied to embodiments of coding block groups (e.g., video coding block groups) and / or coding blocks (e.g., video coding blocks), for example, the video can be encoded using I, P, or B coding block groups and / or coding blocks.

[0242] In one embodiment, Figure 3 The illustrated video encoder 30 may also be used to partition and / or decode an image using slices (also referred to as video slices), where an image may be partitioned or decoded using one or more slices (usually non-overlapping). Each slice may include one or more blocks (e.g., CTUs) or one or more groups of blocks (e.g., coding blocks in the H.265 / HEVC / VVC standard and bricks in the VVC standard).

[0243] In one embodiment, Figure 3 The video decoder 30 shown can also be used to segment and / or decode an image using slices / coding block groups (also called video coding block groups) and / or coding blocks (also called video coding blocks), wherein an image can be segmented or decoded using one or more slices / coding block groups (usually non-overlapping), each slice / coding block group may include one or more blocks (e.g., CTU) or one or more coding blocks, etc., wherein each coding block may be in a rectangular shape, etc., and may include one or more complete or partial blocks (e.g., CTU).

[0244] Other variations of the video decoder 30 may be used to decode the encoded image data 21. For example, the decoder 30 may generate an output video stream without the loop filter unit 320. For example, a non-transform based decoder 30 may directly dequantize the residual signal without the inverse transform processing unit 312 for certain blocks or frames. In another implementation, the video decoder 30 may have the dequantization unit 310 and the inverse transform processing unit 312 combined into a single unit.

[0245] It should be understood that the processing result of the current step can be further processed in the encoder 20 and the decoder 30 and then output to the next step. For example, after interpolation filtering, motion vector derivation or loop filtering, the processing result of interpolation filtering, motion vector derivation or loop filtering can be further operated, such as clipping or shifting operation.

[0246] It should be noted that the derived motion vector of the current block (including but not limited to the control point motion vector of the affine mode, the sub-block motion vector of the affine, plane, ATMVP mode, the temporal motion vector, etc.) can be further operated. For example, the value of the motion vector is limited to a predefined range according to the representation bit of the motion vector. If the representation bit of the motion vector is bitDepth, the range is -2^(bitDepth-1) to 2^(bitDepth-1)-1, where "^" represents the power. For example, if bitDepth is set to 16, the range is -32768 to 32767; if bitDepth is set to 18, the range is -131072 to 131071. For example, the value of the derived motion vector (e.g., the MV of 4 4×4 sub-blocks in an 8×8 block) is limited so that the maximum difference between the integer parts of the 4 4×4 sub-block MVs does not exceed N pixels, for example, not more than 1 pixel. Two methods of limiting motion vectors according to bitDepth are provided here.

[0247] Although the above embodiments mainly describe video coding and decoding, it should be noted that the embodiments of the decoding system 10, the encoder 20 and the decoder 30 and other embodiments described herein can also be used for still image processing or coding and decoding, that is, the processing or coding and decoding of a single image in video coding and decoding that is independent of any previous or consecutive images. In general, if the image processing is limited to a single image 17, the inter-frame prediction unit 244 (encoder) and the inter-frame prediction unit 344 (decoder) may not be available. All other functions (also called tools or techniques) of the video encoder 20 and the video decoder 30 can also be used for still image processing, such as residual calculation 204 / 304, transformation 206, quantization 208, inverse quantization 210 / 310, (inverse) transformation 212 / 312, segmentation 262 / 362, intra-frame prediction 254 / 354 and / or loop filtering 220 / 320, entropy coding 270 and entropy decoding 304.

[0248] Figure 4 Schematic diagram of a video decoding device 400 provided in an embodiment of the present invention. The video decoding device 400 is suitable for implementing the disclosed embodiments described herein. In one embodiment, the video decoding device 400 may be a decoder, such as Figure 1A The video decoder 30 in the embodiment may also be an encoder, for example Figure 1AThe video encoder 20 in.

[0249] The video decoding device 400 includes: an input port 410 (or input port 410) and a receiving unit (Rx) 420 for receiving data; a processor, a logic unit or a central processing unit (CPU) 430 for processing data; for example, the processor 430 here can be a neural network processor 430; a transmitter unit (Tx) 440 and an output port 450 (or output port 450) for transmitting data; and a memory 460 for storing data. The video decoding device 400 may also include an optical-to-electrical (OE) component and an electrical-to-optical (EO) component coupled to the input port 410, the receiving unit 420, the transmitting unit 440 and the output port 450 for the output or output of optical or electrical signals.

[0250] The processor 430 is implemented by hardware and software. The processor 430 can be implemented as one or more processor chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. The processor 430 communicates with the input port 410, the receiving unit 420, the sending unit 440, the output port 450, and the memory 460. The processor 430 includes a decoding module 470 (e.g., a decoding module 470 based on a neural network NN). The decoding module 470 implements the embodiments disclosed above. For example, the decoding module 470 performs, processes, prepares, or provides various encoding operations. Therefore, the decoding module 470 provides substantial improvements to the functions of the video decoding device 400 and affects the switching of the video decoding device 400 to different states. Alternatively, the decoding module 470 is implemented with instructions stored in the memory 460 and executed by the processor 430.

[0251] The memory 460 includes one or more disks, tape drives, and solid-state hard disks, and can be used as an overflow data storage device for storing such programs when such programs are selected for execution, and for storing instructions and data read during program execution. The memory 460 can be volatile and / or non-volatile, and can be a read-only memory (ROM), a random access memory (RAM), a ternary content-addressable memory (TCAM), and / or a static random-access memory (SRAM).

[0252] Figure 5 A simplified block diagram of an apparatus 500 is provided for an exemplary embodiment, and the apparatus 500 may be used as Figure 1A Either or both of the source device 12 and the destination device 14 in .

[0253] The processor 502 in the device 500 may be a central processing unit. Alternatively, the processor 502 may be any other type of device or devices that are currently available or will be developed in the future and are capable of manipulating or processing information. Although a single processor such as the processor 502 shown in the figure may be used to implement the disclosed implementation, using more than one processor is faster and more efficient.

[0254] In one implementation, the memory 504 in the apparatus 500 may be a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may be used as the memory 504. The memory 504 may include code and data 506 accessed by the processor 502 via a bus 512. The memory 504 may also include an operating system 508 and an application 510, which includes at least one program that allows the processor 502 to perform the methods described herein. For example, the application 510 may include applications 1 to N, and also include a video decoding application that performs the methods described herein.

[0255] The apparatus 500 may also include one or more output devices, such as a display 518. In one example, the display 518 may be a touch-sensitive display that combines a display with a touch-sensitive element that may be used to sense touch input. The display 518 may be coupled to the processor 502 via the bus 512.

[0256] Although bus 512 in device 500 is described herein as a single bus, bus 512 may include multiple buses. In addition, auxiliary storage may be directly coupled to other components of device 500 or accessed through a network, and may include a single integrated unit such as a memory card or multiple units such as multiple memory cards. Therefore, device 500 may have a variety of configurations.

[0257] Since the embodiments of the present application involve the application of neural networks, in order to facilitate understanding, some nouns or terms used in the embodiments of the present application are explained below, and these nouns or terms are also considered as part of the content of the invention.

[0258] (1) Neural Network

[0259] Neural Network (NN) is a machine learning model. A neural network can be composed of neural units. A neural unit can refer to an operation unit with xs and intercept 1 as input. The output of the operation unit can be:

[0260]

[0261] Where s=1, 2, ...n, n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolution layer. The activation function can be a sigmoid function. A neural network is a network formed by connecting many of the above-mentioned single neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the characteristics of the local receptive field. The local receptive field can be an area composed of several neural units.

[0262] (2) Deep Neural Networks

[0263] Deep Neural Network (DNN), also known as multi-layer neural network, can be understood as a neural network with many hidden layers. There is no special metric for "many" here. From the position of different layers of DNN, the neural network inside DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the layers in between are all hidden layers. The layers are fully connected, that is, any neuron in the i-th layer must be connected to any neuron in the i+1-th layer. Although DNN looks complicated, the work of each layer is actually not complicated. Simply put, it is the following linear relationship expression: in, is the input vector, is the output vector, is the offset vector, W is the weight matrix (also called coefficient), and α() is the activation function. Each layer is just an input vector After such a simple operation, the output vector Since DNN has many layers, the coefficient W and the offset vector The definition of these parameters in DNN is as follows: Taking coefficient W as an example: Assume that in a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as The superscript 3 represents the layer number of the coefficient W, while the subscripts correspond to the output third layer index 2 and the input second layer index 4. In summary, the coefficients from the kth neuron in the L-1th layer to the jth neuron in the Lth layer are defined as It should be noted that the input layer does not have a W parameter. In a deep neural network, more hidden layers allow the network to better describe complex situations in the real world. Theoretically, the more parameters a model has, the higher its complexity and the greater its "capacity", which means it can complete more complex learning tasks. Training a deep neural network is the process of learning the weight matrix, and its ultimate goal is to obtain the weight matrix of all layers of the trained deep neural network (the weight matrix formed by many layers of vector W).

[0264] (3) Convolutional Neural Network

[0265] A convolutional neural network (CNN) is a deep neural network with a convolutional structure. It is a deep learning architecture. A deep learning architecture refers to multiple levels of learning at different levels of abstraction through machine learning algorithms. As a deep learning architecture, CNN is a feed-forward artificial neural network in which each neuron can respond to the image input into it. A convolutional neural network contains a feature extractor consisting of a convolutional layer and a pooling layer. The feature extractor can be regarded as a filter, and the convolution process can be regarded as using a trainable filter to convolve an input image or convolution feature plane (feature map).

[0266] A convolutional layer is a layer of neurons in a convolutional neural network that performs convolution on the input signal. A convolutional layer can include many convolution operators, also known as kernels. In image processing, the convolution operator is essentially a weight matrix, which is usually predefined. During the convolution operation on the image, the weight matrix is ​​usually processed horizontally on the input image one pixel after another (or two pixels after two pixels... depending on the value of the stride) to complete the work of extracting specific features from the image. The size of the weight matrix should be related to the size of the image. It should be noted that the depth dimension of the weight matrix is ​​the same as the depth dimension of the input image. During the convolution operation, the weight matrix will extend to the entire depth of the input image. Therefore, convolution with a single weight matrix will produce a convolution output with a single depth dimension, but in most cases, instead of using a single weight matrix, multiple weight matrices of the same size (rows × columns) are applied, that is, multiple isotype matrices. The output of each weight matrix is ​​stacked to form the depth dimension of the convolution image, where the dimension can be understood as being determined by the "multiple" mentioned above. Different weight matrices can be used to extract different features in the image, for example, one weight matrix is ​​used to extract image edge information, another weight matrix is ​​used to extract specific colors of the image, and another weight matrix is ​​used to blur unnecessary noise points in the image. The multiple weight matrices have the same size (rows × columns), and the size of the feature maps extracted by the multiple weight matrices of the same size is also the same. The extracted multiple feature maps of the same size are then merged to form the output of the convolution operation. The weight values ​​in these weight matrices need to be obtained through a lot of training in practical applications. The weight matrices formed by the weight values ​​obtained through training can be used to extract information from the input image, so that the convolutional neural network can make correct predictions. When the convolutional neural network has multiple convolutional layers, the initial convolutional layer often extracts more general features, which can also be called low-level features; as the depth of the convolutional neural network deepens, the features extracted by the later convolutional layers become more and more complex, such as high-level semantic features. Features with higher semantics are more suitable for the problem to be solved.

[0267] Since it is often necessary to reduce the number of training parameters, it is often necessary to periodically introduce a pooling layer after the convolution layer. It can be a convolution layer followed by a pooling layer, or multiple convolution layers followed by one or more pooling layers. In the image processing process, the only purpose of the pooling layer is to reduce the spatial size of the image. The pooling layer can include an average pooling operator and / or a maximum pooling operator to sample the input image to obtain an image of smaller size. The average pooling operator can calculate the pixel values ​​in the image within a specific range to produce an average value as the result of average pooling. The maximum pooling operator can take the pixel with the largest value in the range as the result of maximum pooling within a specific range. In addition, just as the size of the weight matrix used in the convolution layer should be related to the image size, the operator in the pooling layer should also be related to the image size. The size of the image output after processing by the pooling layer can be smaller than the size of the image input to the pooling layer. Each pixel in the image output by the pooling layer represents the average value or maximum value of the corresponding sub-region of the image input to the pooling layer.

[0268] After being processed by the convolution layer / pooling layer, the convolution neural network is not sufficient to output the required output information. Because as mentioned above, the convolution layer / pooling layer will only extract features and reduce the parameters brought by the input image. However, in order to generate the final output information (the required class information or other related information), the convolution neural network needs to use the neural network layer to generate one or a group of outputs of the required number of classes. Therefore, the neural network layer may include multiple hidden layers, and the parameters contained in the multiple hidden layers can be pre-trained according to the relevant training data of the specific task type. For example, the task type may include image recognition, image classification, image super-resolution reconstruction, etc.

[0269] Optionally, after the multiple hidden layers in the neural network layer, an output layer of the entire convolutional neural network is also included. The output layer has a loss function similar to the classification cross entropy, which is specifically used to calculate the prediction error. Once the forward propagation of the entire convolutional neural network is completed, the back propagation will begin to update the weight values ​​and biases of the aforementioned layers to reduce the loss of the convolutional neural network and the error between the result output by the convolutional neural network through the output layer and the ideal result.

[0270] (4) Loss Function

[0271] In the process of training a deep neural network, because we hope that the output of the deep neural network is as close as possible to the value we really want to predict, we can compare the predicted value of the current network with the target value we really want, and then update the weight vector of each layer of the neural network according to the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in the deep neural network). For example, if the predicted value of the network is high, adjust the weight vector to make it predict a lower value, and keep adjusting until the deep neural network can predict the target value we really want or a value very close to the target value we really want. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value", which is the loss function or objective function, which are important equations used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of the deep neural network becomes a process of minimizing this loss as much as possible.

[0272] The following will be combined Figure 6 The target model (also called neural network) for intra prediction is described in detail. Figure 6 An example architecture 600 of a target model (eg, a neural network for prediction, abbreviated as prediction network) is shown. The first data block or the second data block is used as input to the neural network, which processes the input data using a convolutional layer and outputs a probability vector of the current image block using a Softmax layer.

[0273] The main structure of the neural network model is the deep residual network (ResNet) in the convolutional neural network. ResNet consists of 9 residual blocks; each residual block has two basic layers, each of which contains a convolution layer, a nonlinear activation function (rectified linear unit, ReLU) layer and a batch normalization (BN) layer. In addition, there is a convolution layer between the first residual block and the data input. This convolution layer can speed up the convergence speed of the neural network model training and reduce the impact of gradient disappearance or gradient explosion on training. After the last residual block, a convolution layer and an adaptive average pooling layer are connected in sequence to convert the feature map into a multi-dimensional vector. Finally, the Softmax layer outputs the probability distribution of 67 intra-frame prediction modes. This probability distribution is a multi-dimensional vector with a dimension of Fig. 9 The sum of the number of intra prediction modes shown is 67.

[0274] It should be understood that the neural network model used in the embodiment of the present application is only a specific example, and the embodiment of the present application is not limited to this. Those skilled in the art can also construct a neural network model through other architectures to achieve the same technical effect as the neural network model in the embodiment of the present application, for example: a neural network model composed of one or more convolutional neural network architectures such as AlexNet, ZFNet, VGGNet, GoogleNet, or non-convolutional neural network architectures. In addition, those skilled in the art can also achieve the same technical effect as the embodiment of the present application by changing part of the structure of the neural network model in the embodiment of the present application, for example: changing the number of residual blocks of the neural network model in the present application, etc.

[0275] After the initial neural network model structure for calculating the probability vector is designed, the neural network model is trained. The training process of the neural network model in the embodiment of the present application is as follows:

[0276] The embodiment of the present application trains the initial neural network model on an image processor. The training database of the neural network model includes a training set, a validation set and a test set, and the training set, the validation set and the test set respectively include a series of image sets. The first data block or the second data block input into the neural network model is referred to as the target data block below. For each size of the target data block, an image with the same size as the target data block is selected from the training set, the validation set and the test set as the training set, the validation set and the test set of the target data block, and each image in the training set of the target data block is compressed to extract the input data and the target data of the neural network model. The input data is a data block obtained after compression of each image, and the target data is the first identifier corresponding to the target intra-frame prediction mode of each image. In each training process, the compressed data block is input into the neural network model to obtain a 67-dimensional probability vector corresponding to the data block. The cross entropy loss function shown in formula (1-2) is used to measure the error between the output and expectation of the neural network model in this training process, so as to determine whether to end the training of the neural network model.

[0277]

[0278] Specifically, when the cross entropy loss function shown in formula (1-2) is used to calculate the cross entropy loss function value after each training, M represents the number of sample categories, where M is the total number of intra-frame prediction modes of the image, that is, M=67; i represents the target data of the image used in each training, that is, the first identifier corresponding to the target intra-frame prediction mode of the image; T(i) is an indicator function, which is 1 when the indicated category is the same as the sample category, otherwise it is 0. For example, when the target data of the current image is 30, when i is 30, the T(i) value is 1; when i takes other values ​​(any value in 0-29 or 31-66), the T(i) value is 0; P i represents the value of the i-th dimension in the probability distribution of the Softmax layer output. It should be understood that P i The larger the value, that is, the greater the probability of using the target intra-frame prediction mode for intra-frame prediction, the smaller the loss function value will be.

[0279] After completing each training and each calculation of the cross entropy loss function value, determine the relationship between each cross entropy loss function value and the preset loss value. When each cross entropy loss function value is greater than the preset loss value, adjust the weights of each parameter in the neural network model according to each cross entropy loss function value, and use the adjusted neural network model for the next training until the cross entropy loss function value is less than or equal to the preset loss value, and then end the training of the neural network model. Then, use the validation set and test set of the target data block to verify and test the neural network model respectively; after verification and testing, the parameters of the neural network model corresponding to the target data block can be determined, that is, the neural network model corresponding to the target data block can be obtained. Perform the above operations for each size of target data block, and you can get neural network models with different parameter configurations corresponding to target data blocks of different sizes.

[0280] It should be understood that the embodiments of the present application may adopt different training databases according to different application scenarios, and the embodiments of the present application do not make specific limitations on this. In addition, the training method of the neural network model in the embodiments of the present application is only a specific example, and the present application does not make specific limitations on this. In addition to the above method, other methods can also be used to train the neural network model. For example, first, the initial neural network model is trained, verified and tested respectively using a training set, a verification set and a test set containing images of a specific size to determine an intermediate neural network model, and then the method described in the above embodiment is used to train, verify and test the intermediate neural network model for target data blocks of different sizes, respectively, to obtain neural network models with different parameter configurations corresponding to target data blocks of different sizes, wherein the above-mentioned specific size can be any size selected according to the actual scenario, and is not specifically limited here.

[0281] It should be understood that target data blocks of different sizes correspond to neural network models with different parameter configurations. After the above-mentioned size transformation operation, the size types of the second data block are less than the size types of the first data block, thereby saving storage space for storing the neural network model corresponding to the target data block.

[0282] The following will describe in detail the process of encoding and decoding the intra-frame prediction mode of the current image block using the method proposed in the present application.

[0283] Figure 7 700 is a flowchart of a method 700 for encoding a target intra-frame prediction mode of a current image block in an embodiment of the present application. The method 700 may be performed by the video encoder 20, specifically, by the intra-frame prediction unit 254 and the entropy coding unit 270 of the video encoder 20. The method 700 is described as a series of steps or operations, and it should be understood that some steps of the method 700 are not limited in execution order, such as step S701 and step S702. Figure 7 The method 700 shown includes steps S701, S702, S703 and S704, which will be described in detail below.

[0284] Step S701: Obtain at least two of a reconstructed block, a predicted block, and a residual block of surrounding image blocks of a current image block.

[0285] Specifically, three surrounding image blocks adjacent to the current image block on the left, upper left, and upper sides are obtained, and the sizes of the three surrounding image blocks are the same as the current image block (such as Fig.10 As shown); each of the three surrounding image blocks includes a reconstructed block, a residual block and a predicted block, and at least two of the reconstructed blocks, residual blocks and predicted blocks of each of the three surrounding image blocks and the current image block are selected. In a feasible implementation, the reconstructed blocks, residual blocks and predicted blocks of each of the three surrounding image blocks and the current image block may be selected for subsequent splicing or cascading operations.

[0286] Step S702, obtaining a probability vector of the current image block based on at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks and a prediction mode probability model, wherein multiple elements in the probability vector correspond one-to-one to multiple intra-frame prediction modes, and any element in the probability vector is used to represent the probability of using the intra-frame prediction mode corresponding to the any element when predicting the current image block.

[0287] In a feasible implementation, when the target intra-frame prediction mode of the current image block is a non-matrix weighted intra-frame prediction MIP mode, the probability vector of the current image block is obtained based on at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks, the current image block and the prediction mode probability model.

[0288] The method of obtaining the probability vector of the current image block based on at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks and the prediction mode probability model specifically includes: obtaining a first data block by splicing or cascading at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks; and obtaining the probability vector of the current image block based on the first data block and the prediction mode probability model. It should be noted that the sizes of the first data blocks obtained by splicing and cascading are different.

[0289] The process of obtaining the first data block by splicing or concatenating at least two of the reconstructed block, the predicted block and the residual block of the surrounding image blocks can be divided into four cases according to the position relationship of the current image block in the current frame: that is, when the current image block is located at the upper left edge, the left edge, the upper edge and the non-edge of the current frame (corresponding to Figure 11-1 to Figure 11-4 In the four positional relationships, the ways of selecting the three surrounding image blocks of the current image block are different. In the embodiment of the present application, a detailed description is given with the width and height of the current image block being M and N respectively, where M and N are positive integers.

[0290] The first position relationship, such as Figure 11-1 As shown, when the current image block is located at a non-edge position of the current frame, that is, when the distances between the current image block and the left edge and the upper edge of the current frame are respectively greater than or equal to M and N, three surrounding image blocks on the left, upper left and upper sides that are equal in size to and adjacent to the current image block can be directly selected from the current frame. At this time, the three surrounding image blocks have been intra-predicted, and each of the three surrounding image blocks contains a reconstructed block, a residual block and a prediction block; for the current image block, there is no data in the residual block, the reconstructed block and the prediction block of the current image block, and the residual block, the reconstructed block and the prediction block of the current image block are filled with default values ​​or 0, where the default value is 2 n −1, n is the pixel bit depth inside the video encoder 100 .

[0291] Optionally, in the first positional relationship, when the reconstructed block, residual block and prediction block of each of the three surrounding image blocks and the current image block are selected, and the three surrounding coding blocks and the current coding block are spliced, the obtained first data block is a three-channel data block with a size of 2M×2N, and the three channels of the first data block are respectively filled with the reconstructed block, residual block and prediction block of the current image block and the surrounding image blocks; when the reconstructed block and prediction block, or the reconstructed block and residual block, or the prediction block and residual block of each of the three surrounding image blocks and the current image block are selected, and the three surrounding coding blocks and the current coding block are spliced, the obtained first data block is a two-channel data block with a size of 2M×2N, and the two channels of the first data block are respectively filled with the reconstructed block and prediction block of the current image block and the surrounding image blocks.

[0292] Optionally, in the first position relationship, if the data filled in the residual block, the reconstructed block and the predicted block of the current image block are all 0, when the reconstructed block, the residual block and the predicted block of each image block in the three surrounding image blocks and the current image block are selected, and the three surrounding coding blocks and the current coding block are cascaded, the first data block obtained is a data block with nine channels and a size of M×N; when the reconstructed block and the predicted block of each image block in the three surrounding image blocks and the current image block are selected, and the three surrounding coding blocks and the current coding block are cascaded, the first data block obtained is a data block with four channels and a size of M×N; it should be understood that when the data blocks selected for cascading in each image block in the surrounding image blocks and the current image block are different, and the data filled in each data block of the current image block are different, the number of channels of the first data block obtained by cascading is different. Except for the two cases listed above, in other cases, the number of channels of the first data block obtained by cascading the surrounding image blocks and the current image block is easy to derive, and will not be repeated here.

[0293] The second position relationship, such as Figure 11-2 As shown in FIG. 1 , when the current image block is located at the upper left edge of the current frame, that is, when the distances between the current image block and the left edge and the upper edge of the current frame are less than M and N respectively, three surrounding image blocks on the left, upper left and upper sides that are the same size and adjacent to the current image block cannot be selected from the current frame. At this time, three surrounding image blocks with the same size as the current image block are expanded to the adjacent left, upper left and upper sides of the current image block respectively; in this positional relationship, the reconstructed blocks and the predicted blocks in each of the three surrounding image blocks are filled with the default value 2 n -1, the residual block in each image block is filled with 0 or 2 n -1; the reconstruction block, residual block and prediction block of the current image block can be filled with 0 or 2 n -1.

[0294] Optionally, in the second position relationship, when a default value of 2 is filled into the reconstructed block and the predicted block in each of the three surrounding image blocks, n -1, the residual block in each image block is filled with 0, and the reconstructed block, residual block and prediction block of the current image block are all filled with 0; when the reconstructed block, residual block and prediction block of each image block in the three surrounding image blocks and the current image block are selected, and the three surrounding coding blocks and the current coding block are spliced, the first data block obtained is a two-channel data block with a size of 2M×2N; when the residual block and prediction block of each image block in the three surrounding image blocks and the current image block are selected, and the three surrounding coding blocks and the current coding block are spliced, the first data block obtained is a one-channel data block with a size of 2M×2N; it should be understood that when the data blocks selected for splicing in each image block in the surrounding image blocks and the current image block are different, and the data filled in the current image block and each data block of the three surrounding image blocks are different, the number of channels of the first data block obtained by splicing is different. In addition to the two cases listed above, in other cases, the number of channels of the first data block obtained by splicing the surrounding image blocks and the current image block is easy to derive, and will not be repeated here.

[0295] Optionally, in the second position relationship, when a default value of 2 is filled into the reconstructed block and the predicted block in each of the three surrounding image blocks, n -1, the residual block in each image block is filled with 0, and the reconstructed block, residual block and prediction block of the current image block are all filled with 0; when the reconstructed block, residual block and prediction block of each image block in the three surrounding image blocks and the current image block are selected, and the three surrounding coding blocks and the current coding block are cascaded, the first data block obtained is a four-channel data block with a size of M×N; it should be understood that when the data blocks selected for cascading in each image block in the surrounding image blocks and the current image block are different, and the data filled in each data block in the current image block and the surrounding image blocks are different, the number of channels of the first data block obtained by cascading is different. In addition to the above-mentioned cases, in other cases, the number of channels of the first data block obtained by cascading the surrounding image blocks and the current image block can be easily derived and will not be repeated here.

[0296] The third position relationship, such as Figure 11-3 As shown in FIG. 1 , when the current image block is located at the left edge of the current frame, that is, the distance between the current image block and the left edge of the current frame is less than M, and the distance between the current image block and the upper edge of the current frame is greater than or equal to N, at this time, an upper image block adjacent to the current image block and having the same size as the current image block can be selected from the current frame, and two surrounding image blocks with the same size as the current image block can be expanded to the left and upper left sides adjacent to the current image block respectively; in this position relationship, the reconstructed block and the predicted block in each image block of the expanded left and upper left image blocks are filled with the default value 2n -1, the residual block in each image block is filled with 0 or 2 n -1; the reconstruction block, residual block and prediction block of the current image block can be filled with 0 or 2 n -1.

[0297] Optionally, in the third position relationship, when a default value of 2 is filled into the reconstructed block and the predicted block in each of the left and upper left image blocks, n -1, the residual block in each image block is filled with 0, and the reconstructed block, residual block and prediction block of the current image block are all filled with 0; when the reconstructed block, residual block and prediction block of each image block in the three surrounding image blocks and the current image block are selected, and the three surrounding coding blocks and the current coding block are spliced, the first data block obtained is a two-channel data block with a size of 2M×2N; when the residual block and prediction block of each image block in the three surrounding image blocks and the current image block are selected, and the three surrounding coding blocks and the current coding block are spliced, the first data block obtained is a one-channel data block with a size of 2M×2N; it should be understood that when the data blocks selected for splicing in each image block in the surrounding image blocks and the current image block are different, and the data filled in the current image block and each data block of the three surrounding image blocks are different, the number of channels of the first data block obtained by splicing is different. In addition to the two cases listed above, in other cases, the number of channels of the first data block obtained by splicing the surrounding image blocks and the current image block is easy to derive, and will not be repeated here.

[0298] Optionally, in the third position relationship, when the reconstructed block, the predicted block and the residual block in each image block on the left and upper left sides are filled with a default value of 2 n -1, fill 0 into the reconstruction block and residual block of the current image block, and fill 2 into the prediction block n -1; when the reconstructed block, the residual block and the predicted block of each of the three surrounding image blocks and the current image block are selected, and the three surrounding coding blocks and the current coding block are cascaded, the first data block obtained is a data block with ten channels and a size of M×N; it should be understood that when the data blocks selected for cascading in each of the surrounding image blocks and the current image block are different, and the data filled in each of the data blocks in the current image block and the surrounding image blocks are different, the number of channels of the first data block obtained by cascading is different. In addition to the above-mentioned cases, in other cases, the number of channels of the first data block obtained by cascading the surrounding image blocks and the current image block can be easily derived and will not be repeated here.

[0299] The fourth position relationship, such as Figure 11-4As shown in FIG. 1 , when the current image block is located at the left edge of the current frame, that is, the distance between the current image block and the left edge of the current frame is greater than or equal to M, and the distance between the current image block and the upper edge of the current frame is less than N, at this time, a left image block adjacent to the current image block and having the same size as the current image block can be selected from the current frame, and then two surrounding image blocks with the same size as the current image block are expanded to the upper side and the upper left side adjacent to the current image block, respectively. In this position relationship, the reconstructed block and the predicted block in each image block of the expanded upper and upper left image blocks are filled with the default value 2 n -1, the residual block in each image block can be filled with 0 or 2 n -1; the reconstruction block, residual block and prediction block of the current image block can be filled with 0 or 2 n -1.

[0300] Optionally, in the fourth position relationship, when the reconstructed block and the predicted block in each image block of the upper and upper left image blocks are filled with a default value of 2 n -1, the residual block in each image block is filled with 0, and the reconstructed block, residual block and prediction block of the current image block are all filled with 0; when the reconstructed block, residual block and prediction block of each image block in the three surrounding image blocks and the current image block are selected, and the three surrounding coding blocks and the current coding block are spliced, the first data block obtained is a two-channel data block with a size of 2M×2N; when the residual block and prediction block of each image block in the three surrounding image blocks and the current image block are selected, and the three surrounding coding blocks and the current coding block are spliced, the first data block obtained is a one-channel data block with a size of 2M×2N; it should be understood that when the data blocks selected for splicing in each image block in the surrounding image blocks and the current image block are different, and the data filled in the current image block and each data block of the three surrounding image blocks are different, the number of channels of the first data block obtained by splicing is different. In addition to the two cases listed above, in other cases, the number of channels of the first data block obtained by splicing the surrounding image blocks and the current image block is easy to derive, and will not be repeated here.

[0301] Optionally, in the fourth position relationship, when the reconstructed block, the predicted block, and the residual block in each of the upper and upper left image blocks are filled with a default value of 2 n -1, fill 0 into the reconstruction block and residual block of the current image block, and fill 2 into the prediction block n-1; when the reconstructed block, the residual block and the predicted block of each of the three surrounding image blocks and the current image block are selected, and the three surrounding coding blocks and the current coding block are cascaded, the first data block obtained is a data block with ten channels and a size of M×N; it should be understood that when the data blocks selected for cascading in each of the surrounding image blocks and the current image block are different, and the data filled in each of the data blocks in the current image block and the surrounding image blocks are different, the number of channels of the first data block obtained by cascading is different. In addition to the above-mentioned cases, in other cases, the number of channels of the first data block obtained by cascading the surrounding image blocks and the current image block can be easily derived and will not be repeated here.

[0302] In addition, the image block described in the present application can be understood as but not limited to: prediction unit (PU) or coding unit CU or transform unit TU, etc. According to the provisions of different video compression codec standards, CU may contain one or more prediction units PU. For example, in HEVC, a CU may contain multiple PUs, but in VVC, one CU corresponds to one PU. The image block may have a fixed or variable size and may differ in size according to different video compression codec standards. In addition, the current image block refers to the image block to be encoded or decoded, such as a prediction unit to be encoded or decoded.

[0303] In a feasible implementation manner, the above-mentioned obtaining of the probability vector of the current image block based on the first data block and the prediction mode probability model may include: when the size of the first data block is the target size, inputting the first data block into the prediction mode probability model for processing to obtain the probability vector of the current image block, or; when the size of the first data block is not the target size, performing a size transformation operation on the first data block to obtain a second data block, the size of the second data block is equal to the target size, the size transformation operation includes at least one of scaling and transposition, the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; inputting the second data block into the prediction mode probability model for processing to obtain the probability vector of the current image block.

[0304] Among them, the above probability pattern prediction model can be Figure 6 The neural network model shown in, or other forms of neural network models, or mathematical models, are not specifically limited in this application.

[0305] Table 1: Size types of the first data block

[0306]

[0307] Among them, for current image blocks of different sizes, after the current image block and the three surrounding coding blocks of the current image block are spliced ​​or cascaded, the sizes of the first data blocks obtained are different. In the embodiment of the present application, there are 17 types of sizes of the first data block (as shown in Table 1), and the first data block is resized to obtain a second data block, and the types of sizes of the second data block are less than 17.

[0308] When the size of the first data block is not the target size, the process of performing a size conversion operation on the first data block to obtain the second data block can be divided into four methods:

[0309] (1) When the size of the first data block is not the target size, and the ratio of the first side length to the second side length of the first data block is equal to the ratio of the first side length to the second side length of the target size, scaling the first data block in both horizontal and vertical directions to obtain a second data block;

[0310] (2) When the size of the first data block is not the target size, and the ratio of the first side length to the second side length of the first data block is equal to the ratio of the second side length to the first side length of the target size, the first data block is transposed, and scaled proportionally in both horizontal and vertical directions to obtain a second data block;

[0311] (3) when the size of the first data block is not the target size, and the first side length and the second side length of the first data block are respectively equal to the second side length and the first side length of the target size, transposing the first data block to obtain a second data block;

[0312] (4) When the size of the first data block is not the target size, the first data block is scaled to obtain the second data block.

[0313] Optionally, there may be seven types of target sizes, that is, one type is selected from the sizes in Table 1 where the ratio of the first side length to the second side length is 1:1, 1:2, 2:1, 1:4, 4:1, 1:8 and 8:1 as the target size. The size of the first data block is compared with the seven target sizes, and according to the relationship between the size of the first data block and the seven target sizes, any one of the above four methods (1), (2) or (3) may be used to obtain the second data block.

[0314] For example, the process of obtaining the second data block in the above embodiment is described in detail with the first side length and the second side length of the first data block being 16×8: if the ratio of the first side length and the second side length in the target size is 2:1 and the first side length and the second side length of the target size are 32×16 respectively, then the first side length and the second side length of the first data block are geometrically enlarged in both the horizontal and vertical directions to obtain a second data block with the first side length and the second side length being 32×16 respectively; if the ratio of the first side length and the second side length in the target size is 2:1 and the first side length and the second side length of the target size are 8×4 respectively, then the first side length and the second side length of the first data block are geometrically reduced in both the horizontal and vertical directions to obtain a second data block with the first side length and the second side length being 8×4 respectively. It should be understood that the specific process of scaling mentioned in other parts of the embodiments of the present application is the same as here.

[0315] Optionally, there may be four types of target sizes, namely, one size in which the ratio of the first side length to the second side length is 1:1, one size in which the ratio is 1:2 or 2:1, one size in which the ratio is 1:4 or 4:1, and one size in which the ratio is 1:8 or 8:1, for a total of four sizes, as target sizes. The size of the first data block is compared with the four target sizes, and according to the relationship between the size of the first data block and the four target sizes, any one of the above four methods (1), (2), or (3) may be used to obtain the second data block.

[0316] It should be noted that in the embodiment of the present application, when the first data block needs to be transposed and scaled, there is no order for the transposition and scaling operations, and the second data blocks obtained when the processing is performed in different operation orders have the same size.

[0317] It can be seen that in an embodiment of the present application, through the above two implementation methods, the types of sizes of the first data block can be reduced from 17 to 7 or 4 types of the second data block, thereby effectively reducing the types of subsequent neural network models corresponding to second data blocks of different sizes, thereby saving storage space for storing neural network models and improving encoding performance.

[0318] Optionally, the target size may be one of the 17 sizes in Table 1, or one other than the 17 sizes in Table 1. The size of the first data block is compared with the one target size, and according to the relationship between the size of the first data block and the four target sizes, the above-mentioned method (4) may be used to obtain the second data block.

[0319] For example, the specific process of implementing the above embodiment is described in detail with the first side length and the second side length of the target size being 4×8: when the first side length and the second side length of the first data block size are 4×4 respectively, the second side length of the first data block is vertically enlarged to obtain a second data block with a first side length and a second side length of 4×8 respectively; when the first side length and the second side length of the first data block size are 16×4 respectively, the first side length and the second side length of the first data block are horizontally reduced and vertically enlarged to obtain a second data block with a first side length and a second side length of 4×8 respectively.

[0320] It can be seen that in the embodiment of the present application, the size types of the first data block can be reduced from 17 to 1, thereby minimizing the size types of the second data block, and subsequently obtaining a neural network model corresponding to the size of the second data block, significantly reducing the storage space used to store the neural network model and improving the encoding efficiency; at the same time, for the first data block with a simpler texture and a larger size, after being reduced to a second data block of a smaller size, the computational complexity of subsequently using the neural network model to calculate the probability vector can be effectively reduced, thereby improving the encoding performance.

[0321] In a feasible implementation manner, the obtaining the probability vector of the current image block according to the first data block and the prediction mode probability model may include: when the size of the first data block is the target size, inputting the first data block into the prediction mode probability model for processing to obtain the probability vector of the current image block, or;

[0322] When the size of the first data block is not the target size, a size transformation operation is performed on the first data block according to a size relationship between a first side length and a second side length of the first data block, and / or a horizontal gradient and a vertical gradient of the current image block to obtain a second data block, the size of the second data block being the target size, the size transformation operation including at least one of scaling and transposition, the scaling including horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; the second data block is input into a prediction mode probability model for processing to obtain a probability vector of the current image block.

[0323] Specifically, first, a target size corresponding to the first data block is determined based on the size relationship between the first side length and the second side length of the first data block, and the size relationship between the absolute value of the horizontal gradient and the absolute value of the vertical gradient and a preset threshold; when the size of the first data block is not the target size, a size transformation operation is performed on the first data block to obtain a second data block.

[0324] Among them, optionally, the value of the preset threshold can be 4*M*N, the first side length and the second side length of the current image block are M and N respectively, and the first side length and the second side length of the first data block are 2M and 2N respectively; the horizontal gradient of the current image block and the vertical gradient of the current image block are calculated by the Sobel operator. It should be understood that the value of the preset threshold can be determined according to actual conditions, and the embodiment of the present application does not limit this; in addition, the operator for calculating the horizontal gradient and the vertical gradient is not limited to the Sobel operator, and can be determined according to actual conditions. For example, the Scharr operator, the laplacian operator or other operators with similar functions can also be selected.

[0325] The gradient of the image represents the change in the grayscale of the image. The horizontal gradient and vertical gradient of the current image block represent the change in the grayscale of the current image block relative to the surrounding image blocks in the horizontal and vertical directions in the current frame. The image can be understood as a two-dimensional discrete function f(x, y). The gradient of the image f(x, y) at the point (x, y) is a vector with size and direction. Let Gx and Gy represent the gradient of the current image block in the horizontal and vertical directions respectively. The vector of this gradient can be expressed as:

[0326]

[0327] The magnitude of this vector is:

[0328]

[0329] The direction angle of this vector is:

[0330]

[0331] When the size of the first data block is not the target size, the process of performing a size transformation operation on the first data block to obtain the second data block according to the relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block can be divided into two cases:

[0332] (1) The size transformation operation includes scaling, and the size transformation operation is performed on the first data block according to the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, including: when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, the first side length and the second side length of the second data block are 4M / N, 4 respectively; when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, scaling the first data block to obtain the second data block. The first data block is scaled to obtain a second data block, and the first side length and the second side length of the second data block are 8M / N, 8 respectively; when the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, the first data block is scaled to obtain a second data block, and the first side length and the second side length of the second data block are 4, 4N / M respectively; when the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, the first data block is scaled to obtain a second data block, and the first side length and the second side length of the second data block are 8, 8N / M respectively; wherein M and N are positive integers.

[0333] (2) the size transformation operation includes at least one of scaling and transposing, and the size transformation operation is performed on the first data block according to the relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, including:

[0334] When the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, the first data block is scaled to obtain a second data block, and the first side length and the second side length of the second data block are 4M / N, 4 respectively; when the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, the first data block is scaled to obtain a second data block, and the first side length and the second side length of the second data block are 8M / N, 8 respectively;

[0335] When the first side length M of the first data block is less than the second side length N of the first data block, the following operations are performed on the first data block to obtain a second data block: when the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is less than a preset threshold, the first data block is transposed and scaled to obtain a second data block, and the first side length and the second side length of the second data block are 4N / M, 4 respectively; when the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is greater than or equal to a preset threshold, the first data block is transposed and scaled to obtain a second data block, and the first side length and the second side length of the second data block are 8N / M, 8 respectively; wherein M and N are positive integers.

[0336] It can be seen that in the embodiment of the present application, the process of determining the second data block based on the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block, makes full use of the current image's own information to determine the size of the second data block, enhances the correlation between the second data block and the current image block, thereby improving the accuracy of the subsequent probability vector obtained based on the second data block, thereby improving the accuracy of the data in the code stream; at the same time, by performing scaling and transposition operations, the number of types of second data block sizes can be reduced, and the number of neural network models corresponding to second data blocks of different sizes will also be reduced, thereby saving storage space for storing neural network models and improving encoding performance.

[0337] Step S703: determine the target intra prediction mode of the current image block.

[0338] Optionally, 67 intra prediction modes (such as Fig. 9 As shown) and the MIP mode, a target intra-frame prediction mode for intra-frame prediction of the current image block is searched. The target intra-frame prediction mode is Fig. 9 The mode value (0-66) in is the corresponding first identifier.

[0339] Among them, see Fig. 9 The 67 intra-frame prediction modes in VVC include direct current (DC) prediction mode, planar prediction mode and 65 angular prediction modes. The first identifiers of DC and planar prediction modes are 1 and 0 respectively, and the first identifiers of the remaining 65 angular prediction modes are 2-66 from the lower left corner to the upper right corner.

[0340] Step S704: Encode the target intra prediction mode into the bitstream according to the target intra prediction mode and the probability vector.

[0341] According to the target intra prediction mode and the probability vector, encoding the target intra prediction mode into the bitstream can be divided into two cases:

[0342] (1) When the resizing operation does not include transposition, a first reference value is determined according to a first identifier and a probability vector of a target intra prediction mode, and the first reference value is encoded to obtain a code stream corresponding to the target intra prediction mode.

[0343] Among them, the target intra prediction mode is Fig. 9 The mode value in is the first identifier corresponding to it, and the probability vector is a 67-dimensional vector. Each element in the probability vector corresponds to a probability interval on the interval (0, 1), and the elements in the probability vector are Fig. 9 The 67 mode values ​​in correspondence one by one.

[0344] The above-mentioned method determines the first reference value based on the first identifier and probability vector of the target intra-frame prediction mode, and encodes the first reference value to obtain the code stream corresponding to the target intra-frame prediction mode. Specifically, the element corresponding to the first identifier in the probability vector is determined according to the first identifier of the target intra-frame prediction mode, and then a numerical value is arbitrarily selected in the probability interval corresponding to the element as the first reference value and encoded into the code stream. The first reference value can be used to characterize the target intra-frame prediction mode of the current image block.

[0345] (2) When the resizing operation includes transposition and the target intra-frame prediction mode is an angular prediction mode, a second identifier is determined according to a preset constant and a first identifier of the target intra-frame prediction mode, and a second reference value is determined according to the second identifier and a probability vector; the second reference value is encoded to obtain a code stream corresponding to the target intra-frame prediction mode; when the resizing operation includes transposition and the target intra-frame prediction mode is a non-angular prediction mode, a first reference value is determined according to the first identifier and a probability vector of the target intra-frame prediction mode; the first reference value is encoded to obtain a code stream corresponding to the target intra-frame prediction mode.

[0346] The preset constant is Figure 3 The number of the multiple intra-frame prediction modes shown in , that is, 67, the angle prediction mode is an intra-frame prediction mode with a mode value of 2-66, and the non-angle prediction mode is a mode value of DC mode or planar mode.

[0347] Specifically, when the resizing operation includes transposition and the target intra-frame prediction mode is an angular prediction mode, the first identifier is subtracted from a preset constant to obtain a second identifier, an element corresponding to the second identifier is selected in the probability vector, and then a numerical value is arbitrarily selected from the probability interval corresponding to the element as the second reference value and encoded into the bitstream; when the resizing operation includes transposition and the target intra-frame prediction mode is a non-angular prediction mode, the encoding method for the target intra-frame prediction mode is the same as method (1), which will not be repeated here.

[0348] It can be seen that in Figure 7 In the illustrated embodiment, at least two of the reconstruction blocks, prediction blocks and residual blocks of the three surrounding image blocks of the current image block are obtained. Since the other one can be calculated from two of the reconstruction blocks, prediction blocks and residual blocks of each surrounding image block, the embodiment of the present application can make full use of the relevant information of the surrounding image blocks, and thus the probability vector of the current image block generated according to at least two of the reconstruction blocks, prediction blocks and residual blocks of the surrounding image blocks is more accurate, thereby improving the accuracy of the data encoded in the bitstream and improving the encoding performance; in addition, since there are multiple sizes of the first data block, by performing a size transformation operation on the first data block to obtain the second data block, the number of types of second data block sizes can be effectively reduced, thereby reducing the number of neural network models corresponding to second data blocks of different sizes, thereby saving storage space for the neural network model and improving encoding performance; at the same time, for a first data block with a simpler texture and a larger size, after being reduced to a second data block of a smaller size through a size transformation operation, the computational complexity of subsequently calculating the probability vector using the neural network model can be effectively reduced, thereby improving encoding efficiency.

[0349] See also Figure 8 , Figure 8 FIG. 8 is a flowchart of another method 800 for encoding a target intra-frame prediction mode of a current image block in the present application. Figure 8 As shown, method 800 includes the following six steps:

[0350] S801, obtaining surrounding image blocks of the current image block.

[0351] Specifically, three surrounding image blocks adjacent to the current image block on the left, upper left, and upper sides may be obtained, and the sizes of the three surrounding image blocks are the same as the current image block (eg Fig.10 As shown), each of the three surrounding image blocks includes a reconstructed block, a residual block and a prediction block, and one or more of the reconstructed blocks, residual blocks and prediction blocks of each of the three surrounding image blocks and the current image block can be selected for subsequent splicing or cascading operations.

[0352] S802: splice or concatenate surrounding image blocks to obtain a first data block.

[0353] S803, performing a size transformation operation on the first data block to obtain a second data block, where the size transformation operation includes at least one of scaling and transposition, and the scaling includes horizontal scaling, vertical scaling, or proportional scaling in both horizontal and vertical directions.

[0354] S804, input the second data block into the prediction mode probability model for processing to obtain a probability vector of the current image block, wherein multiple elements in the probability vector correspond one-to-one to multiple intra-frame prediction modes, and any element in the probability vector is used to represent the probability of using the intra-frame prediction mode corresponding to any element when predicting the current image block.

[0355] S805: Determine a target intra prediction mode for the current image block.

[0356] S806, encoding the target intra-frame prediction mode into the bitstream according to the target intra-frame prediction mode and the probability vector.

[0357] Specifically, steps S802 to S804 are respectively the same as the corresponding processes of step S702, and steps S805 and S806 are respectively the same as the corresponding steps in steps S703 and S704, which are not repeated here.

[0358] It can be seen that in Figure 8 In the illustrated embodiment, there are multiple sizes of the first data block. By performing a size transformation operation on the first data block to obtain the second data block, the number of types of second data block sizes can be effectively reduced, thereby reducing the number of neural network models corresponding to second data blocks of different sizes, thereby saving storage space for the neural network model and improving encoding performance; at the same time, for the first data block with a simpler texture and a larger size, after being reduced to a second data block of a smaller size through a size transformation operation, the computational complexity of subsequently calculating the probability vector using the neural network model can be effectively reduced, thereby improving encoding efficiency.

[0359] See also Fig.12 , Fig.12 This is a flowchart of an encoder encoding the relevant syntax elements of the intra-frame prediction mode of the current image block in an embodiment of the present application. Fig.12As shown, in VVC, for a current image block, the syntax elements related to its intra-frame prediction mode include a MIP flag, a multi-reference line MRL index, identifiers corresponding to the MIP mode and the ISP mode, respectively, a first identifier corresponding to the target intra-frame prediction mode of the current image block, and a probability vector of the current image block; when the entropy encoder 103 encodes the syntax elements related to the intra-frame prediction of the current image block, it is divided into the following steps: according to the description in step S704, a first reference value or a second reference value corresponding to the target intra-frame prediction mode is selected from the probability vector; the MIP flag is encoded in a conventional encoding method, and it is determined whether the MIP flag is a first preset value, and if so, the MIP mode is encoded in a bypass encoding method; if the MIP flag is a second preset value, the MRL index and the identifier corresponding to the ISP mode are encoded in a conventional encoding method in sequence, and the first reference value or the second reference value of the current image block for intra-frame prediction using the target intra-frame prediction mode is encoded in a bypass encoding method, wherein the first preset value and the second preset value are 1 and 0, respectively. It should be noted that the above-mentioned first preset value and second preset value are only a specific example in the embodiment of the present application, and the present application does not make any specific limitation on this.

[0360] Among them, information such as the MIP flag, MRL index and ISP mode has been determined in the process of searching for the target intra-frame prediction mode of the current image block. The MIP flag is used to indicate whether the current image block adopts the MIP mode for intra-frame prediction. The MRL index indicates the index of the reference row used for intra-frame prediction of the current image block. The ISP mode indicates the sub-block division method of the current image block, wherein all sub-blocks of the current image block share a common intra-frame prediction mode, and when the current image block adopts MRL, ISP is not performed.

[0361] It should be understood that the encoder 20 described in the embodiment of the present application adopts the context-adaptive binary arithmetic coding CABAC method to encode the syntax elements related to the intra-frame prediction of the current image block. The syntax elements related to the intra-frame prediction of the current image block can also be encoded in other ways in the present application, which is not specifically limited here.

[0362] See also Fig.13 , Fig.13 is a flowchart of a method 1300 for decoding a target intra-frame prediction mode of a current image block in an embodiment of the present application. Fig.13 The method 1300 shown includes steps S1301, S1302 and S1303, which will be described in detail below.

[0363] Step S1301: Obtain a code stream corresponding to a current image block, and at least two of a reconstructed block, a predicted block, and a residual block of image blocks surrounding the current image block.

[0364] Specifically, the decoder 30 obtains the video code stream encoded by the entropy encoder 304; the specific process of obtaining at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks of the current image block is the same as step S701 and will not be repeated here.

[0365] Step S1302, obtaining a probability vector of the current image block based on at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks and a prediction mode probability model, wherein multiple elements in the probability vector correspond one-to-one to multiple intra-frame prediction modes, and any element in the probability vector is used to represent the probability of using the intra-frame prediction mode corresponding to the any element when predicting the current image block.

[0366] Specifically, the specific execution process of step S1302 is the same as step S702, which will not be repeated here.

[0367] Step S1303: determining a target intra prediction mode of the current image block according to the bit stream and the probability vector, and determining a prediction block of the current image block according to the target intra prediction mode.

[0368] Specifically, the code stream is decoded to obtain a target reference value; and a target intra-frame prediction mode of the current image block is determined according to the target reference value.

[0369] The above-mentioned determination of the target intra prediction mode of the current image block according to the target reference value includes two cases:

[0370] (1) When the resizing operation does not include transposition, the target intra prediction mode is determined according to the target reference value and the probability vector.

[0371] Specifically, the target reference value is between 0 and 1, and its corresponding identifier in the probability vector is determined according to the probability interval in which the target reference value falls, and the target intra-frame prediction mode of the current image block is determined according to the corresponding identifier.

[0372] (2) When the size transformation operation includes transposition, the first identifier is determined according to the target reference value and the probability vector; when the intra-frame prediction mode corresponding to the first identifier is an angle prediction mode, the target intra-frame prediction mode is determined according to a preset constant and the first identifier; when the intra-frame prediction mode corresponding to the first identifier is a non-angle prediction mode, the target intra-frame prediction mode is determined according to the target reference value and the probability vector.

[0373] Specifically, the first identifier of the element corresponding to the target reference value in the probability vector is determined according to the probability interval in which the target reference value falls. When the intra-frame prediction mode corresponding to the first identifier is the angle prediction mode, the first identifier is subtracted from the preset constant to obtain a second identifier. Fig. 9The intra-frame prediction mode corresponding to the first identifier is the target intra-frame prediction mode of the current image block; when the intra-frame prediction mode corresponding to the first identifier is a non-angle prediction mode, the first identifier is Fig. 9 The intra-frame prediction mode corresponding to is the target intra-frame prediction mode of the current image block.

[0374] See also Fig.14 , Fig.14 is a flowchart of a method 1400 for decoding a target intra-frame prediction mode of a current image block in an embodiment of the present application. Fig.14 The method 1400 shown includes steps S1401, S1402, 1403, S1404 and S1405, which will be described in detail below.

[0375] S1401, obtaining a code stream corresponding to a current image block and surrounding image blocks of the current image block.

[0376] Specifically, the above step of obtaining the code stream corresponding to the current image block is the same as the corresponding step in step S1301, and the above step of obtaining the surrounding image blocks of the current image block is the same as the corresponding step in step S801, which will not be repeated here.

[0377] S1402: splice or concatenate surrounding image blocks to obtain a first data block.

[0378] S1403, performing a size transformation operation on the first data block to obtain a second data block, where the size transformation operation includes at least one of scaling and transposition, and the scaling includes horizontal scaling, vertical scaling, or proportional scaling in both horizontal and vertical directions.

[0379] S1404, input the second data block into the prediction mode probability model for processing to obtain a probability vector of the current image block, wherein multiple elements in the probability vector correspond one-to-one to multiple intra-frame prediction modes, and any element in the probability vector is used to represent the probability of using the intra-frame prediction mode corresponding to any element when predicting the current image block.

[0380] S1405 , determining a target intra-frame prediction mode of the current image block according to the code stream and the probability vector, and determining a prediction block of the current image block according to the target intra-frame prediction mode.

[0381] Specifically, the specific execution processes of steps S1402, S1403, S1404 and S1405 are respectively the same as S802, S803, S804 and S1303, and will not be repeated here.

[0382] See also Fig.15 , Fig.151 is a schematic block diagram of a coding device 1500 provided in the present application. It should be understood that the coding device 1500 herein may correspond to Figure 1A Or you can Figure 2 The encoder 20 in the embodiment of the present invention may include:

[0383] The acquisition unit 1501 is used to acquire at least two of the reconstructed blocks, the predicted blocks and the residual blocks of the surrounding image blocks of the current image block.

[0384] Processing unit 1502 is used to obtain a probability vector of the current image block based on at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks and a prediction mode probability model, wherein multiple elements in the probability vector correspond one-to-one to multiple intra-frame prediction modes, and any element in the probability vector is used to represent the probability of using the intra-frame prediction mode corresponding to any element when predicting the current image block.

[0385] The determination unit 1503 is configured to determine a target intra-frame prediction mode for the current image block.

[0386] The encoding unit 1504 is used to encode the target intra-frame prediction mode into the bitstream according to the target intra-frame prediction mode and the probability vector.

[0387] In one possible implementation, the above-mentioned processing unit is specifically used to: when the target intra-frame prediction mode is a non-matrix weighted intra-frame prediction MIP mode, obtain the probability vector of the current image block based on at least two of the reconstructed block, prediction block and residual block of the surrounding image blocks, the current image block and the prediction mode probability model.

[0388] In a possible implementation, the processing unit is specifically used to: splice or cascade at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks to obtain a first data block; and obtain a probability vector of the current image block based on the first data block and the prediction mode probability model.

[0389] In a possible implementation, in the aspect of obtaining the probability vector of the current image block based on the first data block and the prediction mode probability model, the processing unit is specifically used to: when the size of the first data block is the target size, input the first data block into the prediction mode probability model for processing to obtain the probability vector of the current image block, or; when the size of the first data block is not the target size, perform a size transformation operation on the first data block to obtain a second data block, the size of the second data block is equal to the target size, the size transformation operation includes at least one of scaling and transposition, the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; input the second data block into the prediction mode probability model for processing to obtain the probability vector of the current image block.

[0390] In a possible implementation, in the aspect of obtaining the probability vector of the current image block according to the first data block and the prediction mode probability model, the processing unit is specifically used to: when the size of the first data block is the target size, input the first data block into the prediction mode probability model for processing to obtain the probability vector of the current image block, or; when the size of the first data block is not the target size, perform a size transformation operation on the first data block according to the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and vertical gradient of the current image block to obtain a second data block, the size of the second data block is the target size, the size transformation operation includes at least one of scaling and transposition, the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; input the second data block into the prediction mode probability model for processing to obtain the probability vector of the current image block.

[0391] In a possible implementation, the resizing operation includes scaling, and in terms of performing a resizing operation on the first data block to obtain the second data block according to a size relationship between a first side length and a second side length of the first data block and / or a horizontal gradient and a vertical gradient of the current image block, the processing unit is specifically configured to:

[0392] When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of a horizontal gradient and an absolute value of a vertical gradient is less than a preset threshold, scaling the first data block to obtain a second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively;

[0393] When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of a horizontal gradient and an absolute value of a vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain a second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively;

[0394] When a first side length M of the first data block is less than a second side length N of the first data block, and a sum of an absolute value of a horizontal gradient and an absolute value of a vertical gradient is less than a preset threshold, scaling the first data block to obtain a second data block, wherein the first side length and the second side length of the second data block are 4 and 4N / M respectively;

[0395] When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain a second data block, wherein the first side length and the second side length of the second data block are 8 and 8N / M respectively;

[0396] Wherein, M and N are positive integers.

[0397] In a possible implementation manner, the resizing operation includes at least one of scaling and transposition, and in the aspect of performing a resizing operation on the first data block to obtain the second data block according to a size relationship between a first side length and a second side length of the first data block and / or a horizontal gradient and a vertical gradient of the current image block, the processing unit is specifically configured to:

[0398] When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of a horizontal gradient and an absolute value of a vertical gradient is less than a preset threshold, scaling the first data block to obtain a second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively;

[0399] When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of a horizontal gradient and an absolute value of a vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain a second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively;

[0400] When the first side length M of the first data block is less than the second side length N of the first data block, the following operations are performed on the first data block to obtain the second data block:

[0401] When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is less than a preset threshold, the first data block is transposed and scaled to obtain a second data block, wherein the first side length and the second side length of the second data block are 4N / M, 4 respectively;

[0402] When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is greater than or equal to a preset threshold, transposing and scaling the first data block to obtain a second data block, wherein the first side length and the second side length of the second data block are 8N / M, 8 respectively;

[0403] Wherein, M and N are positive integers.

[0404] In a possible implementation, the encoding unit is specifically used to: when the resizing operation does not include transposition, determine a first reference value according to a first identifier and a probability vector of a target intra-frame prediction mode; encode the first reference value to obtain a code stream corresponding to the target intra-frame prediction mode;

[0405] When the resizing operation includes transposition, a second identifier is determined according to a preset constant and the first identifier, and a second reference value is determined according to the second identifier and the probability vector; the second reference value is encoded to obtain a code stream corresponding to the target intra-frame prediction mode.

[0406] The above is the specific implementation of the encoding device 1500 Fig.13Similarly, the encoding device 1500 can also be used to implement Figure 8 All or any steps in the method embodiment shown are not repeated here.

[0407] See also Fig.16 , Fig.16 1 is a schematic block diagram of a coding device 1600 provided in the present application. It should be understood that the coding device 1600 herein may correspond to Figure 1A Or you can Figure 3 The decoder 30 in the encoding device 1600 may include:

[0408] The acquisition unit 1601 is configured to acquire a code stream corresponding to a current image block, and at least two of a reconstructed block, a predicted block, and a residual block of image blocks surrounding the current image block.

[0409] Processing unit 1602 is used to obtain a probability vector of the current image block based on at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks and a prediction mode probability model, wherein multiple elements in the probability vector correspond one-to-one to multiple intra-frame prediction modes, and any element in the probability vector is used to represent the probability of using the intra-frame prediction mode corresponding to any element when predicting the current image block.

[0410] The decoding unit 1603 is used to determine a target intra-frame prediction mode of the current image block according to the code stream and the probability vector, and determine a prediction block of the current image block according to the target intra-frame prediction mode.

[0411] In a possible implementation, the processing unit is specifically used to: splice or cascade at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks to obtain a first data block; and obtain a probability vector of the current image block based on the first data block and the prediction mode probability model.

[0412] In a possible implementation, in the aspect of obtaining the probability vector of the current image block according to the first data block and the prediction mode probability model, the processing unit is specifically configured to: when the size of the first data block is the target size, input the first data block into the prediction mode probability model for processing to obtain the probability vector of the current image block, or;

[0413] When the size of the first data block is not the target size, a size transformation operation is performed on the first data block to obtain a second data block, the size of the second data block is equal to the target size, the size transformation operation includes at least one of scaling and transposition, the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; the second data block is input into a prediction mode probability model for processing to obtain a probability vector of the current image block.

[0414] In a possible implementation, in the aspect of obtaining the probability vector of the current image block according to the first data block and the prediction mode probability model, the processing unit is specifically configured to: when the size of the first data block is the target size, input the first data block into the prediction mode probability model for processing to obtain the probability vector of the current image block, or;

[0415] When the size of the first data block is not the target size, a size transformation operation is performed on the first data block according to a size relationship between a first side length and a second side length of the first data block, and / or a horizontal gradient and a vertical gradient of the current image block to obtain a second data block, wherein the first side length and the second side length are lengths of two mutually perpendicular sides in the first data block, and the size of the second data block is the target size, and the size transformation operation includes at least one of scaling and transposition, and the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; and the second data block is input into a prediction mode probability model for processing to obtain a probability vector of the current image block.

[0416] In a possible implementation manner, the resizing operation includes scaling. In the aspect of performing a resizing operation on the first data block to obtain the second data block according to the size relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block, the processing unit is specifically configured to:

[0417] When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of a horizontal gradient and an absolute value of a vertical gradient is less than a preset threshold, scaling the first data block to obtain a second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively;

[0418] When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of a horizontal gradient and an absolute value of a vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain a second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively;

[0419] When a first side length M of the first data block is less than a second side length N of the first data block, and a sum of an absolute value of a horizontal gradient and an absolute value of a vertical gradient is less than a preset threshold, scaling the first data block to obtain a second data block, wherein the first side length and the second side length of the second data block are 4 and 4N / M respectively;

[0420] When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain a second data block, wherein the first side length and the second side length of the second data block are 8 and 8N / M respectively;

[0421] Wherein, M and N are positive integers.

[0422] In a possible implementation manner, the resizing operation includes at least one of scaling and transposition. In the aspect of performing a resizing operation on the first data block according to a size relationship between a first side length and a second side length of the first data block and / or a horizontal gradient and a vertical gradient of the current image block to obtain the second data block, the processing unit is specifically configured to:

[0423] When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of a horizontal gradient and an absolute value of a vertical gradient is less than a preset threshold, scaling the first data block to obtain a second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively;

[0424] When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of a horizontal gradient and an absolute value of a vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain a second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively;

[0425] When the first side length M of the first data block is less than the second side length N of the first data block, the following operations are performed on the first data block to obtain the second data block:

[0426] When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is less than a preset threshold, the first data block is transposed and scaled to obtain a second data block, wherein the first side length and the second side length of the second data block are 4N / M, 4 respectively;

[0427] When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is greater than or equal to a preset threshold, transposing and scaling the first data block to obtain a second data block, wherein the first side length and the second side length of the second data block are 8N / M, 8 respectively;

[0428] Wherein, M and N are positive integers.

[0429] In a possible implementation, the decoding unit is specifically used to: decode the code stream to obtain a target reference value; when the size transformation operation does not include transposition, determine the target intra-frame prediction mode according to the target reference value and the probability vector;

[0430] When the size transformation operation includes transposition, the first identifier is determined according to the target reference value and the probability vector; when the intra-frame prediction mode corresponding to the first identifier is an angle prediction mode, the target intra-frame prediction mode is determined according to a preset constant and the first identifier; when the intra-frame prediction mode corresponding to the first identifier is a non-angle prediction mode, the target intra-frame prediction mode is determined according to the target reference value and the probability vector.

[0431] The above is the specific implementation of the encoding device 1600 Fig.13 Similarly, the encoding device 1600 can also be used to implement Fig.14 All or any steps in the described method embodiments will not be repeated here.

[0432] Those skilled in the art will appreciate that the functions described in conjunction with the various illustrative logic blocks, modules, and algorithm steps disclosed herein can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions described in the various illustrative logic blocks, modules, and steps can be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to tangible media, such as data storage media, or includes any communication media that facilitates the transfer of computer programs from one place to another (e.g., according to a communication protocol). In this manner, computer-readable media may generally correspond to (1) non-temporary tangible computer-readable storage media, or (2) communication media, such as signals or carrier waves. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in this application. A computer program product may include computer-readable media.

[0433] As an example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other media that can be used to store the desired program code in the form of instructions or data structures and can be accessed by a computer. Also, any connection is properly referred to as a computer-readable medium. For example, if a coaxial cable, optical fiber cable, twisted pair, digital subscriber line DSL, or wireless technologies such as infrared, radio, and microwaves are used to transmit instructions from a website, server, or other remote source, then the coaxial cable, optical fiber cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of media. However, it should be understood that the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other temporary media, but are actually directed to non-temporary tangible storage media. As used herein, disks and optical disks include compact disks CD, laser optical disks, optical optical disks, digital versatile disks DVD, and Blu-ray disks, where disks typically reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0434] Instructions may be executed by one or more processors such as one or more digital signal processors DSP, general-purpose microprocessors, application-specific integrated circuits ASIC, field programmable logic arrays FPGA or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein may refer to any of the aforementioned structures or any other structures suitable for implementing the techniques described herein. In addition, in some aspects, the functions described by the various illustrative logic blocks, modules, and steps described herein may be provided in dedicated hardware and / or software modules configured for encoding and decoding, or incorporated in a combined codec. Moreover, the technology may be fully implemented in one or more circuits or logic elements.

[0435] The techniques of the present application may be implemented in a variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in this application to emphasize functional aspects of devices for performing the disclosed techniques, but do not necessarily need to be implemented by different hardware units. In fact, as described above, the various units may be combined in a codec hardware unit in conjunction with appropriate software and / or firmware, or provided by interoperating hardware units (including one or more processors as described above).

[0436] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A coding method for intra-frame prediction mode, characterized in that: The method comprises: Obtain at least two of a reconstructed block, a predicted block, and a residual block of surrounding image blocks of the current image block; Obtaining a probability vector of the current image block according to at least two of the reconstructed block, the predicted block, and the residual block of the surrounding image blocks and a prediction mode probability model, wherein a plurality of elements in the probability vector correspond one-to-one to a plurality of intra-frame prediction modes, and any element in the probability vector is used to represent the probability of adopting the intra-frame prediction mode corresponding to any element when predicting the current image block; Determining a target intra-frame prediction mode for the current image block; The target intra-frame prediction mode is encoded into a bitstream according to the target intra-frame prediction mode and the probability vector.

2. The method according to claim 1, characterized in that The obtaining the probability vector of the current image block according to at least two of the reconstructed block, the predicted block and the residual block of the surrounding image blocks and the prediction mode probability model comprises: When the target intra prediction mode is a non-matrix weighted intra prediction MIP mode, a probability vector of the current image block is obtained according to at least two of the reconstructed block, the prediction block and the residual block of the surrounding image blocks and the prediction mode probability model.

3. The method according to claim 1 or 2, characterized in that: The obtaining the probability vector of the current image block according to at least two of the reconstructed block, the predicted block and the residual block of the surrounding image blocks and the prediction mode probability model comprises: splicing or concatenating at least two of the reconstructed block, the predicted block, and the residual block of the surrounding image blocks to obtain a first data block; A probability vector of the current image block is obtained according to the first data block and the prediction mode probability model.

4. The method according to claim 3, characterized in that The obtaining the probability vector of the current image block according to the first data block and the prediction mode probability model includes: When the size of the first data block is the target size, inputting the first data block into the prediction mode probability model for processing to obtain a probability vector of the current image block, or; When the size of the first data block is not the target size, a size transformation operation is performed on the first data block to obtain a second data block, the size of the second data block is equal to the target size, the size transformation operation includes at least one of scaling and transposition, the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; the second data block is input into the prediction mode probability model for processing to obtain a probability vector of the current image block.

5. The method according to claim 3, characterized in that: The obtaining the probability vector of the current image block according to the first data block and the prediction mode probability model includes: When the size of the first data block is the target size, inputting the first data block into the prediction mode probability model for processing to obtain a probability vector of the current image block, or; When the size of the first data block is not the target size, a size transformation operation is performed on the first data block according to a size relationship between a first side length and a second side length of the first data block, and / or a horizontal gradient and a vertical gradient of the current image block to obtain a second data block, wherein the size of the second data block is the target size, and the size transformation operation includes at least one of scaling and transposition, and the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; the second data block is input into the prediction mode probability model for processing to obtain a probability vector of the current image block.

6. The method according to claim 5, characterized in that The resizing operation includes the scaling, and performing the resizing operation on the first data block according to the size relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block includes: When the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively; When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of the horizontal gradient and an absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively; When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than the preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4 and 4N / M respectively; When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8 and 8N / M respectively; Wherein, the M and the N are positive integers.

7. The method according to claim 4, characterized in that The resizing operation includes at least one of the scaling and the transposition, and the resizing operation is performed on the first data block according to the size relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, including: When the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively; When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of the horizontal gradient and an absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively; When the first side length M of the first data block is less than the second side length N of the first data block, performing the following operations on the first data block to obtain the second data block: When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is less than the preset threshold, performing the transposition and the scaling on the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4N / M, 4 respectively; When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is greater than or equal to the preset threshold, performing the transposition and the scaling on the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8N / M, 8 respectively; Wherein, the M and the N are positive integers.

8. The method according to any one of claims 4 to 7, characterized in that: The step of encoding the target intra-frame prediction mode into a bitstream according to the target intra-frame prediction mode and the probability vector comprises: When the resizing operation does not include the transposition, determining a first reference value according to the first identifier of the target intra-frame prediction mode and the probability vector; encoding the first reference value to obtain a code stream corresponding to the target intra-frame prediction mode; When the size transformation operation includes the transposition and the target intra-frame prediction mode is an angular prediction mode, a second identifier is determined according to a preset constant and a first identifier of the target intra-frame prediction mode, and a second reference value is determined according to the second identifier and the probability vector; the second reference value is encoded to obtain a code stream corresponding to the target intra-frame prediction mode; when the size transformation operation includes the transposition and the target intra-frame prediction mode is a non-angular prediction mode, a first reference value is determined according to the first identifier of the target intra-frame prediction mode and the probability vector; the first reference value is encoded to obtain a code stream corresponding to the target intra-frame prediction mode.

9. A coding method for intra-frame prediction mode, characterized in that: The method comprises: Get the surrounding image blocks of the current image block; splicing or cascading the surrounding image blocks to obtain a first data block; Performing a size transformation operation on the first data block to obtain a second data block, wherein the size transformation operation includes at least one of scaling and transposition, and the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; Inputting the second data block into a prediction mode probability model for processing to obtain a probability vector of the current image block, wherein a plurality of elements in the probability vector correspond one-to-one to a plurality of intra-frame prediction modes, and any element in the probability vector is used to represent a probability of adopting the intra-frame prediction mode corresponding to any element when predicting the current image block; Determining a target intra-frame prediction mode for the current image block; The target intra-frame prediction mode is encoded into a bitstream according to the target intra-frame prediction mode and the probability vector.

10. The method according to claim 9, characterized in that The performing a size transformation operation on the first data block to obtain a second data block includes: When the target intra prediction mode is a non-matrix weighted intra prediction MIP mode and the size of the first data block is not the target size, the size transformation operation is performed on the first data block to obtain the second data block, and the size of the second data block is equal to the target size.

11. The method according to claim 9, characterized in that The performing a size transformation operation on the first data block to obtain a second data block includes: When the target intra-frame prediction mode is a non-matrix weighted intra-frame prediction MIP mode, and the size of the first data block is not the target size, the size transformation operation is performed on the first data block according to the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, and the size of the second data block is equal to the target size.

12. The method according to claim 10 or 11, characterized in that: The method further comprises: When the target intra prediction mode is a non-matrix weighted intra prediction MIP mode and the size of the first data block is equal to the target size, the first data block is input into the prediction mode probability model for processing to obtain a probability vector of the current image block.

13. The method according to claim 12, characterized in that The resizing operation includes the scaling, and performing the resizing operation on the first data block according to the size relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block includes: When the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively; When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of the horizontal gradient and an absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively; When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than the preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4 and 4N / M respectively; When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8 and 8N / M respectively; Wherein, the M and the N are positive integers.

14. The method according to claim 12, characterized in that The resizing operation includes at least one of the scaling and the transposition, and the resizing operation is performed on the first data block according to the size relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, including: When the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively; When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of the horizontal gradient and an absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively; When the first side length M of the first data block is less than the second side length N of the first data block, performing the following operations on the first data block to obtain the second data block: When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is less than the preset threshold, performing the transposition and the scaling on the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4N / M, 4 respectively; When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is greater than or equal to the preset threshold, performing the transposition and the scaling on the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8N / M, 8 respectively; Wherein, the M and the N are positive integers.

15. The method according to claim 14, characterized in that: The step of encoding the target intra-frame prediction mode into a bitstream according to the target intra-frame prediction mode and the probability vector comprises: When the resizing operation does not include the transposition, determining a first reference value according to the first identifier of the target intra-frame prediction mode and the probability vector; encoding the first reference value to obtain a code stream corresponding to the target intra-frame prediction mode; When the size transformation operation includes the transposition and the target intra-frame prediction mode is an angular prediction mode, a second identifier is determined according to a preset constant and a first identifier of the target intra-frame prediction mode, and a second reference value is determined according to the second identifier and the probability vector; the second reference value is encoded to obtain a code stream corresponding to the target intra-frame prediction mode; when the size transformation operation includes the transposition and the target intra-frame prediction mode is a non-angular prediction mode, a first reference value is determined according to the first identifier of the target intra-frame prediction mode and the probability vector; the first reference value is encoded to obtain a code stream corresponding to the target intra-frame prediction mode.

16. A decoding method for an intra-frame prediction mode, characterized in that: The method comprises: Acquire a code stream corresponding to a current image block, and at least two of a reconstructed block, a predicted block, and a residual block of image blocks surrounding the current image block; Obtaining a probability vector of the current image block according to at least two of the reconstructed block, the predicted block, and the residual block of the surrounding image blocks and a prediction mode probability model, wherein a plurality of elements in the probability vector correspond one-to-one to a plurality of intra-frame prediction modes, and any element in the probability vector is used to represent the probability of adopting the intra-frame prediction mode corresponding to any element when predicting the current image block; A target intra-frame prediction mode of the current image block is determined according to the code stream and the probability vector, and a prediction block of the current image block is determined according to the target intra-frame prediction mode.

17. The method according to claim 16, characterized in that The obtaining the probability vector of the current image block according to at least two of the reconstructed block, the predicted block and the residual block of the surrounding image blocks and the prediction mode probability model comprises: splicing or concatenating at least two of the reconstructed block, the predicted block, and the residual block of the surrounding image blocks to obtain a first data block; A probability vector of the current image block is obtained according to the first data block and the prediction mode probability model.

18. The method according to claim 17, characterized in that The obtaining the probability vector of the current image block according to the first data block and the prediction mode probability model includes: When the size of the first data block is the target size, inputting the first data block into the prediction mode probability model for processing to obtain a probability vector of the current image block, or; When the size of the first data block is not the target size, a size transformation operation is performed on the first data block to obtain a second data block, the size of the second data block is equal to the target size, the size transformation operation includes at least one of scaling and transposition, the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; the second data block is input into the prediction mode probability model for processing to obtain a probability vector of the current image block.

19. The method according to claim 17, characterized in that: The obtaining the probability vector of the current image block according to the first data block and the prediction mode probability model includes: When the size of the first data block is the target size, inputting the first data block into the prediction mode probability model for processing to obtain a probability vector of the current image block, or; When the size of the first data block is not the target size, a size transformation operation is performed on the first data block according to a size relationship between a first side length and a second side length of the first data block, and / or a horizontal gradient and a vertical gradient of the current image block to obtain a second data block, wherein the first side length and the second side length are lengths of two mutually perpendicular sides in the first data block, and the size of the second data block is the target size, and the size transformation operation includes at least one of scaling and transposition, and the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; and the second data block is input into the prediction mode probability model for processing to obtain a probability vector of the current image block.

20. The method according to claim 19, characterized in that: The resizing operation includes the scaling, and performing the resizing operation on the first data block according to the size relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block includes: When the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively; When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of the horizontal gradient and an absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively; When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than the preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4 and 4N / M respectively; When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8 and 8N / M respectively; Wherein, the M and the N are positive integers.

21. The method according to claim 19, characterized in that The resizing operation includes at least one of the scaling and the transposition, and the resizing operation is performed on the first data block according to the size relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, including: When the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively; When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of the horizontal gradient and an absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively; When the first side length M of the first data block is less than the second side length N of the first data block, performing the following operations on the first data block to obtain the second data block: When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is less than the preset threshold, performing the transposition and the scaling on the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4N / M, 4 respectively; When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is greater than or equal to the preset threshold, performing the transposition and the scaling on the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8N / M, 8 respectively; Wherein, the M and the N are positive integers.

22. The method according to any one of claims 18 to 21, characterized in that The determining, according to the bitstream and the probability vector, a target intra-frame prediction mode of the current image block comprises: Decoding the code stream to obtain a target reference value; When the resizing operation does not include the transposition, determining the target intra prediction mode according to the target reference value and the probability vector; When the size transformation operation includes the transposition, a first identifier is determined according to the target reference value and the probability vector; when the intra-frame prediction mode corresponding to the first identifier is an angle prediction mode, the target intra-frame prediction mode is determined according to a preset constant and the first identifier; when the intra-frame prediction mode corresponding to the first identifier is a non-angle prediction mode, the target intra-frame prediction mode is determined according to the target reference value and the probability vector.

23. A decoding method for an intra-frame prediction mode, characterized in that: The method comprises: Obtaining a code stream corresponding to a current image block and surrounding image blocks of the current image block; splicing or cascading the surrounding image blocks to obtain a first data block; Performing a size transformation operation on the first data block to obtain a second data block, wherein the size transformation operation includes at least one of scaling and transposition, and the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; Inputting the second data block into a prediction mode probability model for processing to obtain a probability vector of the current image block, wherein a plurality of elements in the probability vector correspond one-to-one to a plurality of intra-frame prediction modes, and any element in the probability vector is used to represent a probability of adopting the intra-frame prediction mode corresponding to any element when predicting the current image block; A target intra-frame prediction mode of the current image block is determined according to the code stream and the probability vector, and a prediction block of the current image block is determined according to the target intra-frame prediction mode.

24. The method according to claim 23, characterized in that The performing a size transformation operation on the first data block to obtain a second data block includes: When the size of the first data block is not the target size, the size transformation operation is performed on the first data block to obtain the second data block, and the size of the second data block is equal to the target size.

25. The method according to claim 23, characterized in that The performing a size transformation operation on the first data block to obtain a second data block includes: When the size of the first data block is not the target size, the size transformation operation is performed on the first data block according to the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, and the size of the second data block is equal to the target size.

26. The method according to claim 24 or 25, characterized in that The method further comprises: When the size of the first data block is equal to the target size, the first data block is input into the prediction mode probability model for processing to obtain a probability vector of the current image block.

27. The method according to claim 26, characterized in that: The resizing operation includes the scaling, and performing the resizing operation on the first data block according to the size relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block includes: When the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively; When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of the horizontal gradient and an absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively; When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than the preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4 and 4N / M respectively; When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8 and 8N / M respectively; Wherein, the M and the N are positive integers.

28. The method according to claim 26, characterized in that The resizing operation includes at least one of the scaling and the transposition, and the resizing operation is performed on the first data block according to the size relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, including: When the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively; When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of the horizontal gradient and an absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively; When the first side length M of the first data block is less than the second side length N of the first data block, performing the following operations on the first data block to obtain the second data block: When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is less than the preset threshold, performing the transposition and the scaling on the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4N / M, 4 respectively; When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is greater than or equal to the preset threshold, performing the transposition and the scaling on the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8N / M, 8 respectively; Wherein, the M and the N are positive integers.

29. The method according to claim 28, characterized in that The determining, according to the bitstream and the probability vector, a target intra-frame prediction mode of the current image block comprises: Decoding the code stream to obtain a target reference value; When the resizing operation does not include the transposition, determining the target intra prediction mode according to the target reference value and the probability vector; When the size transformation operation includes the transposition, a first identifier is determined according to the target reference value and the probability vector; when the intra-frame prediction mode corresponding to the first identifier is an angle prediction mode, the target intra-frame prediction mode is determined according to a preset constant and the first identifier; when the intra-frame prediction mode corresponding to the first identifier is a non-angle prediction mode, the target intra-frame prediction mode is determined according to the target reference value and the probability vector.

30. A coding device, characterized in that: The device comprises: An acquisition unit, used to acquire at least two of a reconstructed block, a predicted block, and a residual block of surrounding image blocks of the current image block; a processing unit, configured to obtain a probability vector of the current image block according to at least two of the reconstructed block, the predicted block, and the residual block of the surrounding image blocks and a prediction mode probability model, wherein a plurality of elements in the probability vector correspond one-to-one to a plurality of intra-frame prediction modes, and any element in the probability vector is used to represent a probability of adopting the intra-frame prediction mode corresponding to any element when predicting the current image block; A determination unit, configured to determine a target intra-frame prediction mode of the current image block; The encoding unit is used to encode the target intra-frame prediction mode into a bitstream according to the target intra-frame prediction mode and the probability vector.

31. The device according to claim 30, characterized in that The processing unit is specifically used for: When the target intra prediction mode is a non-matrix weighted intra prediction MIP mode, a probability vector of the current image block is obtained according to at least two of the reconstructed blocks, prediction blocks and residual blocks of the surrounding image blocks, the current image block and a prediction mode probability model.

32. The device according to claim 30 or 31, characterized in that The processing unit is specifically used for: splicing or concatenating at least two of the reconstructed block, the predicted block, and the residual block of the surrounding image blocks to obtain a first data block; A probability vector of the current image block is obtained according to the first data block and the prediction mode probability model.

33. The device according to claim 32, characterized in that In the aspect of obtaining the probability vector of the current image block according to the first data block and the prediction mode probability model, the processing unit is specifically configured to: When the size of the first data block is the target size, inputting the first data block into the prediction mode probability model for processing to obtain a probability vector of the current image block, or; When the size of the first data block is not the target size, performing a size transformation operation on the first data block to obtain a second data block, the size of the second data block is equal to the target size, the size transformation operation includes at least one of scaling and transposition, and the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; The second data block is input into the prediction mode probability model for processing to obtain a probability vector of the current image block.

34. The device according to claim 32, characterized in that In the aspect of obtaining the probability vector of the current image block according to the first data block and the prediction mode probability model, the processing unit is specifically configured to: When the size of the first data block is the target size, inputting the first data block into the prediction mode probability model for processing to obtain a probability vector of the current image block, or; When the size of the first data block is not the target size, performing a size transformation operation on the first data block according to a size relationship between a first side length and a second side length of the first data block and / or a horizontal gradient and a vertical gradient of the current image block to obtain a second data block, the size of the second data block being the target size, the size transformation operation comprising at least one of scaling and transposition, the scaling comprising horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; The second data block is input into the prediction mode probability model for processing to obtain a probability vector of the current image block.

35. The device according to claim 34, characterized in that The resizing operation includes the scaling. In the aspect of performing the resizing operation on the first data block according to the size relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, the processing unit is specifically configured to: When the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively; When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of the horizontal gradient and an absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively; When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than the preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4 and 4N / M respectively; When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8 and 8N / M respectively; Wherein, the M and the N are positive integers.

36. The device according to claim 34, characterized in that The resizing operation includes at least one of the scaling and the transposition. In the aspect of performing the resizing operation on the first data block according to the size relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, the processing unit is specifically configured to: When the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively; When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of the horizontal gradient and an absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively; When the first side length M of the first data block is less than the second side length N of the first data block, performing the following operations on the first data block to obtain the second data block: When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is less than the preset threshold, performing the transposition and the scaling on the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4N / M, 4 respectively; When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is greater than or equal to the preset threshold, performing the transposition and the scaling on the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8N / M, 8 respectively; Wherein, the M and the N are positive integers.

37. The device according to any one of claims 33 to 36, characterized in that The encoding unit is specifically used for: When the resizing operation does not include the transposition, determining a first reference value according to the first identifier of the target intra-frame prediction mode and the probability vector; encoding the first reference value to obtain a code stream corresponding to the target intra-frame prediction mode; When the resizing operation includes the transposition, determining a second identifier according to a preset constant and the first identifier, and determining a second reference value according to the second identifier and the probability vector; The second reference value is encoded to obtain a code stream corresponding to the target intra-frame prediction mode.

38. A coding device, characterized in that: The device comprises: An acquisition unit, used for acquiring surrounding image blocks of a current image block; A processing unit, configured to perform splicing or cascading on the surrounding image blocks to obtain a first data block; perform a size transformation operation on the first data block to obtain a second data block, wherein the size transformation operation includes at least one of scaling and transposition, and the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; and input the second data block into a prediction mode probability model for processing to obtain a probability vector of the current image block, wherein multiple elements in the probability vector correspond one-to-one to multiple intra-frame prediction modes, and any element in the probability vector is used to represent the probability of using the intra-frame prediction mode corresponding to any element when predicting the current image block; A determination unit, configured to determine a target intra-frame prediction mode of the current image block; The encoding unit is used to encode the target intra-frame prediction mode into a bitstream according to the target intra-frame prediction mode and the probability vector.

39. The device according to claim 38, characterized in that In the aspect of performing a size transformation operation on the first data block to obtain the second data block, the processing unit is specifically configured to: When the target intra prediction mode is a non-matrix weighted intra prediction MIP mode and the size of the first data block is not the target size, the size transformation operation is performed on the first data block to obtain the second data block, and the size of the second data block is equal to the target size.

40. The device according to claim 38, characterized in that In the aspect of performing a size transformation operation on the first data block to obtain the second data block, the processing unit is specifically configured to: When the target intra-frame prediction mode is a non-matrix weighted intra-frame prediction MIP mode, and the size of the first data block is not the target size, the size transformation operation is performed on the first data block according to the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, and the size of the second data block is equal to the target size.

41. The device according to claim 39 or 40, characterized in that The processing unit is also used for: When the target intra prediction mode is a non-matrix weighted intra prediction MIP mode and the size of the first data block is equal to the target size, the first data block is input into the prediction mode probability model for processing to obtain a probability vector of the current image block.

42. The device according to claim 41, characterized in that The resizing operation includes the scaling, and in the aspect of performing the resizing operation on the first data block according to the size relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, the processing unit is specifically configured to: When the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively; When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of the horizontal gradient and an absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively; When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than the preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4 and 4N / M respectively; When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8 and 8N / M respectively; Wherein, the M and the N are positive integers.

43. The device according to claim 41, characterized in that The resizing operation includes the scaling, and in the aspect of performing the resizing operation on the first data block according to the size relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, the processing unit is specifically configured to: When the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively; When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of the horizontal gradient and an absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively; When the first side length M of the first data block is less than the second side length N of the first data block, performing the following operations on the first data block to obtain the second data block: When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is less than the preset threshold, performing the transposition and the scaling on the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4N / M, 4 respectively; When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is greater than or equal to the preset threshold, performing the transposition and the scaling on the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8N / M, 8 respectively; Wherein, the M and the N are positive integers.

44. The device according to claim 41, characterized in that The encoding unit is specifically used for: When the resizing operation does not include the transposition, determining a first reference value according to the first identifier of the target intra-frame prediction mode and the probability vector; encoding the first reference value to obtain a code stream corresponding to the target intra-frame prediction mode; When the resizing operation includes the transposition, determining a second identifier according to a preset constant and the first identifier, and determining a second reference value according to the second identifier and the probability vector; The second reference value is encoded to obtain a code stream corresponding to the target intra-frame prediction mode.

45. A decoding device, characterized in that: The device comprises: An acquisition unit, configured to acquire a code stream corresponding to a current image block, and at least two of a reconstructed block, a predicted block, and a residual block of image blocks surrounding the current image block; a processing unit, configured to obtain a probability vector of the current image block according to at least two of the reconstructed block, the predicted block, and the residual block of the surrounding image blocks and a prediction mode probability model, wherein a plurality of elements in the probability vector correspond one-to-one to a plurality of intra-frame prediction modes, and any element in the probability vector is used to represent a probability of adopting the intra-frame prediction mode corresponding to any element when predicting the current image block; A decoding unit is used to determine a target intra-frame prediction mode of the current image block according to the code stream and the probability vector, and determine a prediction block of the current image block according to the target intra-frame prediction mode.

46. ​​The device according to claim 45, characterized in that The processing unit is specifically used for: splicing or concatenating at least two of the reconstructed block, the predicted block, and the residual block of the surrounding image blocks to obtain a first data block; A probability vector of the current image block is obtained according to the first data block and the prediction mode probability model.

47. The device according to claim 46, characterized in that In the aspect of obtaining the probability vector of the current image block according to the first data block and the prediction mode probability model, the processing unit is specifically configured to: When the size of the first data block is the target size, inputting the first data block into the prediction mode probability model for processing to obtain a probability vector of the current image block, or; When the size of the first data block is not the target size, performing a size transformation operation on the first data block to obtain a second data block, the size of the second data block is equal to the target size, the size transformation operation includes at least one of scaling and transposition, and the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; The second data block is input into the prediction mode probability model for processing to obtain a probability vector of the current image block.

48. The device according to claim 46, characterized in that In the aspect of obtaining the probability vector of the current image block according to the first data block and the prediction mode probability model, the processing unit is specifically configured to: When the size of the first data block is the target size, inputting the first data block into the prediction mode probability model for processing to obtain a probability vector of the current image block, or; When the size of the first data block is not the target size, a size transformation operation is performed on the first data block according to a size relationship between a first side length and a second side length of the first data block, and / or a horizontal gradient and a vertical gradient of the current image block, so as to obtain a second data block, wherein the first side length and the second side length are lengths of two mutually perpendicular sides in the first data block, and the size of the second data block is the target size, and the size transformation operation includes at least one of scaling and transposition, and the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; The second data block is input into the prediction mode probability model for processing to obtain a probability vector of the current image block.

49. The device according to claim 48, characterized in that The resizing operation includes the scaling. In the aspect of performing the resizing operation on the first data block according to the size relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, the processing unit is specifically configured to: When the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively; When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of the horizontal gradient and an absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively; When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than the preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4 and 4N / M respectively; When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8 and 8N / M respectively; Wherein, the M and the N are positive integers.

50. The device according to claim 48, characterized in that The resizing operation includes at least one of the scaling and the transposition. In the aspect of performing the resizing operation on the first data block according to the size relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, the processing unit is specifically configured to: When the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively; When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of the horizontal gradient and an absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively; When the first side length M of the first data block is less than the second side length N of the first data block, performing the following operations on the first data block to obtain the second data block: When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is less than the preset threshold, performing the transposition and the scaling on the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4N / M, 4 respectively; When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is greater than or equal to the preset threshold, performing the transposition and the scaling on the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8N / M, 8 respectively; Wherein, the M and the N are positive integers.

51. The device according to any one of claims 47 to 50, characterized in that The decoding unit is specifically used for: Decoding the code stream to obtain a target reference value; When the resizing operation does not include the transposition, determining the target intra prediction mode according to the target reference value and the probability vector; When the size transformation operation includes the transposition, a first identifier is determined according to the target reference value and the probability vector; when the intra-frame prediction mode corresponding to the first identifier is an angle prediction mode, the target intra-frame prediction mode is determined according to a preset constant and the first identifier; when the intra-frame prediction mode corresponding to the first identifier is a non-angle prediction mode, the target intra-frame prediction mode is determined according to the target reference value and the probability vector.

52. A decoding device, characterized in that: The device comprises: An acquisition unit, used to acquire a code stream corresponding to a current image block and surrounding image blocks of the current image block; A processing unit, configured to perform splicing or cascading on the surrounding image blocks to obtain a first data block; perform a size transformation operation on the first data block to obtain a second data block, wherein the size transformation operation includes at least one of scaling and transposition, and the scaling includes horizontal scaling, vertical scaling, or geometric scaling in both horizontal and vertical directions; and input the second data block into a prediction mode probability model for processing to obtain a probability vector of the current image block, wherein multiple elements in the probability vector correspond one-to-one to multiple intra-frame prediction modes, and any element in the probability vector is used to represent the probability of using the intra-frame prediction mode corresponding to any element when predicting the current image block; A decoding unit is used to determine a target intra-frame prediction mode of the current image block according to the code stream and the probability vector, and determine a prediction block of the current image block according to the target intra-frame prediction mode.

53. The device according to claim 52, characterized in that In the aspect of performing a size transformation operation on the first data block to obtain the second data block, the processing unit is specifically configured to: When the size of the first data block is not the target size, the size transformation operation is performed on the first data block to obtain the second data block, and the size of the second data block is equal to the target size.

54. The device according to claim 52, characterized in that In the aspect of performing a size transformation operation on the first data block to obtain the second data block, the processing unit is specifically configured to: When the size of the first data block is not the target size, the size transformation operation is performed on the first data block according to the size relationship between the first side length and the second side length of the first data block, and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, and the size of the second data block is equal to the target size.

55. The device according to claim 53 or 54, characterized in that The processing unit is also used for: When the size of the first data block is equal to the target size, the first data block is input into the prediction mode probability model for processing to obtain a probability vector of the current image block.

56. The device according to claim 55, characterized in that The resizing operation includes the scaling. In the aspect of performing the resizing operation on the first data block according to the size relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, the processing unit is specifically configured to: When the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively; When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of the horizontal gradient and an absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively; When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than the preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4 and 4N / M respectively; When the first side length M of the first data block is less than the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is greater than or equal to the preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8 and 8N / M respectively; Wherein, the M and the N are positive integers.

57. The device according to claim 55, characterized in that The resizing operation includes at least one of the scaling and the transposition. In the aspect of performing the resizing operation on the first data block according to the size relationship between the first side length and the second side length of the first data block and / or the horizontal gradient and the vertical gradient of the current image block to obtain the second data block, the processing unit is specifically configured to: When the first side length M of the first data block is greater than or equal to the second side length N of the first data block, and the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient is less than a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4M / N, 4 respectively; When a first side length M of the first data block is greater than or equal to a second side length N of the first data block, and a sum of an absolute value of the horizontal gradient and an absolute value of the vertical gradient is greater than or equal to a preset threshold, scaling the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8M / N, 8 respectively; When the first side length M of the first data block is less than the second side length N of the first data block, performing the following operations on the first data block to obtain the second data block: When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is less than the preset threshold, performing the transposition and the scaling on the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 4N / M, 4 respectively; When the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the first data block is greater than or equal to the preset threshold, performing the transposition and the scaling on the first data block to obtain the second data block, wherein the first side length and the second side length of the second data block are 8N / M, 8 respectively; Wherein, the M and the N are positive integers.

58. The device according to claim 57, characterized in that The decoding unit is specifically used for: Decoding the code stream to obtain a target reference value; When the resizing operation does not include the transposition, determining the target intra prediction mode according to the target reference value and the probability vector; When the size transformation operation includes the transposition, a first identifier is determined according to the target reference value and the probability vector; when the intra-frame prediction mode corresponding to the first identifier is an angle prediction mode, the target intra-frame prediction mode is determined according to a preset constant and the first identifier; when the intra-frame prediction mode corresponding to the first identifier is a non-angle prediction mode, the target intra-frame prediction mode is determined according to the target reference value and the probability vector.

59. An encoder (20), characterized in that Comprising processing circuitry for performing a method according to any one of claims 1 to 15.

60. A decoder (30), characterized in that Comprising processing circuitry for performing a method according to any one of claims 16 to 29.

61. An encoder, characterized in that include: one or more processors; A non-transitory computer-readable storage medium, coupled to the processor and storing a program executed by the processor, wherein the program, when executed by the processor, causes the encoder to perform the method according to any one of claims 1 to 15.

62. A decoder, characterized in that include: one or more processors; A non-transitory computer-readable storage medium, coupled to the processor and storing a program executed by the processor, wherein the program, when executed by the processor, causes the decoder to perform the method according to any one of claims 16 to 29.

63. A computer program product, characterized in that The method comprises a program code for executing the method according to any one of claims 1 to 29 when the program code is executed on a computer or a processor.

64. A non-transitory computer-readable storage medium, characterized in that: The method comprises a program code for executing the method according to any one of claims 1 to 29 when the program code is executed by a computer device.

Images