Image prediction method and device and computer readable storage medium

Abstract

The invention provides a prediction method and device and a computer readable storage medium, and the method comprises the steps: calculating an airspace continuity parameter of a given image region, and determining intra-frame sub-block division information which is used for indicating whether a coding block uses intra-frame sub-block division or not; and selecting a prediction mode with the minimum prediction cost to predict the current coding block according to the indication of the intra-frame sub-block division information. Unnecessary intra-frame sub-block division in an image area with weak spatial correlation is avoided, and the calculation complexity is reduced, so that the coding efficiency is improved.

Description

Image prediction method, device and computer-readable storage medium Technical Field

[0001] The embodiments of the present disclosure relate to, but are not limited to, the field of video data processing technology, and in particular to a video encoding and decoding method, device, and storage medium. Background Art

[0002] Digital video compression technology primarily compresses massive amounts of digital video data for easier transmission and storage. With the surge in Internet video usage and increasing demand for higher-quality video, while existing digital video compression standards can save significant amounts of video data, there is a continued need for better digital video compression technologies to reduce bandwidth and traffic pressures associated with digital video transmission, enabling more efficient video encoding, decoding, transmission, and storage.

[0003] In order to provide the best video data compression results, it is often necessary to try encoding and optimize under a variety of specific available configuration schemes. Therefore, under the premise of meeting the requirements of video data playback and transmission, in the field of coding technology, on the one hand, we need to seek better video compression technology solutions, and on the other hand, we also need to take into account the encoding and decoding efficiency.

[0004] Summary of the Invention

[0005] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0006] The present disclosure provides an image prediction method, including:

[0007] Obtaining intra-frame sub-block division information, when the intra-frame sub-block division information indicates that the current block uses intra-frame sub-block division,

[0008] Based on at least one intra-frame sub-block partitioning, determining at least one first intra-frame prediction mode and a prediction cost corresponding to the at least one first intra-frame prediction mode,

[0009] determining at least one first other prediction mode and a prediction cost corresponding to the at least one first other prediction mode,

[0010] The intra-frame sub-block division information is obtained by calculating a spatial domain continuity parameter, and the intra-frame sub-block division information is written into a bitstream.

[0011] An embodiment of the present disclosure further provides an image prediction device, comprising a processor and a memory storing a computer program that can be run on the processor, wherein when the processor executes the computer program, the image prediction method as described in any embodiment of the present disclosure is implemented.

[0012] An embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the image prediction method as described in any embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] FIG1 is a schematic block diagram of a video encoding and decoding system according to an embodiment of the present application;

[0014] FIG2 is a schematic block diagram of a video encoder according to an embodiment of the present application;

[0015] FIG3 is a schematic block diagram of a video decoder according to an embodiment of the present application;

[0016] FIG4A is a schematic flowchart of an image prediction method provided in one embodiment of the present application;

[0017] FIG4B is a flow chart of calculating spatial continuity parameters according to an embodiment of the present application;

[0018] FIG5 is a schematic flow chart of an image prediction method provided in an embodiment of the present application;

[0019] Figure 6 is a schematic flowchart of an image prediction method provided in one embodiment of the present application. DETAILED DESCRIPTION

[0020] The present disclosure can be applied to the field of image coding and decoding, the field of video coding and decoding, the field of hardware video coding and decoding, the field of dedicated circuit video coding and decoding, the field of real-time video coding and decoding, etc. For example, the solution of the present disclosure can be combined with an audio and video coding standard (AVS), such as the H.264 / audio video coding (AVC) standard, the H.265 / high efficiency video coding (HEVC) standard, and the H.266 / versatile video coding (VVC) standard. Alternatively, the solution of the present disclosure can be combined with other proprietary or industry standards and operate, and the standards include ITU-TH.261, ISO / IEC MPEG-1 Visual, ITU-TH.262 or ISO / IEC MPEG-2 Visual, ITU-TH.263, ISO / IEC MPEG-4 Visual, ITU-TH.264 (also known as ISO / IEC MPEG-4 AVC), including scalable video coding (SVC) and multi-view video coding (MVC) extensions. It should be understood that the techniques of this disclosure are not limited to any particular codec standard or technology.

[0021] For ease of understanding, the video encoding and decoding system involved in the embodiment of the present disclosure is first introduced with reference to FIG1 .

[0022] FIG1 is a schematic block diagram of a video encoding and decoding system according to an embodiment of the present disclosure. It should be noted that FIG1 is merely an example, and the video encoding and decoding system according to an embodiment of the present disclosure includes but is not limited to that shown in FIG1. ​​As shown in FIG1, the video encoding and decoding system 100 includes an encoding device 110 and a decoding device 120. The encoding device is used to encode (which can be understood as compressing) video data to generate a code stream, and transmit the code stream to a decoding device. The decoding device decodes the code stream generated by the encoding device to obtain decoded video data.

[0023] The encoding device 110 of the embodiment of the present disclosure can be understood as a device with a video encoding function, and the decoding device 120 can be understood as a device with a video decoding function, that is, the embodiment of the present disclosure includes a wider range of devices for the encoding device 110 and the decoding device 120, such as smartphones, desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, televisions, cameras, display devices, digital media players, video game consoles, car computers, etc.

[0024] In some embodiments, the encoding device 110 may transmit the encoded video data (eg, a code stream) to the decoding device 120 via a channel 130. The channel 130 may include one or more media and / or devices capable of transmitting the encoded video data from the encoding device 110 to the decoding device 120.

[0025] In one example, the channel 130 includes one or more communication media that enable the encoding device 110 to transmit the encoded video data directly to the decoding device 120 in real time. In this example, the encoding device 110 can modulate the encoded video data according to a communication standard and transmit the modulated video data to the decoding device 120. The communication media includes wireless communication media, such as radio frequency spectrum. Optionally, the communication media may also include wired communication media, such as one or more physical transmission lines.

[0026] In another example, channel 130 includes a storage medium that can store the video data encoded by encoding device 110. The storage medium includes various locally accessible data storage media, such as optical disks, DVDs, and flash memories. In this example, decoding device 120 can retrieve the encoded video data from the storage medium.

[0027] In another example, the channel 130 may include a storage server that can store the video data encoded by the encoding device 110. In this example, the decoding device 120 can download the stored encoded video data from the storage server. Alternatively, the storage server can store the encoded video data and transmit the encoded video data to the decoding device 120, such as a web server (e.g., for a website), a file transfer protocol (FTP) server, etc.

[0028] In some embodiments, the encoding device 110 includes a video encoder 112 and an output interface 113. The output interface 113 may include a modulator / demodulator (modem) and / or a transmitter.

[0029] In some embodiments, the encoding device 110 may further include a video source 111 in addition to the video encoder 112 and the input interface 113 .

[0030] The video source 111 may include at least one of a video acquisition device (eg, a video camera), a video archive, a video input interface, and a computer graphics system, wherein the video input interface is used to receive video data from a video content provider, and the computer graphics system is used to generate video data.

[0031] The video encoder 112 encodes the video data from the video source 111 to generate a bitstream. The video data may include one or more pictures or a sequence of pictures. The bitstream contains the coding information of the picture or picture sequence in the form of a bitstream. The coding information may include the coded picture data and associated data. The associated data may include a sequence parameter set (SPS), a picture parameter set (PPS), and other syntax structures. The SPS may contain parameters that apply to one or more sequences. The PPS may contain parameters that apply to one or more pictures. The syntax structure refers to a set of zero or more syntax elements arranged in a specified order in the bitstream.

[0032] The video encoder 112 transmits the encoded video data directly to the decoding device 120 via the output interface 113. The encoded video data may also be stored in a storage medium or a storage server for subsequent reading by the decoding device 120.

[0033] In some embodiments, the decoding device 120 includes an input interface 121 and a video decoder 122 .

[0034] In some embodiments, the decoding device 120 may further include a display device 123 in addition to the input interface 121 and the video decoder 122 .

[0035] The input interface 121 includes a receiver and / or a modem and can receive the encoded video data via the channel 130 .

[0036] The video decoder 122 is configured to decode the encoded video data to obtain decoded video data, and transmit the decoded video data to the display device 123 .

[0037] The decoded video data is displayed on the display device 123. The display device 123 may be integrated with the decoding device 120 or external to the decoding device 120. The display device 123 may include various display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.

[0038] In addition, FIG1 is only an example, and the technical solutions of the embodiments of the present disclosure are not limited to FIG1 . For example, the technology of the present disclosure can also be applied to unilateral video encoding or unilateral video decoding.

[0039] The following introduces the video coding framework involved in the embodiments of the present disclosure.

[0040] FIG2 is a schematic block diagram of a video encoder according to an embodiment of the present disclosure. It should be understood that the video encoder 200 can be used to perform lossy compression or lossless compression on an image. The lossless compression can be visually lossless or mathematically lossless.

[0041] The video encoder 200 can be applied to image data in a luminance and chrominance (YCbCr, YUV) format. For example, the YUV ratio can be 4:2:0, 4:2:2, or 4:4:4, where Y represents brightness (Luma), Cb (U) represents blue chrominance, Cr (V) represents red chrominance, and U and V represent chrominance (Chroma) for describing color and saturation. For example, in terms of color format, 4:2:0 means that every 4 pixels have 4 luminance components and 2 chrominance components (YYYYCbCr), 4:2:2 means that every 4 pixels have 4 luminance components and 4 chrominance components (YYYYCbCrCbCr), and 4:4:4 means full pixel display (YYYYCbCrCbCrCbCrCbCr).

[0042] For example, the video encoder 200 reads video data, and for each frame of the video data, divides the frame into a number of coding tree units (CTUs). In some examples, CTB may be referred to as a "tree block", "largest coding unit" (LCU) or "coding tree block" (CTB). Each CTU may be associated with a pixel block of equal size within the image. Each pixel may correspond to a luminance (luminance or luma) sample and two chrominance (chroma) samples. Therefore, each CTU may be associated with a luminance sample block and two chroma sample blocks. The size of a CTU is, for example, 128×128, 64×64, 32×32, etc. A CTU can be further divided into a number of coding units (CUs) for encoding. The CU may be a rectangular block or a square block. A CU can be further divided into prediction units (PUs) and transform units (TUs), allowing for separation of coding, prediction, and transform, and greater flexibility in processing. In one example, a CTU is divided into CUs using a quadtree, and a CU is divided into TUs and PUs using a quadtree.

[0043] The video encoder and video decoder can support various PU sizes. Assuming that the size of a particular CU is 2N×2N, the video encoder and video decoder can support PU sizes of 2N×2N or N×N for intra-frame prediction, and support symmetric PUs of 2N×2N, 2N×N, N×2N, N×N, or similar sizes for inter-frame prediction. The video encoder and video decoder can also support asymmetric PUs of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter-frame prediction.

[0044] In some embodiments, as shown in FIG2 , the video encoder 200 may include a prediction unit 210, a residual unit 220, a transform / quantization unit 230, an inverse transform / quantization unit 240, a reconstruction unit 250, a loop filter unit 260, a decoded image buffer 270, and an entropy coding unit 280. It should be noted that the video encoder 200 may include more, fewer, or different functional components.

[0045] Optionally, in the present disclosure, the current block may be referred to as the current coding unit (CU) or the current prediction unit (PU), etc. The prediction block may also be referred to as a predicted image block or an image prediction block, and the reconstructed image block may also be referred to as a reconstructed block or an image reconstructed image block.

[0046] In some embodiments, the prediction unit 210 includes an inter-frame prediction unit 211 and an intra-frame estimation unit 212. Because there is a strong correlation between adjacent pixels in a video frame, intra-frame prediction is used in video coding and decoding technologies to eliminate spatial redundancy between adjacent pixels. Because there is a strong similarity between adjacent frames in a video, inter-frame prediction is used in video coding and decoding technologies to eliminate temporal redundancy between adjacent frames, thereby improving coding efficiency.

[0047] The inter-frame prediction unit 211 is used for inter-frame prediction. Inter-frame prediction includes motion estimation and motion compensation. It can refer to image information from different frames. Inter-frame prediction uses motion information to find a reference block from a reference frame and generates a prediction block based on the reference block to eliminate temporal redundancy. The frames used for inter-frame prediction can be P-frames and / or B-frames. P-frames refer to forward-predicted frames, while B-frames refer to bidirectionally predicted frames. Inter-frame prediction uses motion information to find a reference block from a reference frame and generates a prediction block based on the reference block. Motion information includes the reference frame list in which the reference frame is located, the reference frame index, and the motion vector. The motion vector can be integer pixel or fractional pixel. If the motion vector is fractional pixel, interpolation filtering is required to generate the required fractional pixel block in the reference frame. Here, the integer pixel or fractional pixel block in the reference frame found based on the motion vector is called the reference block. Some technologies directly use the reference block as the prediction block, while others further process the reference block to generate the prediction block. Further processing the reference block to generate the prediction block can also be understood as using the reference block as the prediction block and then processing the prediction block to generate the prediction block for the current block.

[0048] The intra-frame estimation unit 212 only refers to the information of the same frame image to predict the pixel information in the current code image block to eliminate spatial redundancy. The frame used for intra-frame prediction can be an I frame.

[0049] Intra-frame prediction has multiple prediction modes. For example, the H-series international digital video coding standard H.264 / AVC has eight angular prediction modes and one non-angular prediction mode. H.265 / HEVC expands this to 33 angular prediction modes and two non-angular prediction modes. HEVC uses planar, DC, and 33 angular modes for a total of 35 intra-frame prediction modes. VVC uses planar, DC, and 65 angular modes for a total of 67 intra-frame prediction modes.

[0050] It should be noted that with the increase of angle modes, intra-frame prediction will be more accurate and more in line with the needs of high-definition and ultra-high-definition digital video development.

[0051] The residual unit 220 may generate a residual block for the CU based on the pixel blocks of the CU and the prediction blocks of the PUs of the CU. For example, the residual unit 220 may generate the residual block for the CU such that each sample in the residual block has a value equal to the difference between the sample in the pixel blocks of the CU and the corresponding sample in the prediction blocks of the PUs of the CU.

[0052] The transform / quantization unit 230 may quantize the transform coefficients. The transform / quantization unit 230 may quantize the transform coefficients associated with the TUs of a CU based on a quantization parameter (QP) value associated with the CU. The video encoder 200 may adjust the degree of quantization applied to the transform coefficients associated with the CU by adjusting the QP value associated with the CU.

[0053] The inverse transform / quantization unit 240 may apply inverse quantization and inverse transform, respectively, to the quantized transform coefficients to reconstruct a residual block from the quantized transform coefficients.

[0054] Reconstruction unit 250 may add samples of the reconstructed residual block to corresponding samples of one or more prediction blocks generated by prediction unit 210 to generate a reconstructed image block associated with the TU. By reconstructing the sample blocks of each TU of a CU in this manner, video encoder 200 can reconstruct the pixel blocks of the CU.

[0055] The loop filter unit 260 is used to process the inverse transformed and inverse quantized pixels to compensate for distortion information and provide a better reference for subsequent coded pixels. For example, it can perform a deblocking filtering operation to reduce the blocking effect of pixel blocks associated with the CU.

[0056] In some embodiments, the loop filtering unit 260 includes a deblocking filtering unit and a sample adaptive offset / adaptive loop filtering (SAO / ALF) unit, wherein the deblocking filtering unit is used to remove blocking effects, and the SAO / ALF unit is used to remove ringing effects.

[0057] The decoded image buffer 270 may store the reconstructed pixel blocks. The inter prediction unit 211 may use a reference image containing the reconstructed pixel blocks to perform inter prediction on PUs of other images. In addition, the intra estimation unit 212 may use the reconstructed pixel blocks in the decoded image buffer 270 to perform intra prediction on other PUs in the same image as the CU.

[0058] The entropy coding unit 280 may receive the quantized transform coefficients from the transform / quantization unit 230. The entropy coding unit 280 may perform one or more entropy coding operations on the quantized transform coefficients to generate entropy-coded data.

[0059] FIG3 is a schematic block diagram of a video decoder according to an embodiment of the present disclosure.

[0060] 3 , the video decoder 300 includes an entropy decoding unit 310, a prediction unit 320, an inverse quantization / transformation unit 330, a reconstruction unit 340, a loop filter unit 350, and a decoded picture buffer 360. It should be noted that the video decoder 300 may include more, fewer, or different functional components.

[0061] The video decoder 300 may receive a bitstream. The entropy decoding unit 310 may parse the bitstream to extract syntax elements from the bitstream. As part of parsing the bitstream, the entropy decoding unit 310 may parse the entropy-encoded syntax elements in the bitstream. The prediction unit 320, the inverse quantization / transform unit 330, the reconstruction unit 340, and the loop filter unit 350 may decode the video data based on the syntax elements extracted from the bitstream, thereby generating decoded video data.

[0062] In some embodiments, the prediction unit 320 includes an inter-frame prediction unit 321 and an intra-frame estimation unit 322 .

[0063] Intra estimation unit 322 may perform intra prediction to generate a prediction block for a PU. Intra estimation unit 322 may use an intra prediction mode to generate a prediction block for the PU based on pixel blocks of spatially neighboring PUs. Intra estimation unit 322 may also determine the intra prediction mode for the PU based on one or more syntax elements parsed from a codestream.

[0064] The inter-frame prediction unit 321 may construct a first reference picture list (List 0) and a second reference picture list (List 1) based on syntax elements parsed from the codestream. In addition, if a PU is encoded using inter-frame prediction, the entropy decoding unit 310 may parse the motion information of the PU. The inter-frame prediction unit 321 may determine one or more reference blocks for the PU based on the motion information of the PU. The inter-frame prediction unit 321 may generate a prediction block for the PU based on the one or more reference blocks of the PU.

[0065] The inverse quantization / transform unit 330 may inversely quantize (ie, dequantize) the transform coefficients associated with the TU. The inverse quantization / transform unit 330 may use the QP value associated with the CU of the TU to determine the degree of quantization.

[0066] After inverse quantizing the transform coefficients, the inverse quantization / transform unit 330 may apply one or more inverse transforms to the inverse quantized transform coefficients in order to generate a residual block associated with the TU.

[0067] The reconstruction unit 340 uses the residual block associated with the TU of the CU and the prediction block of the PU of the CU to reconstruct the pixel block of the CU. For example, the reconstruction unit 340 can add samples of the residual block to corresponding samples of the prediction block to reconstruct the pixel block of the CU to obtain a reconstructed image block.

[0068] The loop filtering unit 350 may perform a deblocking filtering operation to reduce blocking artifacts of pixel blocks associated with a CU.

[0069] The video decoder 300 may store the reconstructed image of the CU in the decoded image buffer 360. The video decoder 300 may use the reconstructed image in the decoded image buffer 360 as a reference image for subsequent prediction, or transmit the reconstructed image to a display device for presentation.

[0070] The basic process of video encoding and decoding is as follows: At the encoder end, a frame of image is divided into blocks. For the current block, the prediction unit 210 uses intra-frame prediction or inter-frame prediction to generate a prediction block for the current block. The residual unit 220 calculates a residual block based on the predicted block and the original block of the current block. This residual block is the difference between the predicted block and the original block of the current block. This residual block is also called residual information. This residual block undergoes transformation and quantization by the transform / quantization unit 230, removing information that is insensitive to the human eye and eliminating visual redundancy. Optionally, the residual block before transformation and quantization by the transform / quantization unit 230 is called a time-domain residual block, and the time-domain residual block after transformation and quantization by the transform / quantization unit 230 is called a frequency residual block or a frequency-domain residual block. The entropy coding unit 280 receives the quantized change coefficients output by the change quantization unit 230 and performs entropy coding on these quantized change coefficients to output a bitstream. For example, the entropy coding unit 280 can eliminate character redundancy based on the target context model and probability information of the binary bitstream.

[0071] At the decoding end, the entropy decoding unit 310 can parse the code stream to obtain the prediction information, quantization coefficient matrix, etc. of the current block. The prediction unit 320 uses intra-frame prediction or inter-frame prediction for the current block based on the prediction information to generate a prediction block for the current block. The inverse quantization / transformation unit 330 uses the quantization coefficient matrix obtained from the code stream to inverse quantize and inverse transform the quantization coefficient matrix to obtain a residual block. The reconstruction unit 340 adds the prediction block and the residual block to obtain a reconstructed block. The reconstructed blocks constitute a reconstructed image, and the loop filtering unit 350 performs loop filtering on the reconstructed image based on the image or block to obtain a decoded image. The encoding end also requires similar operations as the decoding end to obtain a decoded image. The decoded image can also be called a reconstructed image, and the reconstructed image can be used as a reference frame for inter-frame prediction for subsequent frames.

[0072] It should be noted that the block division information determined by the encoder, as well as mode information or parameter information such as prediction, transform, quantization, entropy coding, and loop filtering, etc., are carried in the bitstream when necessary. The decoder parses the bitstream and analyzes the existing information to determine the same block division information, prediction, transform, quantization, entropy coding, loop filtering, etc. mode information or parameter information as the encoder, thereby ensuring that the decoded image obtained by the encoder and the decoder are identical.

[0073] The above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of the framework or process may be optimized. The present disclosure is applicable to the basic process of the video codec under the block-based hybrid coding framework, but is not limited to the framework and process.

[0074] Currently, common video codec standards all use a block-based hybrid coding framework. Each video frame is divided into square maximum coding units (LCUs) or coding tree units (CTUs) of the same size (e.g., 128x128, 64x64, etc.). Each LCU or CTU can be divided into rectangular coding units (CUs) according to a set of rules.

[0075] In H.264 / AVC, the input image is divided into fixed-size blocks as the basic encoding unit, called macroblocks (MBs). These blocks consist of one luminance block and two chrominance blocks, with the luminance block size being 16×16. If 4:2:0 sampling is used, the chrominance block size is half the size of the luminance block. During the prediction phase, macroblocks are further divided into smaller blocks for prediction, depending on the prediction mode. In intra-frame prediction, macroblocks can be divided into 16×16, 8×8, or 4×4 blocks, with each block performing intra-frame prediction separately. During the transform and quantization phase, macroblocks are divided into 4×4 or 8×8 blocks, and the prediction residuals in each block are transformed and quantized to obtain quantized coefficients.

[0076] Compared with H.264 / AVC, H.265 / HEVC has taken improvements in multiple coding links. In H.265 / HEVC, an image is divided into coding tree units (CTUs), which are the basic units of coding (corresponding to macroblocks in H.264 / AVC). A CTU contains a luma coding tree block (CTB) and two chroma coding tree blocks. The maximum size of a CU in the H.265 / HEVC standard is generally 64×64. To accommodate a variety of video content and video features, the CTU is iteratively divided into a series of coding units (CUs) using a quadtree (QT) method. The CU is the basic unit for intra / inter coding. A CU contains a luma coding block (CB) and two chroma coding blocks and related syntax structures. The maximum CU size is CTU, and the minimum CU size is 8×8. The leaf CUs obtained by coding tree partitioning can be divided into three types based on the prediction method: intra CUs (for intra-frame prediction), inter CUs (for inter-frame prediction), and skipped CUs. Skipped CUs can be considered a special case of inter CUs, lacking motion information or prediction residual information. A leaf CU contains one or more prediction units (PUs). H.265 / HEVC supports PUs ranging from 4×4 to 64×64, with eight partitioning modes. For intra coding mode, two possible partitioning modes are Part_2Nx2N and Part_NxN. For the prediction residual signal, the CU is partitioned into transform units (TUs) using a prediction residual quadtree. A TU consists of one luma transform block (TB) and two chroma transform blocks. Only square partitioning is allowed, dividing a CB into one or four PBs. The same TU uses the same transform and quantization process, supporting sizes from 4×4 to 32×32. Different from previous coding standards, in inter-frame prediction, TB can cross the boundary of PB to further maximize the coding efficiency of inter-frame coding.

[0077] In H.266 / VVC, the video encoding image is first divided into coding tree units (CTUs), similar to those in H.265 / HEVC, but with a maximum size increased from 64×64 to 128×128. H.266 / VVC introduces quadtree and nested multi-type tree (MTT) partitioning. MTT includes binary tree (BT) and ternary tree (TT). It unifies the concepts of CU, PU, ​​and TU in H.265 / HEVC and supports more flexible CU partitioning structures. CTUs are partitioned using a quadtree structure, with leaf nodes further partitioned using MTT. The leaf nodes of the MTT become coding units (CUs). When a CU is no larger than the maximum transform unit (64×64), it is not further partitioned for subsequent prediction and transform. In most cases, CUs, PUs, and TUs are of the same size. Considering the different characteristics of luma and chroma and the parallelism required for specific implementations, H.266 / VVC allows chroma to use a separate partitioning tree, rather than aligning it with the luma partitioning tree. In H.266 / VVC, the chroma division of I frames uses a chroma separation tree, while the chroma division of P frames and B frames is consistent with the luminance division.

[0078] Building on this, H.266 / VVC also introduces a technique called intra-frame sub-block partitioning. Intra-frame sub-block partitioning (ISP) is an intra-frame prediction technique based on sub-block partitioning. When using ISP mode, a coding block is further divided horizontally or vertically into multiple (e.g., 2 or 4) sub-blocks of equal size, with each sub-block decoded and reconstructed in sequence. For example, a 16x8 current block can be divided horizontally into 4 16x2 sub-blocks or vertically into 4 4x8 sub-blocks. A 4x8 block can be divided into 2 4x4 blocks or 2 2x8 sub-blocks. Note that the number of sub-blocks can be only 2 (for 4x8 and 8x4 sub-blocks). Since a sub-block contains at least 16 pixels, the 4x4 block is not further divided, and no corresponding syntax declaration is required on the encoder or decoder. The specific partitioning rules are shown in Table 3. Whether to partition horizontally or vertically is determined by the encoder's RDO criterion. In order to save encoding time, the divided intra-frame sub-blocks share the intra-frame prediction mode of the current block (that is, the sub-blocks obtained by dividing the current block have the same intra-frame prediction mode as the current block).

[0079] Exemplarily, in some embodiments, the image to be encoded is divided into a plurality of non-overlapping CTU blocks. Each CTU is processed in sequence according to the raster scan order, and the CTU is divided into several CUs in different ways. The main steps for determining the coding block division method are as follows: For a certain CTU, block division is performed using a conventional block division method (quadtree, ternary tree, binary tree). For a certain division method Split[i], intra-frame prediction of different prediction modes is performed on each CU respectively, and the optimal conventional intra-frame prediction mode bestRegIntraMode[i] and prediction cost bestRegIntraCost[i] are selected. If the block size of a certain CU meets the ISP condition, the CU is further ISP sub-divided to obtain several sub-blocks, and intra-frame prediction of different prediction modes is performed on each sub-block respectively, and the optimal prediction mode bestIspIntraMode[i] and prediction cost bestIspIntraCost[i] are selected. The optimal intra-frame prediction mode bestIntraMode[i] and prediction cost bestIntraCost[i] of the current division method are selected. Use other methods (such as inter-frame prediction) for prediction and select the optimal prediction mode bestOtherMode[i] and prediction cost bestOtherCost[i]. Compare bestRegIntraCost[i], bestIspIntraCost[i], and bestOtherCost[i] to select the optimal prediction mode bestMode[i] and prediction cost bestCost[i] for the current partitioning method. Traverse all block partitioning methods and select the block partitioning method Split[opt] and corresponding prediction mode bestMode[opt] that minimizes the prediction cost of the current CTU. Finally, predict the residual block according to the optimal block partitioning method, transform, quantize, and entropy encode the residual block, encode the prediction information such as the block partitioning mode and prediction mode, and output the bitstream.

[0080] It can be seen that among these achievable H.266 / VVC codec schemes, even if only quadtree partitioning is considered, there are 4 5 +4 4 +4 3 +4 2 +4 1 +4 0 = 1365 partitioning modes, far exceeding the 341 modes in H.265 / HEVC. Adding binary and ternary tree partitioning methods, the theoretical total number of possible partitioning methods reaches several thousand. Therefore, these feasible QTMT technical solutions result in H.266 / VVC's coding complexity far exceeding that of H.265 / HEVC. If intra-frame sub-block partitioning is also considered, the coding complexity increases further.

[0081] The image prediction method provided by the embodiment of the present disclosure is introduced below with reference to FIG4A .

[0082] Step S401: Obtain sub-block division information within a frame.

[0083] The intra-frame sub-block division information is used to indicate whether the current coding block uses intra-frame sub-block division.

[0084] It is understood that during the image encoding process, the encoder receives a video stream consisting of a series of image frames, performs video encoding on each frame in the video stream, and divides the image into blocks to obtain a current encoding block. In some embodiments, the current encoding block is also referred to as the current block, current image block, encoding block, current encoding unit, current block to be encoded, current image block to be encoded, etc. The current encoding block can also be divided into sub-blocks within a frame.

[0085] Furthermore, indication information may be used to indicate whether intra-frame sub-block division is performed on the current coding block. For example, a flag bit may be used to indicate the intra-frame sub-block division information. When the flag bit takes a first value, it indicates that intra-frame sub-block division is performed on the current block. When the flag bit takes a second value, it indicates that intra-frame sub-block division is not performed on the current block.

[0086] Furthermore, the intra-frame sub-block division information can be obtained by calculating a spatial continuity parameter. The spatial continuity parameter can represent the continuity of pixel changes in a given image region.

[0087] In some embodiments, if the spatial continuity parameter is greater than or equal to a certain threshold, the intra-frame sub-block partitioning information indicates that the current block uses intra-frame sub-block partitioning. Conversely, if the continuity parameter is less than a certain threshold, the intra-frame sub-block partitioning information indicates that the current block does not use intra-frame sub-block partitioning.

[0088] In other embodiments, if the spatial continuity parameter is greater than a certain threshold, the intra-frame sub-block partitioning information indicates that the current block uses intra-frame sub-block partitioning. Conversely, if the continuity parameter is less than or equal to a certain threshold, the intra-frame sub-block partitioning information indicates that the current block does not use intra-frame sub-block partitioning.

[0089] In some embodiments, the given image region may be a single image, a single stripe, a single coding tree unit, or a single coding block. In other embodiments, the given image region may be an image region consisting of multiple spatially adjacent stripes, multiple spatially adjacent coding tree units, or multiple spatially adjacent coding blocks. It is understood that the method for calculating the spatial continuity parameter for a given image region may be applicable to the above various possible given image regions, and the embodiments of the present disclosure do not limit this.

[0090] 4B , in the embodiments of the present disclosure, the manner of determining the spatial continuity parameter of a given image region includes but is not limited to the following manner:

[0091] The continuity parameter of the image region is determined by the gradient map, that is, the above step S401 includes the following steps S401-A1 and S401-A2:

[0092] S401-A1, determining a gradient map of a given image region, and a gradient map of the gradient map;

[0093] S401-A2: Determine the continuity parameter of the image region according to the gradient map of the gradient map.

[0094] The gradient map of an image region can reflect the changing trend of pixels in the image region. The gradient map of the gradient map of an image region can reflect the rate of change of pixels in the image region, that is, the continuity of change. Therefore, in this method, the continuity parameter of the image region is determined by calculating the gradient map of the gradient map of the gradient map of the image region.

[0095] Exemplarily, the gradient map Gmap of the image region is determined according to the following formula (1):

[0096] Gmap(x,y)=|I(x,y)-I(x+1,y)|+|I(x,y)-I(x,y+1)| (1)

[0097] Where I(x, y) is the pixel value of image region I at position (x, y), I(x+1, y) is the pixel value of image region I at position (x+1, y), I(x, y+1) is the pixel value of image region I at position (x, y+1), and Gmap(x, y) is the gradient value of the pixel point at position (x, y) in image region I.

[0098] Exemplarily, the gradient map GGmap of the gradient map of the image region is determined according to the following formula (2):

[0099] GGmap(x,y)=|Gmap(x,y)-Gmap(x+1,y)|+|Gmap(x,y)-Gmap(x,y+1)| (2)

[0100] Where Gmap(x+1,y) is the gradient value of the pixel at position (x+1,y) in image region I, and Gmap(x,y+1) is the gradient value of the pixel at position (x,y+1) in image region I. Gmap(x+1,y) and Gmap(x,y+1) can be determined according to the above formula (1). GGmap(x,y) is the gradient value of the gradient value of the pixel at position (x,y) in image region I.

[0101] The above formulas (1) and (2) show how to determine the gradient value and the gradient value of the gradient value of the pixel point (x, y) in the image region I. The gradient value of each pixel point in the image region I is determined by referring to the pixel point (x, y) for other pixels in the image region I, thereby obtaining the gradient map Gmap of the image region I. The above formula (2) is used to calculate the gradient value corresponding to each pixel point in the gradient map Gmap of the image region I, thereby obtaining the gradient map GGmap of the gradient map of the image region I.

[0102] It should be noted that the above formula (1) and formula (2) are only examples. The embodiments of the present disclosure can also adopt other methods to determine the gradient map of the image area and the gradient map of the gradient map. For example, the above formula (1) and formula (2) are deformed and the deformed formulas are used to determine the gradient map and the gradient map of the gradient map.

[0103] According to the above method, after the gradient map of the gradient map of the image region is determined, the continuity parameter of the image region is determined according to the gradient map GGmap of the gradient map.

[0104] The embodiment of the present disclosure does not limit the method of determining the continuity parameter of the image region according to the gradient map of the gradient map in the above S401-A2.

[0105] In one example, a median value of each gradient value in a gradient map of a gradient map of an image region is determined as a continuity parameter of the image region.

[0106] In another example, an average value of gradient values ​​in a gradient map of a gradient map of an image region is determined as the continuity parameter of the image region.

[0107] It's easy to understand that the fundamental principle of intra-frame sub-block division is to make finer divisions of the coding block based on spatial correlation. Since the resulting sub-blocks are highly correlated, they can further reduce the residual error of intra-frame prediction, thereby improving compression efficiency. Obviously, while intra-frame sub-block division improves compression rate, it also increases the number of block division possibilities, leading to more rate-distortion optimization calculations and significantly increasing coding complexity. For video images with weak spatial correlation, the performance gain of intra-frame sub-block division will be very limited, but it will still significantly increase coding complexity.

[0108] As can be seen, calculating the spatial continuity parameter to obtain intra-frame sub-block division information can avoid unnecessary intra-frame sub-block division in image regions with weak spatial correlation, reducing computational complexity and thus improving coding efficiency. Furthermore, in the above-mentioned method, determining the continuity parameter of an image region by determining the gradient map of the image region's gradient map provides a simple and accurate calculation method with low computational overhead, further improving coding efficiency.

[0109] Step S402 : when the intra subblock partitioning information indicates that the current block uses intra subblock partitioning, determining at least one first intra prediction mode and a prediction cost corresponding to the at least one first intra prediction mode based on at least one intra subblock partitioning.

[0110] For coding blocks that can be divided into sub-blocks within a frame, they can be further divided into multiple sub-blocks. It should be noted that the sub-block division method is not unique. In other words, there are multiple sub-block divisions for the current block. In order to perform predictive coding on the current block, it is necessary to consider the prediction conditions of each of the multiple sub-block divisions.

[0111] In some embodiments, based on a certain sub-block division, different first intra prediction modes are used to predict the current block. It is understood that the same first intra prediction mode is used to predict multiple sub-blocks of the current block. That is, based on a certain first intra prediction mode, intra prediction is performed sequentially on the sub-blocks of the current block to obtain predicted values ​​for all sample positions of the current block.

[0112] Exemplarily, the first intra-frame prediction mode that can be adopted includes, but is not limited to, a planar mode, a direct current mode, a multi-angle mode, a matrix weighted intra-frame prediction mode (Matrix weighted intra-frame prediction, MIP), and the like.

[0113] It can be understood that the prediction cost of each intra-frame prediction mode can be calculated for each block. Prediction cost is typically used to represent a comprehensive measure of the bitrate overhead and image distortion caused by encoding using a prediction mode. Common prediction costs include, but are not limited to, rate-distortion cost.

[0114] It can be seen that based on a certain sub-block division, when different first intra prediction modes are used to predict the current block, corresponding multiple prediction costs will also be obtained.

[0115] Step S403 : determining at least one first other prediction mode and a prediction cost corresponding to the at least one first other prediction mode.

[0116] For the current coding block, multiple other prediction modes may be used for prediction. In some embodiments, the other prediction modes include inter-frame prediction mode, combined inter and intra prediction (CIIP), and the like.

[0117] It is understood that intra-frame prediction is not prohibited in frames using inter-frame prediction. That is, for coding blocks in P-frames and B-frames, both intra-frame and inter-frame prediction modes need to be considered when determining their prediction modes.

[0118] Similarly, by predicting the current block using each other prediction mode, the prediction cost of that other prediction can be calculated. Prediction cost is typically used to represent a comprehensive measure of the bitrate overhead and image distortion introduced by encoding using a prediction mode. Common prediction costs include, but are not limited to, rate-distortion cost.

[0119] It can be seen that when different first other prediction modes are used to predict the current block, corresponding multiple prediction costs will also be obtained.

[0120] Step S404: determine the prediction mode with the minimum prediction cost among the at least one first intra prediction mode and the at least one first other prediction mode as the coding prediction mode of the current block, and use the prediction mode with the minimum cost to predict the current block.

[0121] It is easy to understand that in steps S402 and S403, the current block has been predicted using at least one first intra prediction mode and at least one first other prediction mode to obtain prediction costs corresponding to these prediction modes. It is easy to imagine that by comparing these prediction costs, the prediction mode and prediction cost actually used in the predictive coding can be determined, and the current block can be predicted based on the prediction mode.

[0122] In some embodiments, the prediction mode with the smallest prediction cost is determined as the prediction mode used in predictive encoding.

[0123] Step S405: writing the intra-frame sub-block division information into the bitstream.

[0124] The indication information may be used to indicate whether to perform intra-frame sub-block division on the current coding block. For example, a flag bit may be used to indicate the intra-frame sub-block division information, and the flag bit is written into the bitstream.

[0125] In some embodiments, the flag is a flag at the sequence level. When the flag takes the first value, it indicates that in the current image sequence, blocks in at least one image use intra-frame sub-block division. When the flag takes the second value, it indicates that all blocks in the image sequence do not use intra-frame sub-block division.

[0126] In some embodiments, the flag is a flag at the image level. When the flag takes a first value, it indicates that in the current image, blocks in at least one strip use intra-frame sub-block division. When the flag takes a second value, it indicates that all blocks in the image do not use intra-frame sub-block division.

[0127] In some embodiments, the flag is a flag at the slice level. When the flag takes the first value, it indicates that in the current slice, at least one block in the coding tree unit uses intra-frame sub-block division. When the flag takes the second value, it indicates that all blocks in the slice do not use intra-frame sub-block division.

[0128] In some embodiments, the flag is a flag at the coding tree level. When the flag takes the first value, it indicates that at least one block in the current coding tree unit uses intra-frame sub-block division. When the flag takes the second value, it indicates that all blocks in the coding tree unit do not use intra-frame sub-block division.

[0129] In some embodiments, the flag is a flag at the block level. When the flag takes a first value, it indicates that the current block uses intra-frame sub-block division. When the flag takes a second value, it indicates that the current block does not use intra-frame sub-block division.

[0130] 5 , an image prediction method provided by another embodiment of the present disclosure is introduced below.

[0131] Step S501: Obtaining sub-block division information within a frame

[0132] It is understood that during the image encoding process, the encoder receives a video stream consisting of a series of image frames, performs video encoding on each image frame in the video stream, and divides the image frame into blocks to obtain a current encoding block. In some embodiments, the current encoding block is also referred to as the current block, current image block, encoding block, current encoding unit, current block to be encoded, current image block to be encoded, etc. The current encoding block can also be further divided into sub-blocks within the frame.

[0133] Furthermore, indication information may be used to indicate whether intra-frame sub-block division is performed on the current coding block. For example, a flag bit may be used to indicate the intra-frame sub-block division information. When the flag bit takes a first value, it indicates that intra-frame sub-block division is performed on the current block. When the flag bit takes a second value, it indicates that intra-frame sub-block division is not performed on the current block.

[0134] Furthermore, the intra-frame sub-block division information can be obtained by calculating a spatial continuity parameter. The spatial continuity parameter can represent the continuity of pixel changes in a given image region.

[0135] In some embodiments, if the spatial continuity parameter is greater than or equal to a certain threshold, the intra-frame sub-block partitioning information indicates that the current block uses intra-frame sub-block partitioning. Conversely, if the continuity parameter is less than a certain threshold, the intra-frame sub-block partitioning information indicates that the current block does not use intra-frame sub-block partitioning.

[0136] In other embodiments, if the spatial continuity parameter is greater than a certain threshold, the intra-frame sub-block partitioning information indicates that the current block uses intra-frame sub-block partitioning. Conversely, if the continuity parameter is less than or equal to a certain threshold, the intra-frame sub-block partitioning information indicates that the current block does not use intra-frame sub-block partitioning.

[0137] In some embodiments, the given image region may be a single image, a single stripe, a single coding tree unit, or a single coding block. In other embodiments, the given image region may be an image region consisting of multiple spatially adjacent stripes, multiple spatially adjacent coding tree units, or multiple spatially adjacent coding blocks. It is understood that the method for calculating the spatial continuity parameter for a given image region may be applicable to the above various possible given image regions, and the embodiments of the present disclosure do not limit this.

[0138] In the embodiment of the present disclosure, the method for determining the spatial continuity parameter of a given image region can be the same as that of S401-A1 and S401-A2 above, which will not be repeated here. That is, step S501 includes steps S401-A1 and S401-A2.

[0139] It's easy to understand that the fundamental principle of intra-frame sub-block division is to achieve a finer-grained division of the coding block based on spatial correlation. Since the resulting sub-blocks are highly correlated, the residual error of intra-frame prediction can be further reduced, thereby improving compression efficiency. Obviously, while intra-frame sub-block division improves compression rate, it also increases the number of block division possibilities, leading to more rate-distortion optimization calculations and significantly increasing coding complexity. For video images with weak spatial correlation, the performance gain from intra-frame sub-block division will be very limited, but it will still significantly increase coding complexity.

[0140] It can be seen that by calculating the spatial domain continuity parameter to obtain the intra-frame sub-block division information, it is possible to avoid unnecessary intra-frame sub-block division in image areas with weak spatial domain correlation, reduce the computational complexity, and thus improve the coding efficiency. In the above method, the continuity parameter of the image area is determined by determining the gradient map of the image area. The calculation method is simple and accurate, the computational overhead is low, and the coding efficiency is further improved.

[0141] Step S502: When the intra sub-block partitioning information indicates that the current block does not use intra sub-block partitioning, determine at least one second intra prediction mode and a prediction cost corresponding to the at least one second intra prediction mode.

[0142] For a coding block that is not divided into intra-frame sub-blocks, since it is no longer divided into multiple sub-blocks, the block is predictively coded as a whole.

[0143] In some embodiments, different second intra-frame prediction modes are used to predict the current block. Exemplary second intra-frame prediction modes that can be used include, but are not limited to, planar mode, DC mode, multiple angle modes, matrix weighted intra-frame prediction (MIP), etc.

[0144] It is understood that by predicting the current block using each second intra-frame prediction mode, a prediction cost for the second intra-frame prediction can be calculated. Prediction cost is typically used to represent a comprehensive measure of bitstream overhead and image distortion caused by encoding when using a prediction mode. Common prediction costs include, but are not limited to, rate-distortion cost.

[0145] It can be seen that when different second intra prediction modes are used to predict the current block, corresponding multiple prediction costs will also be obtained.

[0146] Step S503 : determining at least one second other prediction mode and a prediction cost corresponding to the at least one second other prediction mode.

[0147] For the current coding block, multiple second other prediction modes may be used for prediction. In some embodiments, the second other prediction modes include inter-frame prediction mode, intra-frame and inter-frame joint prediction, etc.

[0148] It is understood that intra-frame prediction is not prohibited in frames using inter-frame prediction. That is, for coding blocks in P-frames and B-frames, both intra-frame and inter-frame prediction modes need to be considered when determining their prediction modes.

[0149] Similarly, by predicting the current block using each of the second alternative prediction modes, the prediction cost of the second alternative prediction can be calculated. Prediction cost is typically used to represent a comprehensive measure of the bitrate overhead and image distortion caused by encoding using a prediction mode. Common prediction costs include, but are not limited to, rate-distortion cost.

[0150] It can be seen that when different second frame prediction modes are used to predict the current block, corresponding multiple prediction costs will also be obtained.

[0151] Step S504 : determining a prediction mode with the minimum prediction cost among at least one second intra prediction mode and at least one second other prediction mode as the encoding prediction mode of the current block, and using the prediction mode with the minimum cost to predict the current block.

[0152] It is easy to understand that in steps S502 and S503, the current block has been predicted using at least one second intra prediction mode and at least one second other prediction mode to obtain prediction costs corresponding to these prediction modes. It is easy to imagine that by comparing these prediction costs, the prediction mode and prediction cost actually used in the predictive coding can be determined, and prediction of the current block can be performed based on the prediction mode.

[0153] In some embodiments, the prediction mode with the smallest prediction cost is determined as the prediction mode used in predictive encoding.

[0154] Step S505: writing the intra-frame sub-block division information into the bitstream.

[0155] The indication information may be used to indicate whether to perform intra-frame sub-block division on the current coding block. For example, a flag bit may be used to indicate the intra-frame sub-block division information, and the flag bit is written into the bitstream.

[0156] In some embodiments, the flag is a flag at the sequence level. When the flag takes the first value, it indicates that in the current image sequence, blocks in at least one image use intra-frame sub-block division. When the flag takes the second value, it indicates that all blocks in the image sequence do not use intra-frame sub-block division.

[0157] In some embodiments, the flag is a flag at the image level. When the flag takes a first value, it indicates that in the current image, blocks in at least one strip use intra-frame sub-block division. When the flag takes a second value, it indicates that all blocks in the image do not use intra-frame sub-block division.

[0158] In some embodiments, the flag is a flag at the slice level. When the flag takes the first value, it indicates that in the current slice, at least one block in the coding tree unit uses intra-frame sub-block division. When the flag takes the second value, it indicates that all blocks in the slice do not use intra-frame sub-block division.

[0159] In some embodiments, the flag is a flag at the coding tree level. When the flag takes the first value, it indicates that at least one block in the current coding tree unit uses intra-frame sub-block division. When the flag takes the second value, it indicates that all blocks in the coding tree unit do not use intra-frame sub-block division.

[0160] In some embodiments, the flag is a flag at the block level. When the flag takes a first value, it indicates that the current block uses intra-frame sub-block division. When the flag takes a second value, it indicates that the current block does not use intra-frame sub-block division.

[0161] The following describes an image prediction method provided by yet another embodiment of the present disclosure in conjunction with FIG6 .

[0162] Step S601: obtaining an image frame to be encoded, and calculating a spatial continuity parameter of the image frame. The spatial continuity parameter may represent the continuity of changes in pixels in a given image region.

[0163] In the embodiments of the present disclosure, the methods for determining the spatial continuity parameter of a given image region include but are not limited to the following methods:

[0164] Determine the continuity parameters of an image region using a gradient map:

[0165] determining a gradient map for a given image region, and a gradient map of the gradient map;

[0166] Based on the gradient map of the gradient map, a continuity parameter of the image region is determined.

[0167] The gradient map of an image region can reflect the changing trend of pixels in the image region. The gradient map of the gradient map of an image region can reflect the rate of change of pixels in the image region, that is, the continuity of change. Therefore, in this method, the continuity parameter of the image region is determined by calculating the gradient map of the gradient map of the gradient map of the image region.

[0168] Exemplarily, the gradient map Gmap of the image region is determined according to the following formula (1):

[0169] Gmap(x,y)=|I(x,y)-I(x+1,y)|+|I(x,y)-I(x,y+1)| (1)

[0170] Where I(x, y) is the pixel value of image region I at position (x, y), I(x+1, y) is the pixel value of image region I at position (x+1, y), I(x, y+1) is the pixel value of image region I at position (x, y+1), and Gmap(x, y) is the gradient value of the pixel point at position (x, y) in image region I.

[0171] Exemplarily, the gradient map GGmap of the gradient map of the image region is determined according to the following formula (2):

[0172] GGmap(x,y)=|Gmap(x,y)-Gmap(x+1,y)|+|Gmap(x,y)-Gmap(x,y+1)| (2)

[0173] Where Gmap(x+1,y) is the gradient value of the pixel at position (x+1,y) in image region I, and Gmap(x,y+1) is the gradient value of the pixel at position (x,y+1) in image region I. Gmap(x+1,y) and Gmap(x,y+1) can be determined according to the above formula (1). GGmap(x,y) is the gradient value of the gradient value of the pixel at position (x,y) in image region I.

[0174] The above formulas (1) and (2) show how to determine the gradient value and the gradient value of the gradient value of the pixel point (x, y) in the image region I. The gradient value of each pixel point in the image region I is determined by referring to the pixel point (x, y) for other pixels in the image region I, thereby obtaining the gradient map Gmap of the image region I. The above formula (2) is used to calculate the gradient value corresponding to each pixel point in the gradient map Gmap of the image region I, thereby obtaining the gradient map GGmap of the gradient map of the image region I.

[0175] It should be noted that the above formula (1) and formula (2) are only examples. The embodiments of the present disclosure can also adopt other methods to determine the gradient map of the image area and the gradient map of the gradient map. For example, the above formula (1) and formula (2) are deformed and the deformed formulas are used to determine the gradient map and the gradient map of the gradient map.

[0176] According to the above method, after the gradient map of the gradient map of the image region is determined, the continuity parameter of the image region is determined according to the gradient map GGmap of the gradient map.

[0177] The embodiments of the present disclosure do not limit the method of determining the continuity parameter of the image area based on the gradient map of the gradient map in the above SZ01-A2.

[0178] In one example, a median value of each gradient value in a gradient map of a gradient map of an image region is determined as a continuity parameter of the image region.

[0179] In another example, an average value of gradient values ​​in a gradient map of a gradient map of an image region is determined as the continuity parameter of the image region.

[0180] Step S602: Determine whether the continuity parameter is greater than or equal to a predetermined threshold. If yes, proceed to step S603; if not, proceed to step S604.

[0181] Step S603 : for a block to be encoded in the image frame to be encoded, determine that the intra-frame sub-block division information indicates that the block to be encoded uses intra-frame sub-block division, and determine a prediction mode with the minimum prediction cost based on the intra-frame sub-block division of the block to be encoded.

[0182] It is understandable that the method of determining the prediction mode with the minimum prediction cost based on the intra-frame sub-block division can be the same as the above S402-S404, which will not be repeated here.

[0183] Step S604 : for a block to be coded in a frame to be coded, determining that the intra-frame sub-block division information indicates that the block to be coded does not use intra-frame sub-block division, performing predictive coding on the block to be coded as a whole, and determining a prediction mode with the minimum prediction cost.

[0184] It is understandable that the block to be coded is predictively coded as a whole, and a prediction mode with the minimum prediction cost is determined in the same manner as in S502 to S504 above, which will not be described in detail here.

[0185] Step S605 , predicting the block to be coded using the prediction mode with the minimum prediction cost, and predicting the block to be coded using the prediction mode with the minimum prediction cost determined in step S603 or step S604 .

[0186] Step S606: Write the intra-frame sub-block division information into the bitstream, and use indication information to indicate whether to perform intra-frame sub-block division on the current coding block. For example, a flag bit can be used to indicate the intra-frame sub-block division information, and the flag bit is written into the bitstream.

[0187] It is understood that in some embodiments, since the spatial continuity parameter is calculated for a single image to be coded, this flag is a flag at the image level. When the flag takes the first value, it indicates that in the current image, at least one block in a coding tree unit uses intra sub-block partitioning. When the flag takes the second value, it indicates that all coding tree units in the image do not use intra sub-block partitioning.

[0188] In other embodiments, the given image region for calculating the spatial continuity parameter may also be a single slice, a single coding tree unit, or a single coding block. Therefore, the flag bit used to indicate the intra-frame sub-block division information may also be located at the slice level, coding tree level, or block level.

[0189] The present disclosure also provides a video encoding device, including a processor and a memory storing a computer program that can be run on the processor, wherein when the processor executes the computer program, it implements the video encoding method as described in any embodiment of the present disclosure.

[0190] The present disclosure also provides a computer storage medium having a computer program stored thereon, which, when executed by a computer, enables the computer to perform the method of the above-mentioned method embodiment. Alternatively, the embodiments of the present disclosure also provide a computer program product containing instructions, which, when executed by a computer, enables the computer to perform the method of the above-mentioned method embodiment.

[0191] When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present disclosure is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid state drive (SSD)).

[0192] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0193] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the unit is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0194] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment. For example, the functional units in the various embodiments of the present disclosure may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit.

[0195] The above content is merely a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. An image prediction method, characterized in that: include: Obtaining intra-frame sub-block division information, when the intra-frame sub-block division information indicates that the current block uses intra-frame sub-block division, Based on at least one intra-frame sub-block partitioning, determining at least one first intra-frame prediction mode and a prediction cost corresponding to the at least one first intra-frame prediction mode, determining at least one first other prediction mode and a prediction cost corresponding to the at least one first other prediction mode, Determine, among at least one first intra-frame prediction mode and at least one first other prediction mode, a prediction mode with the smallest prediction cost as the encoding prediction mode of the current block, and use the prediction mode with the smallest prediction cost to predict the current block. The intra-frame sub-block division information is obtained by calculating a spatial domain continuity parameter, and the intra-frame sub-block division information is written into a bitstream.

2. The prediction method according to claim 1, characterized in that When the intra-frame sub-block partitioning information indicates that the current block does not use intra-frame sub-block partitioning: determining at least one second intra-frame prediction mode and a prediction cost corresponding to the at least one second intra-frame prediction mode, determining at least one second other prediction mode and a prediction cost corresponding to the at least one second other prediction mode, Determine the prediction mode with the minimum prediction cost among at least one second intra-frame prediction mode and at least one second other prediction mode as the encoding prediction mode of the current block, and use the prediction mode with the minimum prediction cost to predict the current block.

3. The prediction method according to claim 1 or 2, characterized in that: The calculation of spatial domain continuity parameters includes: A first gradient map of a given image region is calculated, a gradient map of the first gradient map is calculated to obtain a second gradient map, and an average value of the second gradient map is taken as the spatial domain continuity parameter.

4. The prediction method according to claim 3, characterized in that The given image region is a single image, a single slice, a single coding tree block or a single coding block, and the current block is located in the given image region.

5. The prediction method according to claim 1, wherein: The intra-frame sub-block division information is located at a sequence level, a picture level, a slice level, a coding tree level or a block level in a code stream.

6. The prediction method according to claim 1 or 2, characterized in that: The at least one first other prediction mode comprises inter prediction or joint intra and inter prediction.

7. The prediction method according to claim 1 or 2, characterized in that: The at least one second other prediction mode comprises inter prediction or joint intra and inter prediction.

8. An image prediction device, characterized in that: including processor and memory; The memory shown is used to store computer programs; The processor is configured to call and execute the computer program stored in the memory to implement the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that For storing computer programs; The computer program enables a computer to execute the method according to any one of claims 1 to 7.

Images