Image prediction method, apparatus, and system, device, and storage medium
The image prediction method optimizes decoding by determining node splitting based on block sizes to enhance processing performance and coding speed through parallel processing of luma and chroma blocks, addressing inefficiencies in existing technologies.
Patent Information
- Application Number
- JP2024135393
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2019-07-30
- Filing Date
- 2024-08-14
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2040-01-08
AI Technical Summary
Existing video decoding technologies face challenges in efficiently processing small chroma blocks due to increased complexity in intra prediction, coefficient coding, and inter prediction, leading to decreased decoding performance.
An image prediction method that determines whether to split a current node based on preset sizes for luma or chroma blocks, allowing for parallel processing of coding blocks to improve prediction performance and coding speed by performing intra or inter prediction on all blocks covered by the current node.
This approach enhances image prediction processing performance and coding efficiency by enabling parallel processing of luma and chroma blocks, thereby improving decoding speed and reducing complexity.
Smart Images

Figure 2024161480000001_ABST
Abstract
Description
[Technical field]
[0001] This application claims priority to Chinese Patent Application No. 201910016466.3, entitled “VIDEO DECODING METHOD AND VIDEO DECODER”, filed on January 8, 2019, with the Patent Office of the State Intellectual Property Office of the People's Republic of China; priority to Chinese Patent Application No. 201910173454.1, entitled “VIDEO ENCODER, VIDEO DECODER, AND CORRESPONDING METHODS”, filed on March 7, 2019, with the Patent Office of the People's Republic of China; priority to Chinese Patent Application No. 201910219440.9, entitled “VIDEO ENCODER, VIDEO DECODER, AND CORRESPONDING MEHODS”, filed on March 21, 2019, with the Patent Office of the People's Republic of China; and priority to Chinese Patent Application No. 201910219440.9, entitled “VIDEO ENCODER, VIDEO DECODER, AND CORRESPONDING MEHODS”, filed on July 30, 2019, with the Patent Office of the People's Republic of China. This application claims priority to Chinese Patent Application No. 201910696741.0, entitled "METHOD FOR IMPROVING AN INTERFACE, DEVICE, AND SYSTEM, DEVICE, AND STORAGE MEDIUM," which are incorporated herein by reference in their entireties.
[0002] FIELD OF THE DISCLOSURE The embodiments of the present application relate to the field of video coding technology, and in particular to an image prediction method, apparatus, system, device, and storage medium. [Background technology]
[0003] Digital video capabilities may be incorporated into a wide variety of devices, including digital televisions, digital live broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cell phones or satellite radio phones (also called "smartphones"), video conferencing devices, video streaming devices, etc. Digital video devices implement video compression techniques, such as those described in standards defined by MPEG-2, MPEG-4, ITU H.263, and ITU H.264 / MPEG-4 Part 10 AVC (advanced video coding), video coding standard H.265 / High Efficiency Video Coding (HEVC) standards, and enhancements to such standards. Video devices may implement video compression techniques to more efficiently transmit, receive, encode, decode, and / or store digital video information.
[0004] With the development of information technology, video services such as high-definition television, web conferencing, IPTV, and 3D television are developing rapidly. Video signals become the most important way to obtain information in people's daily life due to advantages such as intuitiveness and high efficiency. Video signals contain a large amount of data, so they occupy a large transmission bandwidth and storage space. In order to transmit and store video signals effectively, they need to be compressed and encoded. Video compression technology has become an increasingly indispensable key technology in the field of video applications.
[0005] The encoding process mainly includes the following steps: intra prediction, inter prediction, transform, quantization, entropy encode, in-loop filtering (mainly de-blocking filtering), etc. After dividing an image into coding blocks, intra prediction or inter prediction is performed. Then, after obtaining the residual, transform and quantization are performed. Finally, entropy encoding is performed to output a bitstream. In this specification, a coding block is an array with a size of M×N pixels (where M may or may not be equal to N). In addition, the value of the pixel at each pixel position is known. Video decoding is the reverse process of video encoding. For example, entropy decoding, dequantization, and inverse transform are first performed to obtain the residual information; then, whether intra prediction or inter prediction is performed for the current block is determined by decoding the bitstream. When intra-coding is performed, a prediction block is constructed based on pixel values of pixels in a reconstructed area around a current image using an intra-prediction method. When inter-coding is performed, motion information needs to be obtained by analysis, and a reference block is determined in a reconstructed image based on the motion information obtained by analysis, and pixel values of pixels in the reference block are used as a prediction block (such a process is called motion compensation (MC)). To obtain reconstructed information, the prediction block and residual information are added, and a filtering operation is performed.
[0006] Currently, two child nodes each having a size of 4×M (or M×4) are generated by dividing a node having a size of 8×M (or M×8) by vertical binary tree division (or horizontal binary tree division). Similarly, two child nodes each having a size of 4×M (or M×4) and one child node having a size of 8×M (or N×8) are generated by dividing a node having a size of 16×M (or M×16) by vertical ternary tree division (or horizontal ternary tree division). In the case of a YUV4:2:0 data format, the resolution of a chroma component is 1 / 2 of the resolution of a luma component. In other words, one 4×M node includes one 4×M luma block and two 2×(M / 2) chroma blocks. Therefore, small chroma blocks having sizes such as 2×2, 2×4, or 4×2 can be generated by dividing a current node in a preset division mode. It is relatively complicated for a hardware decoder to process small chroma blocks. The complexity is particularly reflected in three aspects:
[0007] (1) Intra prediction: To increase processing speed, hardware is generally designed to process 16 pixels at a time during intra prediction. However, small chroma blocks with sizes such as 2×2, 2×4, or 4×2 contain less than 16 pixels. As a result, the processing performance of intra prediction is reduced.
[0008] (2) Coefficient coding: In HEVC, transform coefficient coding is performed based on a coefficient group (CG) containing 16 coefficients. However, small chroma blocks with sizes such as 2×2, 2×4, or 4×2 contain 4 or 8 transform coefficients. As a result, to support coefficient coding of these small blocks, a coefficient group containing 4 or 8 coefficients needs to be added. As a result, the implementation complexity increases.
[0009] (3) Inter prediction: Inter prediction for small chroma blocks has a relatively high requirement on data bandwidth, which affects the decoding process speed. Summary of the Invention
[0010] The present application provides an image prediction method, apparatus, system, device, and storage medium for improving the processing performance of image prediction and increasing the coding processing speed.
[0011] A first aspect of the present application provides an image prediction method, the method including: obtaining a partition mode of a current node; determining whether an image block having a preset size is obtained by partitioning the current node in the partition mode of the current node, the image block including a luma block or a chroma block; and performing intra prediction or inter prediction on all coding blocks covered by the current node when it is determined that an image block having the preset size is obtained by partitioning the current node in the partition mode of the current node.
[0012] Optionally, the image block with the preset size may be a luma block with a size less than a threshold, which may be a number of luma samples, such as 128, 64, or 32, or a number of chroma samples, such as 32, 16, or 8. The size of the current node may be equal to or greater than the threshold.
[0013] Optionally, the step of performing intra prediction may perform the prediction by using a common intra prediction mode or an intra block copy (IBC) mode.
[0014] Optionally, all coding blocks covered by the current node are all coding blocks located in the region corresponding to the current node. Alternatively, a coding block may be a coding unit.
[0015] Optionally, if the slice type in which the current node is located is of intra type, then instead of inter prediction, intra prediction is performed for all coding blocks covered by the current node.
[0016] The advantageous effects of the embodiment of the present application are as follows: In the present application, it is considered that a luma block or a chroma block having a preset size is obtained by dividing an image block corresponding to a current node. When the above case exists, the encoder side or the decoder side performs intra prediction or inter prediction on all coding blocks obtained by dividing or not dividing the current node serving as a root node, to implement parallel processing of the luma block or the chroma block having a preset size. This improves the processing performance of image prediction, and further improves the coding performance.
[0017] Optionally, the following two cases relate to image blocks having a preset size: a luma block having a first preset size and a chroma block having a second preset size. The step of performing intra prediction or inter prediction on all coding blocks covered by the current node includes: determining whether a luma block having a first preset size is obtained by splitting the current node in a partition mode; performing intra prediction on all coding blocks covered by the current node if it is determined that a luma block having the first preset size is obtained by splitting the current node in the partition mode; or performing intra prediction or inter prediction on all coding blocks covered by the current node if it is determined that a luma block having the first preset size is not obtained by splitting the current node in the partition mode.
[0018] Optionally, the step of performing intra prediction or inter prediction on all coding blocks covered by the current node when it is determined that a luma block having a first preset size is not obtained by dividing the current node in the division mode may include: analyzing a prediction mode status flag of the current node when it is determined that a luma block having a first preset size is not obtained by dividing the current node in the division mode; performing inter prediction on all coding blocks covered by the current node when the value of the prediction mode status flag is a first value; or performing intra prediction on all coding blocks covered by the current node when the value of the prediction mode status flag is a second value.
[0019] With regard to the first aspect, in a first possible implementation example of the first aspect, the image block having the preset size includes a luma block having a first preset size, and the step of determining whether an image block having the preset size can be obtained by splitting the current node in a split mode includes a step of determining whether a luma block having the first preset size can be obtained by splitting the current node in a split mode based on the size of the current node and the split mode of the current node.
[0020] Optionally, the luma block having the first preset size may be a luma block having a pixel size of 4x4 or 8x8, or a luma block having an area of 16 pixels or 32 pixels.
[0021] Optionally, if the luma block having the first preset size is a luma block having a pixel size of 4×4 or an area of 16 pixels, determining whether a luma block having the first preset size is obtained by splitting the current node in a split mode based on the size of the current node and the split mode of the current node may be performed based on one of the following conditions: The number of samples in the luma block of the current node is 64, and the partitioning mode is quadtree partitioning; The number of samples in the luma block of the current node is 64 and the partitioning mode is ternary tree partitioning; or The number of samples in the luma block of the current node is 32, and the partitioning mode is binary tree partitioning.
[0022] With respect to a first possible implementation of the first aspect, optionally, the step of performing intra prediction or inter prediction on all coding blocks covered by the current node when it is determined that an image block having a predetermined size is obtained by splitting the current node in the split mode of the current node includes a step of performing intra prediction on all coding blocks covered by the current node when it is determined that a luma block having a first predetermined size is obtained by splitting the current node in the split mode.
[0023] With respect to a first possible implementation of the first aspect, optionally, when it is determined that a luma block having a first preset size is not obtained by splitting the current node in the split mode, the method further includes a step of determining whether a chroma block having a second preset size is obtained by splitting the current node in the split mode; and a step of performing intra prediction or inter prediction on all coding blocks covered by the current node when it is determined that a chroma block having the second preset size is obtained by splitting the current node in the split mode.
[0024] In conclusion, in the above-mentioned first embodiment, it is determined that intra prediction or inter prediction is performed on all coding blocks obtained by dividing or not dividing the current node that functions as a root node, so that parallel processing can be performed on luma blocks or chroma blocks with a preset size.This improves the processing performance of image prediction, and further improves coding performance.
[0025] Optionally, the luma block having the first preset size may be a 4×4 luma block or a luma block having an area of 16 pixels. When the luma block having the first preset size is a 4×4 luma block, the chroma block having the second preset size may be a chroma block having a pixel size of 2×4 or 4×2, or a chroma block having an area of 8 pixels, excluding a chroma block having a pixel size of 2×2 or having an area of 4 pixels.
[0026] Optionally, the luma block having the first preset size may be a 4×4 luma block or a luma block having an area of 16 pixels. When the luma block having the first preset size is a 4×4 luma block, the chroma block having the second preset size may be a luma block having a pixel size of 4×8 or 8×4, or a luma block having an area of 32 pixels, excluding the luma block having a pixel size of 4×4 or having an area of 16 pixels.
[0027] Optionally, if the chroma block having the second preset size is a chroma block having a pixel size of 2×4 or 4×2 or a chroma block having an area of 8 pixels, or a chroma block having a pixel size of 4×8 or 8×4 or a luma block having an area of 32 pixels, the step of determining whether a chroma block having the second preset size is obtained by splitting the current node in the split mode may be performed based on one of the following conditions: (1) The number of samples in the luma block of the current node is 64 and the partitioning mode is binary tree partitioning; or (2) The number of samples in the luma block of the current node is 128, and the partitioning mode is ternary tree partitioning.
[0028] With regard to the first aspect, in a second possible implementation of the first aspect, the image block having the preset size includes a chroma block having a second preset size, and the step of determining whether an image block having the preset size can be obtained by splitting the current node in a split mode includes a step of determining whether a chroma block having the second preset size can be obtained by splitting the current node in a split mode based on the size of the current node and the split mode of the current node.
[0029] Optionally, the chroma block having the second pre-set size may be a chroma block having a pixel size of 2x2, 2x4, or 4x2, or a chroma block having an area of 4 pixels or 8 pixels.
[0030] Optionally, the step of determining whether a chroma block having a second preset size is obtained by splitting the current node in a split mode based on the size of the current node and the split mode of the current node may include a step of determining whether a luma block having a third preset size is obtained by splitting the current node in a split mode based on the size of the current node and the split mode of the current node.
[0031] Optionally, the luma block having a third preset size may be a luma block having a pixel size of 4x4, 4x8, or 8x4, or a luma block having an area of 16 pixels or 32 pixels.
[0032] Optionally, the step of determining whether a chroma block having a second preset size is obtained by splitting the current node in the split mode may be performed based on one of the following conditions: (1) The number of samples in the luma block of the current node is 64, and the partitioning mode is quadtree partitioning; (2) The number of samples in the luma block of the current node is 64, and the partitioning mode is ternary tree partitioning; (3) The number of samples in the luma block of the current node is 32, and the partitioning mode is binary tree partitioning; (4) The number of samples in the luma block of the current node is 64, and the partitioning mode is binary tree partitioning; or (5) The number of samples in the luma block of the current node is 128, and the partitioning mode is ternary tree partitioning.
[0033] Optionally, the chroma block having the second preset size may be a chroma block having a pixel size of 2×4 or 4×2, or a chroma block having an area of 8 pixels, excluding a chroma block having a pixel size of 2×2 or a chroma block having an area of 4 pixels. Similarly, the luma block having the third preset size may be a luma block having a pixel size of 4×8 or 8×4, or a luma block having an area of 32 pixels, excluding a luma block having a pixel size of 4×4 or a luma block having an area of 16 pixels. Correspondingly, the step of determining whether a chroma block having a second preset size is obtained by splitting the current node in the split mode may be performed based on one of the following conditions: (1) The number of samples in the luma block of the current node is 64 and the partitioning mode is binary tree partitioning; or (2) The number of samples in the luma block of the current node is 128, and the partitioning mode is ternary tree partitioning.
[0034] With respect to the first or second embodiment, when it is determined that a chroma block having a second preset size is obtained by splitting the current node in the split mode, the step of performing intra prediction or inter prediction for all coding blocks covered by the current node includes the steps of analyzing a prediction mode status flag of the current node; performing inter prediction for all coding blocks covered by the current node when the value of the prediction mode status flag is a first value; or performing intra prediction for all coding blocks covered by the current node when the value of the prediction mode status flag is a second value. This embodiment is used for a video decoder. The prediction mode used for all coding blocks obtained by splitting or not splitting the current node serving as a root node is determined by analyzing the prediction mode status flag from the bitstream. Compared with the conventional technology, the analysis only needs to be performed once, which increases the processing speed of video decoding.
[0035] Optionally, the slice type in which the current node is located is not of Intra type.
[0036] With respect to the first embodiment or the second embodiment, when it is determined that a chroma block having a second preset size is obtained by splitting the current node in the split mode, the step of performing intra prediction or inter prediction for all coding blocks covered by the current node includes: performing inter prediction for all coding blocks covered by the current node if a prediction mode used for any coding block covered by the current node is inter prediction; or performing intra prediction for all coding blocks covered by the current node if a prediction mode used for any coding block covered by the current node is intra prediction; Optionally, any coding block is a first coding block of all coding blocks covered by the current node in decoding order. This embodiment is used for a video decoder. A prediction mode used for any coding block of the current node is parsed from a bitstream, and the prediction mode obtained by the parsing is used for prediction of all coding blocks obtained by splitting or not splitting the current node serving as a root node. Compared with the conventional technology, the parsing only needs to be performed once, which increases the processing speed of video decoding.
[0037] Regarding the second embodiment, optionally, the step of performing intra prediction or inter prediction on all coding blocks covered by the current node when it is determined that a chroma block having a second preset size is obtained by dividing the current node in the division mode includes: determining whether a luma block having a first preset size is obtained by dividing the current node in the division mode; and performing intra prediction on all coding blocks covered by the current node when it is determined that a luma block having a first preset size is obtained by dividing the current node in the division mode. In this embodiment, since it is determined that intra prediction is performed on all coding blocks obtained by dividing or not dividing the current node serving as a root node, parallel processing can be performed on luma blocks having a first preset size and chroma blocks having a second preset size. This improves the processing performance of image prediction and further improves coding performance.
[0038] Optionally, when it is determined that no luma block having a first preset size is obtained by splitting the current node in the split mode, performing intra prediction or inter prediction for all coding blocks covered by the current node includes: analyzing a prediction mode status flag of the current node; performing inter prediction for all coding blocks covered by the current node when the value of the prediction mode status flag is a first value; or performing intra prediction for all coding blocks covered by the current node when the value of the prediction mode status flag is a second value. This embodiment is used for a video decoder. The prediction mode used for all coding blocks obtained by splitting or not splitting the current node serving as a root node is determined by analyzing the prediction mode status flag from the bitstream. Compared with the conventional technology, the analysis only needs to be performed once, which increases the processing speed of video decoding.
[0039] Optionally, when it is determined that no luma block having the first preset size is obtained by splitting the current node in the split mode, the step of performing intra prediction or inter prediction for all coding blocks covered by the current node includes: performing inter prediction for all coding blocks covered by the current node if the prediction mode used for any coding block covered by the current node is inter prediction; or performing intra prediction for all coding blocks covered by the current node if the prediction mode used for any coding block covered by the current node is intra prediction. This embodiment is used for a video decoder. The prediction mode used for any coding block of the current node is parsed from the bitstream, and the prediction mode obtained by the parsing is used for prediction of all coding blocks obtained by splitting or not splitting the current node serving as a root node. Compared with the conventional technology, the parsing is only required once, which increases the processing speed of video decoding.
[0040] Optionally, the given coding block is the first coding block of all coding blocks covered by the current node in decoding order.
[0041] With respect to the first aspect or any one of the possible implementations of the first aspect, in a third possible implementation of the first aspect, the step of performing intra prediction or inter prediction on all coding blocks covered by the current node includes the steps of: dividing a luma block included in the current node in a division mode to obtain a luma block obtained by division, performing intra prediction on the luma block obtained by division, and using a chroma block included in the current node as a chroma coding block to perform intra prediction on the chroma coding block; or dividing a luma block included in the current node in a division mode to obtain a luma block obtained by division, performing inter prediction on the luma block obtained by division, dividing a chroma block included in the current node in a division mode to obtain a chroma block obtained by division, and performing inter prediction on the chroma block obtained by division. In this implementation, regardless of whether intra prediction or inter prediction is performed on all coding blocks covered by the current node, the luma block of the current node is always divided, and in the case of an inter prediction mode, the chroma block of the current node may be divided, but in the case of an intra prediction mode, the chroma block of the current node is not divided. In this embodiment, a chroma block having a second preset size on which intra prediction is performed is not generated, thereby avoiding the case where intra prediction is performed on a small chroma block, thereby increasing the processing speed of video coding.
[0042] With respect to the first aspect or any one of the possible implementations of the first aspect, in a fourth possible implementation of the first aspect, the step of performing intra prediction or inter prediction on all coding blocks covered by the current node includes the steps of: dividing a luma block included in the current node in a partition mode to obtain a luma block obtained by partitioning, performing intra prediction on the luma block obtained by partitioning, and using a chroma block included in the current node as a chroma coding block to perform intra prediction on the chroma coding block; or dividing a luma block included in the current node in a partition mode to obtain a luma block obtained by partitioning, performing inter prediction on the luma block obtained by partitioning, and using a chroma block included in the current node as a chroma coding block to perform inter prediction on the chroma coding block. In this implementation, regardless of whether intra prediction or inter prediction is performed on all coding blocks covered by the current node, the chroma block of the current node is not partitioned and the luma block is partitioned in the partition mode of the luma block. In this embodiment, a chroma block having a second preset size on which intra prediction is performed is not generated, thereby avoiding the case where intra prediction is performed on a small chroma block, thereby increasing the processing speed of video coding.
[0043] Regarding the first aspect or any one of the possible implementation forms of the first aspect, in a fifth possible implementation form of the first aspect, if inter prediction is performed on all coding blocks covered by the current node, the step of performing inter prediction on all coding blocks covered by the current node includes: The method includes: splitting the current node in a partition mode of the current node to obtain a child node of the current node; determining a partition mode that is not permitted for the child node of the current node based on a size of the child node of the current node; determining a block partition policy of the child node of the current node based on the partition mode that is not permitted for the child node of the current node; obtaining a coding block corresponding to the child node of the current node according to the block partition policy of the child node of the current node; and performing inter prediction on the corresponding coding block. In this embodiment, in the case of inter prediction, a luma block having a first preset size is not generated.
[0044] The child node can be obtained by dividing the current node once, or can be obtained by dividing the current node N times, where N is an integer greater than one.
[0045] The partitioning policy may include no partitioning, one partitioning, or N partitioning, where N is an integer greater than one.
[0046] A second aspect of the present application provides an image prediction device, comprising: an acquisition module configured to acquire a partition mode of a current node; A determining module configured to determine whether an image block having a preset size is obtained by dividing a current node in a division mode, where the image block includes a luma block or a chroma block; and an execution module configured to perform intra prediction or inter prediction on all coding blocks covered by the current node when it is determined that an image block having a preset size is obtained by dividing the current node in the division mode.
[0047] A third aspect of the present application provides a video encoding apparatus including a processor and a memory configured to store executable instructions for the processor, the processor executing a method according to the first aspect of the present application.
[0048] A fourth aspect of the present application provides a video decoding apparatus including a processor and a memory configured to store executable instructions for the processor, the processor executing a method according to the first aspect of the present application.
[0049] A fifth aspect of the present application provides an image prediction system including a video acquisition device, a video encoding device according to the third aspect of the present application, a video decoding device according to the fourth aspect of the present application, and a display device. The video encoding device is connected to both the video acquisition device and the video decoding device. The video decoding device is connected to the display device.
[0050] A sixth aspect of the present application provides a computer-readable storage medium storing a computer program, the computer program being executed by a processor to perform the method according to the first aspect of the present application.
[0051] A seventh aspect of the present application provides a method for predicting an image, the method comprising: The method includes: obtaining a partition mode of a current node, where the current node is an image block in a coding tree unit of a current image; determining whether the current node satisfies a first condition based on the partition mode of the current node and a size of the current node; and when it is determined that the current node satisfies the first condition, performing intra prediction on all coding blocks belonging to the current node to obtain predictors of all coding blocks belonging to the current node.
[0052] The size of the current node is determined based on the size of the coding tree node that corresponds to the current node and the partitioning mode used to obtain the current node.
[0053] The type of the slice in which the current node is located is type B or type P. It should be understood that if the type of the slice in which the current node is located is type I, by default, intra prediction needs to be performed on all coding blocks covered by the current node.
[0054] Optionally, a step of determining whether the current node satisfies a first condition may be performed based on the split mode of the current node and the size of the current node, if the size of the current node is less than or equal to a preset size upper limit. The preset size upper limit may be 256, 128, or 64.
[0055] The first condition may include: (1) The number of samples in the luma block of the current node is 64, and the partitioning mode is quadtree partitioning; (2) The number of samples in the luma block of the current node is 64 and the partitioning mode is ternary tree partitioning; or (3) The number of samples in the luma block of the current node is 32, and the partitioning mode is binary tree partitioning.
[0056] With regard to the seventh aspect, in a first possible implementation example of the seventh aspect, when it is determined that the current node does not satisfy the first condition, the method further includes a step of determining whether the current node satisfies a second condition based on a partition mode of the current node and a size of the current node; when it is determined that the current node satisfies the second condition, performing prediction on all coding blocks belonging to the current node using the same prediction mode to obtain predictors for all coding blocks belonging to the current node; where the prediction mode is intra prediction or inter prediction.
[0057] Determining whether the current node satisfies the second condition based on the split mode of the current node and the size of the current node may include determining whether the current node satisfies the second condition based on the split mode of the current node, the size of the current node, and a current chroma format.
[0058] The second condition may include: (1) The number of samples in the luma block of the current node is 64, and the partitioning mode is binary tree partitioning; (2) The number of samples in the luma block of the current node is 128, and the partitioning mode is ternary tree partitioning; (3) The number of samples in the luma block of the current node is 64, the partitioning mode is binary tree partitioning, and the chroma format is 4:2:0; (4) The number of samples in the luma block of the current node is 128, the partitioning mode is ternary tree partitioning, and the chroma format is 4:2:0; or (5) The chroma block width of the child node of the current node is 2.
[0059] With respect to a first possible implementation example of the seventh aspect, in a second possible implementation example of the seventh aspect, the step of performing prediction for all coding blocks belonging to the current node using the same prediction mode includes a step of analyzing a prediction mode status flag of the current node; and a step of performing inter prediction for all coding blocks belonging to the current node if the value of the prediction mode status flag is a first value; or a step of performing intra prediction for all coding blocks belonging to the current node if the value of the prediction mode status flag is a second value.
[0060] The prediction mode status flag may be a syntax element that is parsed during block partitioning, i.e., during partitioning of a coding tree. When the syntax element is parsed, the coding unit prediction mode status flag (cu_pred_mode) of the coding unit in the coverage area of the current node may not be parsed, and the value of the coding unit prediction mode status flag is a default value corresponding to the value of the prediction mode status flag.
[0061] With respect to the first or second possible implementation example of the seventh aspect, in a third possible implementation example of the seventh aspect, the step of performing inter prediction on all coding blocks belonging to the current node includes: splitting the current node in a partition mode of the current node to obtain a child node of the current node; determining a partition mode that is not permitted for the child node of the current node based on a size of the child node of the current node; determining a block partition policy of the child node of the current node based on the partition mode that is not permitted for the child node of the current node; and obtaining coding blocks corresponding to the child nodes of the current node according to the block partition policy of the child node of the current node, and performing inter prediction on the corresponding coding blocks.
[0062] If a child node is restricted to perform only inter prediction and the number of luma samples of the child node is 32 (i.e., the product of the node's width and height is 32), binary tree partitioning (including horizontal binary tree partitioning or vertical binary tree partitioning) is not allowed for the child node. If a child node is restricted to perform only inter prediction and the number of luma samples of the child node is 64 (i.e., the product of the node's width and height is 64), ternary tree partitioning (including horizontal ternary tree partitioning or vertical ternary tree partitioning) is not allowed for the child node. Such a determination method is applicable to both YUV4:2:0 and YUV4:2:2 video data formats.
[0063] For example, if the size of the current node is 8x8 and two 8x4 (or 4x8) nodes are generated by horizontal binary tree splitting (or vertical binary tree splitting), further splitting the 8x4 (or 4x8) node will generate a 4x4 block. In this case, vertical binary tree splitting (or horizontal binary tree splitting) is not allowed for the 8x4 (or 4x8) node, or further splitting of the 8x4 (or 4x8) node is not allowed. As another example, if the quantity of samples of the luma block of the current node is 128 and the splitting mode is horizontal ternary tree splitting or vertical ternary tree splitting, the quantity of samples of the luma block may be determined to be 64. However, if the horizontal ternary tree splitting or vertical ternary tree splitting of the child node is used for a luma node with a quantity of samples of 64, a 4x4 luma block is obtained. Therefore, when restricted to performing only inter prediction, no horizontal or vertical ternary tree splitting is allowed for a node with a sample quantity of 64, or any further splitting of a node with a sample quantity of 64 is allowed.
[0064] With respect to the seventh aspect or any one of the preceding possible implementations of the seventh aspect, in a fourth possible implementation of the seventh aspect, the step of performing intra prediction on all coding blocks belonging to the current node includes, in a division mode, a step of dividing a luma block included in the current node to obtain a luma block obtained by the division, and performing intra prediction on the luma block obtained by the division; and a step of using a chroma block included in the current node as a chroma coding block, and performing intra prediction on the chroma coding block.
[0065] An eighth aspect of the present application provides an image prediction method, the method including: obtaining a partition mode of a current node, the current node being an image block in a coding tree unit of a current image; determining whether the current node satisfies a preset condition based on the partition mode of the current node and a size of the current node; and if it is determined that the current node satisfies the preset condition, performing prediction on all coding blocks belonging to the current node using the same prediction mode to obtain predictors of all coding blocks belonging to the current node; the prediction mode is intra prediction or inter prediction.
[0066] The size of the current node may be determined based on the size of the coding tree node that corresponds to the current node and the partitioning mode used to obtain the current node.
[0067] The step of determining whether the current node satisfies the preset condition based on the split mode of the current node and the size of the current node may include the step of determining whether the current node satisfies the preset condition based on the split mode of the current node, the size of the current node, and a current chroma format.
[0068] The predefined conditions may include: (1) The number of samples in the luma block of the current node is 64 and the partitioning mode is binary tree partitioning; or (2) The number of samples in the luma block of the current node is 128, and the partitioning mode is ternary tree partitioning.
[0069] With regard to the eighth aspect, in a first possible implementation example of the eighth aspect, the step of performing prediction for all coding blocks belonging to the current node using the same prediction mode includes a step of analyzing a prediction mode status flag of the current node; and a step of performing inter prediction for all coding blocks belonging to the current node if the value of the prediction mode status flag is a first value; or a step of performing intra prediction for all coding blocks belonging to the current node if the value of the prediction mode status flag is a second value.
[0070] With respect to the eighth aspect or the first possible implementation example of the eighth aspect, in a second possible implementation example of the eighth aspect, the step of performing inter prediction on all coding blocks belonging to the current node includes: splitting the current node in a partition mode of the current node to obtain a child node of the current node; determining a partition mode that is not permitted for the child node of the current node based on a size of the child node of the current node; determining a block partition policy of the child node of the current node based on the partition mode that is not permitted for the child node of the current node; and obtaining coding blocks corresponding to the child nodes of the current node according to the block partition policy of the child node of the current node, and performing inter prediction on the corresponding coding blocks.
[0071] If a child node is restricted to perform only inter prediction and the number of luma samples of the child node is 32 (i.e., the product of the node's width and height is 32), binary tree partitioning (including horizontal binary tree partitioning or vertical binary tree partitioning) is not allowed for the child node. If a child node is restricted to perform only inter prediction and the number of luma samples of the child node is 64 (i.e., the product of the node's width and height is 64), ternary tree partitioning (including horizontal ternary tree partitioning or vertical ternary tree partitioning) is not allowed for the child node. Such a determination method is applicable to both YUV4:2:0 and YUV4:2:2 video data formats.
[0072] For example, if the size of the current node is 8x8 and two 8x4 (or 4x8) nodes are generated by horizontal binary tree splitting (or vertical binary tree splitting), further splitting the 8x4 (or 4x8) node will generate a 4x4 block. In this case, vertical binary tree splitting (or horizontal binary tree splitting) is not allowed for the 8x4 (or 4x8) node, or further splitting of the 8x4 (or 4x8) node is not allowed. As another example, if the quantity of samples of the luma block of the current node is 128 and the splitting mode is horizontal ternary tree splitting or vertical ternary tree splitting, the quantity of samples of the luma block may be determined to be 64. However, if the horizontal ternary tree splitting or vertical ternary tree splitting of the child node is used for a luma node with a quantity of samples of 64, a 4x4 luma block is obtained. Therefore, when restricted to performing only inter prediction, no horizontal or vertical ternary tree splitting is allowed for a node with a sample quantity of 64, or any further splitting of a node with a sample quantity of 64 is allowed.
[0073] A ninth aspect of the present application provides an image prediction method, the method including: determining a partition mode not allowed for a current node based on a size of the current node and a prediction mode of the current node, the current node being an image block in a coding tree unit of a current image; determining a block partition policy of the current node based on the partition mode not allowed for the current node; obtaining a coding block corresponding to the current node according to the block partition policy of the current node, and performing inter prediction on the corresponding coding block;
[0074] The size of the current node may be determined based on the size of the coding tree node that corresponds to the current node and the partitioning mode used to obtain the current node.
[0075] With regard to the ninth aspect, in a first possible implementation example of the ninth aspect, the step of determining a partitioning mode that is not permitted for the current node based on the size of the current node and the prediction mode of the current node includes the steps of determining whether only inter prediction mode is used for predicting all coding blocks belonging to the current node and determining whether the number of samples of the luma block of the current node is 32; and determining that binary tree partitioning is a partitioning mode that is not permitted for the current node if it is determined that only inter prediction mode is used for predicting all coding blocks belonging to the current node and the number of samples of the luma block of the current node is 32.
[0076] With respect to the ninth aspect or the first possible implementation of the ninth aspect, in a second possible implementation of the ninth aspect, the step of determining a partitioning mode that is not permitted for the current node based on the size of the current node and the prediction mode of the current node includes the steps of determining whether only inter prediction mode is used for predicting all coding blocks belonging to the current node and determining whether the sample amount of the luma block of the current node is 64; and determining that ternary tree partitioning is a partitioning mode that is not permitted for the current node if it is determined that only inter prediction mode is used for predicting all coding blocks belonging to the current node and the sample amount of the luma block of the current node is determined to be 64.
[0077] A tenth aspect of the present application provides an image prediction device, comprising: an acquisition module configured to acquire a partition mode of a current node, the current node being an image block in a coding tree unit of a current image; a determining module configured to determine whether the current node satisfies a first condition based on a partitioning mode of the current node and a size of the current node; and an execution module configured, when it is determined that the current node satisfies a first condition, to perform intra prediction on all coding blocks belonging to the current node to obtain predictors for all coding blocks belonging to the current node.
[0078] Regarding the tenth aspect, in a first possible implementation of the tenth aspect, The determination module is further configured to, when it is determined that the current node does not satisfy the first condition, determine whether the current node satisfies a second condition based on a partition mode of the current node and a size of the current node; The execution module is further configured, when it is determined that the current node satisfies a second condition, to perform prediction on all coding blocks belonging to the current node using the same prediction mode to obtain predictors for all coding blocks belonging to the current node, where the prediction mode is intra prediction or inter prediction.
[0079] The determination module may be configured to determine whether the current node satisfies a second condition based on a split mode of the current node, a size of the current node, and a current chroma format.
[0080] Regarding the first possible implementation example of the tenth aspect, in a second possible implementation example of the tenth aspect, the execution module Analyzing the prediction mode status flag of the current node; and The prediction mode status flag is configured to: perform inter prediction on all coding blocks belonging to the current node when the value of the prediction mode status flag is a first value; or perform intra prediction on all coding blocks belonging to the current node when the value of the prediction mode status flag is a second value.
[0081] With respect to the first or second possible implementation of the tenth aspect, in a third possible implementation of the tenth aspect, the execution module Splitting the current node with the split mode of the current node to get the child nodes of the current node; determining disallowed splitting modes for the child nodes of the current node based on the size of the child nodes of the current node; determining a block partitioning policy for the child nodes of the current node based on the partitioning modes that are not permitted for the child nodes of the current node; and The present invention is configured to: obtain a coding block corresponding to a child node of the current node according to a block partitioning policy of the child node of the current node; and perform inter prediction on the corresponding coding block.
[0082] With respect to the tenth aspect or any one of the possible implementations of the tenth aspect, in a fourth possible implementation of the tenth aspect, the execution module is configured to, in a division mode, divide a luma block included in the current node to obtain a luma block obtained by the division, and perform intra prediction on the luma block obtained by the division; and use a chroma block included in the current node as a chroma coding block, and perform intra prediction on the chroma coding block.
[0083] An eleventh aspect of the present application provides an image prediction device, the device including: an acquisition module configured to acquire a partition mode of a current node, the current node being an image block in a coding tree unit of a current image; a determination module configured to determine whether the current node satisfies a preset condition based on the partition mode of the current node and a size of the current node; and an execution module configured to perform prediction on all coding blocks belonging to the current node using the same prediction mode when it is determined that the current node satisfies the preset condition, to acquire predictors of all coding blocks belonging to the current node, the prediction mode being intra prediction or inter prediction.
[0084] The determination module may be configured to determine whether the current node satisfies a preset condition based on the division mode of the current node, the size of the current node, and the current chroma format.
[0085] Regarding the eleventh aspect, in a first possible implementation example of the eleventh aspect, the execution module Analyzing a prediction mode status flag of the current node; and The prediction mode status flag is configured to: perform inter prediction on all coding blocks belonging to the current node when the value of the prediction mode status flag is a first value; or perform intra prediction on all coding blocks belonging to the current node when the value of the prediction mode status flag is a second value.
[0086] With respect to the 11th aspect or the first possible implementation example of the 11th aspect, in a second possible implementation example of the 11th aspect, the execution module is configured to: split the current node in a partition mode of the current node to obtain a child node of the current node; determine a partition mode that is not permitted for the child node of the current node based on a size of the child node of the current node; determine a block partition policy for the child node of the current node based on the partition mode that is not permitted for the child node of the current node; and obtain a coding block corresponding to the child node of the current node according to the block partition policy of the child node of the current node, and perform inter prediction on the corresponding coding block.
[0087] A twelfth aspect of the present application provides an image prediction device, including: a decision unit configured to determine a partition mode not allowed for a current node based on a size of the current node and a prediction mode of the current node, the current node being an image block in a coding tree unit of a current image, the decision unit further configured to determine a block partition policy of the current node based on the partition mode not allowed for the current node; and a prediction unit configured to obtain a coding block corresponding to the current node according to the block partition policy of the current node, and perform inter prediction on the corresponding coding block.
[0088] Regarding the twelfth aspect, in a first possible implementation example of the twelfth aspect, the decision unit Determining whether only inter prediction mode is used for prediction of all coding blocks belonging to the current node, and determining whether the number of samples of the luma block of the current node is 32; and The method is configured to: determine that binary tree partitioning is a partitioning mode that is not permitted for the current node when it is determined that only inter prediction mode is used for predicting all coding blocks belonging to the current node and the number of samples of the luma block of the current node is determined to be 32.
[0089] With regard to the twelfth aspect or the first possible implementation of the twelfth aspect, in a second possible implementation of the twelfth aspect, the decision unit Determining whether only inter prediction mode is used for prediction of all coding blocks belonging to the current node, and determining whether the number of samples of the luma block of the current node is 64; and The method is configured to: determine that, when it is determined that only inter prediction mode is used for predicting all coding blocks belonging to the current node, and the number of samples of the luma block of the current node is determined to be 64, determine that ternary tree partitioning is the partitioning mode that is not permitted for the current node.
[0090] For features and embodiments of the seventh, eighth and ninth aspects of the present application, please refer to the features and embodiments of the method of the first aspect.
[0091] The method of the seventh aspect of the present application can be performed by the device of the tenth aspect of the present application. For the functions of the device of the tenth aspect of the present application and different embodiments of the device, please refer to the features and embodiments of the method of the seventh aspect of the present application.
[0092] The method of the eighth aspect of the present application can be performed by the device of the eleventh aspect of the present application. For the functions of the device of the eleventh aspect of the present application and different embodiments of the device, please refer to the features and embodiments of the method of the eighth aspect of the present application.
[0093] The method of the ninth aspect of the present application can be performed by the device of the twelfth aspect of the present application. For the functions of the device of the twelfth aspect of the present application and different embodiments of the device, please refer to the features and embodiments of the method of the ninth aspect of the present application.
[0094] A thirteenth aspect of the present application provides a video stream decoding device including a processor and a memory, the memory storing instructions that enable the processor to perform a method according to the seventh, eighth or ninth aspect.
[0095] A fourteenth aspect of the present application provides a video stream encoding device including a processor and a memory, the memory storing instructions that enable the processor to perform a method according to the seventh, eighth or ninth aspect.
[0096] A fifteenth aspect of the present application provides a computer-readable storage medium having stored thereon instructions that, when executed, enable one or more processors to encode video data. The instructions enable the one or more processors to perform any one of the seventh aspect, the eighth aspect, the ninth aspect, or possible implementations of the seventh aspect, the eighth aspect, or the ninth aspect.
[0097] A sixteenth aspect of the present application provides a computer program comprising program code which, when executed on a computer, performs the method of the seventh, eighth or ninth aspect or any one of the possible implementations of the seventh, eighth or ninth aspect.
[0098] A seventeenth aspect of the present application provides a computer-readable storage medium. The computer-readable storage medium stores a bitstream, and the bitstream carries a prediction mode status flag of a current node of a current coding tree. When the value of the prediction mode status flag is a first value, it indicates that an inter prediction mode is used for all coding blocks belonging to the current node; or when the value of the prediction mode status flag is a second value, it indicates that an intra prediction mode is used for all coding blocks belonging to the current node.
[0099] It should be understood that the technical solutions of the second aspect to the seventeenth aspect of the present application are consistent with the technical solutions of the first aspect of the present application, and the advantageous effects achieved by the aspects and corresponding feasible embodiments are similar, and the details will not be described again.
[0100] The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description, drawings, and claims.
[0101] The present application provides an image prediction method, an apparatus, a system, an apparatus, and a storage medium. The method includes: obtaining a partition mode of a current node; and determining whether an image block having a preset size is obtained by partitioning the current node with the partition mode of the current node, where the image block includes a luma block or a chroma block. If an image block having a preset size is obtained by partitioning the current node with the partition mode of the current node, intra prediction or inter prediction is performed on all coding blocks covered by the current node. According to the above method, intra prediction or inter prediction is performed on all coding blocks of the current node, so that parallel processing can be performed on all coding blocks of the current node. This improves the processing performance of image prediction and increases the coding processing speed. [Brief description of the drawings]
[0102] [Figure 1A] FIG. 1 is a block diagram of an example video coding system 10 for implementing an embodiment of the present application. [Figure 1B] FIG. 4 is a block diagram of an example of a video coding system 40 for implementing an embodiment of the present application. [Diagram 2] FIG. 2 is a block diagram of an exemplary structure of an encoder 20 for implementing an embodiment of the present application. [Diagram 3] FIG. 3 is a block diagram of an exemplary structure of a decoder 30 for implementing an embodiment of the present application. [Figure 4] 4 is a block diagram of an example of a video coding device 400 for implementing an embodiment of the present application. [Diagram 5] FIG. 2 is a block diagram of another example of an encoding device or a decoding device for implementing an embodiment of the present application. [Figure 6(a)] FIG. 2 is a schematic block diagram of a block division mode for implementing an embodiment of the present application; [Figure 6(b)]FIG. 2 is a schematic block diagram of a block division mode for implementing an embodiment of the present application; [Figure 6(c)] FIG. 2 is a schematic block diagram of a block division mode for implementing an embodiment of the present application; [Figure 6(d)] FIG. 2 is a schematic block diagram of a block division mode for implementing an embodiment of the present application; [Figure 6(e)] FIG. 2 is a schematic block diagram of a block division mode for implementing an embodiment of the present application; [Figure 6(f)] FIG. 2 is a schematic block diagram of a block division mode for implementing an embodiment of the present application; [Figure 6(g)] FIG. 2 is a schematic block diagram of a block division mode for implementing an embodiment of the present application; [Figure 7] FIG. 2 is a schematic block diagram of intra prediction for implementing an embodiment of the present application; [Figure 8] 1 is a schematic block diagram of a video communication system for implementing an embodiment of the present application; [Figure 9] 1 is a schematic flowchart of a first image prediction method according to an embodiment of the present application; [Figure 10] 4 is a schematic flowchart of a second image prediction method according to an embodiment of the present application. [Figure 11] 11 is a schematic flowchart of a third image prediction method according to an embodiment of the present application. [Figure 12] 11 is a schematic flowchart of a fourth image prediction method according to an embodiment of the present application. [Figure 13] 11 is a schematic flowchart of a fifth image prediction method according to an embodiment of the present application. [Figure 14] 11 is a schematic flowchart of a sixth image prediction method according to an embodiment of the present application. [Figure 15] FIG. 2 is a functional schematic structural diagram of an image prediction device according to an embodiment of the present application; [Figure 16] FIG. 2 is a schematic structural diagram of the hardware of a video encoding device according to an embodiment of the present application; [Figure 17]FIG. 2 is a schematic structural diagram of the hardware of a video decoding device according to an embodiment of the present application; [Figure 18] FIG. 1 is a schematic structural diagram of an image prediction system according to an embodiment of the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0103] The following describes the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. In the following description, reference is made to the accompanying drawings, which form a part of this disclosure and which show, by way of example, certain aspects of the embodiments of the present application or in which the embodiments of the present application can be used. It should be understood that the embodiments of the present application can be used in other ways and may include structural or logical changes not shown in the accompanying drawings. Therefore, the following detailed description should not be construed as limiting, and the scope of the present application is defined by the appended claims. For example, it should be understood that what is disclosed with respect to a described method may also apply to a corresponding apparatus or system configured to perform the method, and vice versa. For example, when one or more specific method steps are described, the corresponding apparatus may include one or more units, such as functional units, for performing the described method steps, even if such one or more units are not explicitly described or shown in the accompanying drawings (e.g., one unit performs one or more steps; or multiple units each perform one or more of the multiple steps). Furthermore, when describing a particular apparatus based on one or more units, e.g., functional units, a corresponding method may include a step used to perform one or more functions of the one or more units (e.g., a step is used to perform one or more functions of the one or more units; or multiple steps are each used to perform one or more functions of one or more units within multiple units), even if such step or steps are not explicitly described or shown in the attached drawings. Furthermore, it should be understood that the features of various exemplary embodiments and / or aspects described herein can be combined with each other, unless specified otherwise.
[0104] Video coding typically refers to the processing of a sequence of pictures, which form a video or video sequence. In the field of video coding, the terms "picture", "frame" and "image" may be used synonymously. Video coding as used herein refers to video encoding or video decoding. Video encoding is performed at a source side and typically processes an original video picture (e.g., by compressing) to reduce the amount of data to represent the video picture for more efficient storage and / or transmission. Video decoding is performed at a destination side and typically involves the reverse processing compared to the processing of an encoder to reconstruct a video picture. The "coding" of a video picture in the embodiments should be understood as the "encoding" or "decoding" of a video sequence. The combination of the encoding part and the decoding part is also called CODEC (encoding and decoding).
[0105] A video sequence includes a series of images, which are further divided into slices, and slices are further divided into blocks. Video coding processes are performed block by block. In some new video coding standards, the concept of "block" is further extended. For example, a macroblock may be further divided into multiple predictive blocks (partitions) for predictive coding. Alternatively, basic concepts such as coding unit (CU), prediction unit (PU), and transform unit (TU) are used, multiple block units are obtained by functional partitioning, and a completely new tree-based structure is used for description. For example, a CU may be divided into smaller CUs by quadtree partitioning, and the smaller CUs may be further partitioned to form a quadtree structure. A CU is a basic unit for partitioning and coding a coding image. There may also be a similar tree structure for PUs and TUs. A PU may correspond to a predictive block and is a basic unit for predictive coding. A CU is further partitioned into multiple PUs in partition mode. A TU may correspond to a transform block, which is the basic unit for transforming a prediction residual, although CU, PU, and TU are all essentially concepts of a block (or image block).
[0106] A CTU is partitioned into multiple CUs using a quad-tree structure, represented as a coding tree. The decision on coding a picture region by inter-picture (temporal) or intra-picture (spatial) prediction is made at the CU depth. Each CU may be further partitioned into one, two, or four PUs in a PU partitioning pattern. Within one PU, the same prediction process is applied, and related information is transmitted to the decoder on a PU basis. After obtaining the residual block by applying the prediction process based on the PU partitioning pattern, the CU may be partitioned into transform units (TUs) based on another quad-tree structure similar to the coding tree used for CUs. In recent developments in video compression technology, a quad-tree and binary tree (QTBT) partition frame is used to partition the coding block. In the QTBT block structure, the CU may be square or rectangular.
[0107] In this specification, for ease of explanation and understanding, an image block to be coded in a current coding image may be called a current block. For example, in encoding, the current block is a block currently being coded; and in decoding, the current block is a block currently being decoded. A decoded image block in a reference image used to predict a current block is called a reference block. In other words, a reference block is a block that provides a reference signal to the current block, where the reference signal represents a pixel value in the image block. A block that provides a prediction signal to a current block in a reference image may be called a prediction block, where the prediction signal represents a pixel value, a sample value, or a sample signal in the prediction block. For example, an optimal reference block is found after traversing multiple reference blocks, and the optimal reference block provides a prediction of the current block, where this block is called a prediction block.
[0108] In the case of lossless video coding, the original video picture can be reconstructed. In other words, the reconstructed video picture has the same quality as the original video picture (assuming no transmission or other data loss occurs during storage or transmission). In the case of lossy video coding, further compression is performed, for example by quantization, to reduce the amount of data required to represent the video picture, and the video picture cannot be completely reconstructed at the decoder side. In other words, the quality of the reconstructed video picture is lower or inferior to the quality of the original video picture.
[0109] Some H.261 video coding standards are used for "lossy hybrid video coding" (i.e., spatial and temporal prediction in the sample domain is combined with 2D transform coding to apply quantization in the transform domain). Each picture of a video sequence is typically partitioned into a set of non-overlapping blocks, and coding is typically performed at the block level. In other words, at the encoder side, a video is typically processed, i.e., encoded, at the block (video block) level, e.g., using spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to generate a predictive block, subtract the predictive block from a current block (the block currently being processed or to be processed) to obtain a residual block, transform the residual block, and quantize the residual block in the transform domain to reduce the amount of data to be transmitted (compressed), whereas at the decoder side, a reverse process compared to that of the encoder is applied to the coded or compressed block to reconstruct the current block for representation. Additionally, the encoder replicates the decoder processing loop, so that the encoder and decoder generate identical predictions (eg, intra-prediction and inter-prediction) and / or reconstructions for processing, i.e., coding, subsequent blocks.
[0110] The following describes a system architecture to which an embodiment of the present application is applied. FIG. 1A is a schematic block diagram of an example of a video coding system 10 to which an embodiment of the present application is applied. As shown in FIG. 1A, the video coding system 10 may include a source device 12 and a destination device 14. The source device 12 generates encoded video data, and therefore the source device 12 may be referred to as a video encoding device. The destination device 14 can decode the encoded video data generated by the source device 12, and therefore the destination device 14 may be referred to as a video decoding device. The source device 12, the destination device 14, or solutions of various implementations of the source device 12 or the destination device 14 may include one or more processors and memories coupled to the one or more processors. The memories include, but are not limited to, RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store a desired program code in the form of computer-accessible instructions or data structures as described herein. The source device 12 and destination device 14 may include a variety of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, handheld phones such as “smartphones,” television sets, cameras, display devices, digital media players, video game consoles, in-vehicle computers, wireless communication devices, or similar devices.
[0111] 1A illustrates source device 12 and destination device 14 as separate devices, an embodiment of the devices may include both or both functions (source device 12 or corresponding functions and destination device 14 or corresponding functions). In such an embodiment, source device 12 or corresponding functions and destination device 14 or corresponding functions may be implemented using the same hardware and / or software, or using separate hardware and / or software, or any combination thereof.
[0112] The communication connection between the source device 12 and the destination device 14 may be implemented via a link 13, and the destination device 14 may receive the encoded video data from the source device 12 via the link 13. The link 13 may include one or more media or equipment that may move the encoded video data from the source device 12 to the destination device 14. In one example, the link 13 may include one or more communication media that enable the source device 12 to transmit the encoded video data directly to the destination device 14 in real time. In this example, the source device 12 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and transmit the modulated video data to the destination device 14. The one or more communication media may include wireless communication media and / or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, which may be, for example, a local area network, a wide area network, or a global network (e.g., the Internet). The communication medium or mediums may include a router, a switch, a base station, or another device that facilitates communication from source device 12 to destination device 14 .
[0113] Source device 12 includes an encoder 20, and optionally, source device 12 may further include a picture source 16, a picture preprocessor 18, and a communication interface 22. In certain implementations, the encoder 20, the picture source 16, the picture preprocessor 18, and the communication interface 22 may be hardware components of source device 12 or may be software programs on source device 12. A detailed description follows.
[0114] The picture source 16 may include or be, for example, any type of picture capture device configured to capture real-world pictures; and / or any type of device for generating pictures or comments (in case of encoding of screen content, also some text on the screen is considered as part of the picture or image to be encoded), for example a computer graphics processor configured to generate computer animated pictures; or any type of device configured to obtain and / or provide real-world or computer animated pictures (e.g. screen content or virtual reality (VR) pictures); and / or any combination thereof (e.g. augmented reality (AR) pictures). The picture source 16 may be a camera configured to capture pictures, or a memory configured to store pictures. The picture source 16 may further include any type of (internal or external) interface for storing previously captured or generated pictures and / or for obtaining or receiving pictures. If the picture source 16 is a camera, it can be, for example, a local camera or an integrated camera integrated in the source device. If the picture source 16 is a memory, it can be, for example, a local memory or an integrated memory integrated in the source device. If the picture source 16 includes an interface, it can be, for example, an external interface for receiving pictures from an external video source. The external video source can be, for example, an external picture capturing device such as a camera, an external memory, or an external picture generating device. The external picture generating device can be, for example, an external computer graphics processor, a computer, or a server. The interface can be any type of interface, for example, a wired or wireless interface or an optical interface, according to any proprietary or standardized interface protocol.
[0115] A picture can be considered as a two-dimensional array or matrix of picture elements. The picture elements in the array may also be referred to as samples. The number of samples in the horizontal and vertical directions (or axes) of the array or picture defines the size and / or resolution of the picture. Three color components are typically used to represent color. For example, a picture may be represented as or include three sample arrays. For example, in an RGB format or color space, a picture includes corresponding red, green, and blue sample arrays. However, in video coding, each pixel is typically represented in a luminance / chrominance format or color space. For example, a picture in a YUV format includes a luminance component denoted Y (although L may be used instead) and two chrominance components denoted U and V. The luminance (luma) component Y represents the brightness or intensity of the gray level (e.g., both are the same in a grayscale picture), while the two chrominance (chroma) components U and V represent the chromaticity or color information components. Thus, a picture in YUV format includes a luma sample array of luma sample values (Y) and two chroma sample arrays of chroma values (U and V). A picture in RGB format can be converted or transformed into a picture in YUV format and vice versa, a process also known as color conversion or transformation. If the picture is monochrome, the picture may only include a luma sample array. In this embodiment of the present application, the image sent by the picture source 16 to the picture processor may also be referred to as raw picture data 17.
[0116] The picture preprocessor 18 is configured to receive raw picture data 17 and to preprocess the raw picture data 17 to obtain a preprocessed picture 19 or preprocessed picture data 19. The preprocessing performed by the picture preprocessor 18 may include cropping, color format conversion (e.g. from RGB format to YUV format), color correction, or noise removal.
[0117] Encoder 20 (also referred to as video encoder 20) is configured to receive pre-processed picture data 19 and process the pre-processed picture data 19 using an associated prediction mode (such as the prediction mode of the embodiments herein) to provide encoded picture data 21. (Details of the structure of encoder 20 are further described below with reference to FIG. 2, FIG. 4, or FIG. 5.) In some embodiments, encoder 20 may be configured to execute the embodiments described below to implement the encoder-side application of the chroma block prediction methods described herein.
[0118] The communications interface 22 may be configured to receive the encoded picture data 21 and transmit the encoded picture data 21 over the link 13 to the destination device 14 or any other device (e.g., memory) for storage or direct reconstruction. The any other device may be any device used for decoding or storage. The communications interface 22 may be configured, for example, to package the encoded picture data 21 into an appropriate format, for example, data packets, for transmission over the link 13.
[0119] The destination device 14 includes a decoder 30, and optionally, the destination device 14 may further include a communication interface 28, a picture post-processor 32, and a display device 34. A detailed description of each is provided below.
[0120] The communication interface 28 may be configured to receive the encoded picture data 21 from the source device 12 or any other source. The any other source may be, for example, a storage device, which may be, for example, a storage device for the encoded picture data. The communication interface 28 may be configured to transmit or receive the encoded picture data 21 via a link 13 between the source device 12 and the destination device 14 or via any type of network. The link 13 may be, for example, a direct wired or wireless connection, and the any type of network may be, for example, a wired or wireless network or any combination thereof, or any type of private or public network, or any combination thereof. The communication interface 28 may be configured, for example, to de-package data packets transmitted via the communication interface 22 to obtain the encoded picture data 21.
[0121] Both communication interface 28 and communication interface 22 may be configured as unidirectional or bidirectional communication interfaces and may be configured, for example, to send and receive messages to set up a connection, verify and exchange a communication link and / or any other information related to data transmission, such as encoded picture data transmission.
[0122] The decoder 30 (also referred to as decoder 30) is configured to receive the encoded picture data 21 and to provide decoded picture data 31 or a decoded picture 31 (details of the structure of the decoder 30 are further described below with reference to Figure 3, Figure 4 or Figure 5). In some embodiments, the decoder 30 may be configured to execute the embodiments described below in order to implement the application of the decoder-side chroma block prediction methods described herein.
[0123] The picture post-processor 32 is configured to post-process the decoded picture data 31 (also called reconstructed picture data) to obtain post-processed picture data 33. The post-processing performed by the picture post-processor 32 may include color format conversion (e.g. from YUV format to RGB format), color correction, cropping, resampling, or any other processing. The picture post-processor 32 may further be configured to transmit the post-processed picture data 33 to a display device 34.
[0124] The display device 34 is configured to receive the post-processed picture data 33 and display the picture, for example to a user or viewer. The display device 34 may be or include any type of display configured to present the reconstructed picture, for example an integrated or external display or monitor. For example, the display may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display.
[0125] 1A illustrates source device 12 and destination device 14 as separate devices, an embodiment of the devices may include both or both functions (source device 12 or corresponding functions, and destination device 14 or corresponding functions). In such an embodiment, source device 12 or corresponding functions, and destination device 14 or corresponding functions may be implemented using the same hardware and / or software, or using separate hardware and / or software, or any combination thereof.
[0126] Based on the description, one skilled in the art may understand that the presence and (exact) division into functions or different units of the source device 12 and / or destination device 14 shown in Fig. 1A may vary depending on the actual device and application. The source device 12 and destination device 14 may each be any one of a wide range of devices including any type of handheld or fixed device, such as a notebook or laptop computer, a mobile phone, a smart phone, a pad or tablet computer, a video camera, a desktop computer, a set-top box, a television, a camera, an in-vehicle device, a display device, a digital media player, a video game console, a video streaming device (such as a content service server or a content delivery server), a broadcast receiver device, or a broadcast transmitter device, and may not use any type of operating system or may use an operating system.
[0127] The encoder 20 and the decoder 30 may each be implemented as any of a variety of suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. Where the techniques are implemented in part using software, an apparatus may store software instructions in a suitable non-transitory computer-readable storage medium and execute the instructions using hardware, such as one or more processors, to perform the techniques of this disclosure. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be considered to be one or more processors.
[0128] In some cases, the video coding system 10 shown in FIG. 1A is merely an example, and the techniques of the present application are applicable to video coding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between an encoding device and a decoding device. In other examples, data may be retrieved from a local memory, streamed over a network, etc. A video encoding device may encode data and store the encoded data in memory, and / or a video decoding device may retrieve data from memory and decode data. In some examples, encoding and decoding are performed by devices that do not communicate with each other, but simply encode data into memory and / or retrieve data from memory and decode data.
[0129] 1B is an illustrative diagram of an example of a video coding system 40 including the encoder 20 of FIG. 2 and / or the decoder 30 of FIG. 3, according to an example embodiment. The video coding system 40 may implement a combination of various techniques in embodiments of the present application. In the illustrated implementation, the video coding system 40 may include an imaging device 41, an encoder 20, a decoder 30 (and / or a video encoder / decoder implemented by logic circuitry 47 of a processing unit 46), an antenna 42, one or more processors 43, one or more memories 44, and / or a display device 45.
[0130] 1B, imaging device 41, antenna 42, processing unit 46, logic circuitry 47, encoder 20, decoder 30, processor 43, memory 44, and / or display device 45 may be in communication with one another. As discussed, although video coding system 40 is shown using encoder 20 and decoder 30, video coding system 40 may include only encoder 20 or only decoder 30 in different examples.
[0131] In some examples, antenna 42 may be configured to transmit or receive an encoded bitstream of video data. Further, in some examples, display device 45 may be configured to present the video data. In some examples, logic circuitry 47 may be implemented by processing unit 46. Processing unit 46 may include application-specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, etc. Video coding system 40 may alternatively include optional processor 43. Optional processor 43 may also include application-specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, etc. In some examples, logic circuitry 47 may be implemented by hardware, e.g., dedicated video coding hardware, and processor 43 may be implemented by general-purpose software, an operating system, etc. Further, memory 44 may be any type of memory, such as volatile memory (e.g., Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), or non-volatile memory (e.g., flash memory). In a non-limiting example, memory 44 may be implemented by a cache memory. In some examples, logic circuitry 47 may access memory 44 (e.g., for implementing an image buffer). In other examples, logic circuitry 47 and / or processing unit 46 may include memory (e.g., a cache) for implementing an image buffer, etc.
[0132] In some examples, encoder 20 implemented by logic circuitry may include an image buffer (e.g., implemented by processing unit 46 or memory 44) and a graphics processing unit (e.g., implemented by processing unit 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include encoder 20 implemented by logic circuitry 47 to implement various modules described with reference to FIG. 2 and / or any other encoder system or subsystems described herein. The logic circuitry may be configured to perform various operations described herein.
[0133] In some examples, the decoder 30 may be implemented by logic circuitry 47 in a similar manner to implement the various modules described with reference to the decoder 30 of FIG. 3 and / or any other decoder system or subsystem described herein. In some examples, the decoder 30 implemented by logic circuitry may include an image buffer (e.g., implemented by processing unit 2820 or memory 44) and a graphics processing unit (e.g., implemented by processing unit 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include the decoder 30 implemented by logic circuitry 47 to implement the various modules described with reference to FIG. 3 and / or any other decoder system or subsystem described herein.
[0134] In some examples, antenna 42 may be configured to receive an encoded bitstream of video data. As described, the encoded bitstream may include data, indicators, index values, mode selection data, etc., related to the video frame coding described herein, such as data related to coding partitioning (e.g., transform coefficients or quantized transform coefficients, optional indicators (as described), and / or data defining the coding partitioning). Video coding system 40 may further include a decoder 30 coupled to antenna 42 and configured to decode the encoded bitstream. Display device 45 is configured to present the video frames.
[0135] In this embodiment of the present application, in the examples described with reference to the encoder 20, it should be understood that the decoder 30 may be configured to perform the inverse process. With respect to signaling syntax elements, the decoder 30 may be configured to receive and parse such syntax elements and correspondingly decode the associated video data. In some examples, the encoder 20 may entropy code the syntax elements into the encoded video bitstream. In such examples, the decoder 30 may parse such syntax elements and correspondingly decode the associated video data.
[0136] It should be noted that the decoding method described in this embodiment of the present application is mainly used in the decoding process, which exists in both the encoder 20 and the decoder 30.
[0137] Figure 2 is a schematic / conceptual block diagram of an example of an encoder 20 configured to implement an embodiment of the present application. In the example of Figure 2, the encoder 20 includes a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a buffer 216, a loop filter unit 220, a decoded picture buffer (DPB) 230, a prediction processing unit 260, and an entropy coding unit 270. The prediction processing unit 260 may include an inter prediction unit 244, an intra prediction unit 254, and a mode selection unit 262. The inter prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). The encoder 20 shown in Figure 2 may also be referred to as a hybrid video encoder or a video encoder based on a hybrid video codec.
[0138] For example, the residual computation unit 204, the transform processing unit 206, the quantization unit 208, the prediction processing unit 260, and the entropy coding unit 270 form a forward signal path of the encoder 20, while for example the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoded picture buffer (DPB) 230, and the prediction processing unit 260 form a backward signal path of the encoder. The backward signal path of the encoder corresponds to the signal path of the decoder (see decoder 30 in FIG. 3).
[0139] The encoder 20 receives, for example, via an input 202, a picture 201 or an image block 203 of the picture 201, for example a picture in a series of pictures forming a video or a video sequence. The image block 203 may also be called a current picture block or a picture block to be coded. The picture 201 may also be called a current picture or a picture to be coded (particularly in video coding, to distinguish the current picture from other pictures, which are for example previously coded and / or decoded pictures in the same video sequence, i.e. a video sequence which also includes the current picture).
[0140] An embodiment of the encoder 20 may include a partitioning unit (not shown in FIG. 2) configured to partition a picture 201 into multiple blocks, such as image blocks 203. The picture 201 is typically partitioned into multiple non-overlapping blocks. The partitioning unit may be configured to use the same block size for all pictures in a video sequence, use a corresponding grid that defines the block sizes, or vary the block size among pictures or subsets or groups of pictures, and partition each picture into corresponding blocks.
[0141] In one example, prediction processing unit 260 of encoder 20 may be configured to perform any combination of the above partitioning techniques.
[0142] Similar to picture 201, image block 203 may also be or be considered as a two-dimensional array or matrix of samples having sample values, although the size of image block 203 is smaller than the size of picture 201. In other words, image block 203 may include, for example, one sample array (e.g., a luma array in case of monochrome picture 201), three sample arrays (e.g., one luma array and two chroma arrays in case of color picture), or any other quantity and / or type of arrays depending on the applied color format. The number of samples in the horizontal and vertical directions (or axes) of image block 203 defines the size of image block 203.
[0143] The encoder 20 shown in FIG. 2 is arranged to code a picture 201 block-by-block, eg to code and predict each image block 203 .
[0144] The residual calculation unit 204 is configured to calculate the residual block 205 based on the picture image block 203 and the prediction block 265 (further details regarding the prediction block 265 are provided below), for example to obtain the residual block 205 in the sample domain by subtracting sample values of the prediction block 265 from the sample values of the picture image block 203 on a sample-by-sample (pixel-by-pixel) basis.
[0145] The transform processing unit 206 is configured to apply a transform, such as a discrete cosine transform (DCT) or a discrete sine transform (DST), to the sample values of the residual block 205 to obtain transform coefficients 207 in a transform domain. The transform coefficients 207 may also be referred to as transform residual coefficients and represent the residual block 205 in the transform domain.
[0146] The transform processing unit 206 may be configured to apply an integer approximation of a DCT / DST, such as a transform specified in AVS, AVS2, or AVS3. Compared to an orthogonal DCT transform, such an integer approximation is typically scaled by a certain factor. Applying additional scale factors to preserve the norm of the residual blocks processed by the forward and inverse transforms is part of the transform process. The scale factors are typically selected based on some constraints. For example, the scale factors are powers of two for shift operations, bit depth of the transform coefficients, a trade-off between accuracy and implementation cost, etc. A specific scale factor is specified, for example, for the inverse transform, e.g., by the inverse transform processing unit 212 of the decoder side 30 (and the corresponding inverse transform, e.g., by the inverse transform processing unit 212 of the encoder side 20), and the corresponding scale factor of the forward transform, e.g., by the transform processing unit 206 of the encoder side 20, may be specified accordingly.
[0147] The quantization unit 208 is configured to quantize the transform coefficients 207, for example by applying scalar quantization or vector quantization, to obtain quantized transform coefficients 209. The quantized transform coefficients 209 may also be referred to as quantized residual coefficients 209. The quantization process may reduce a bit depth associated with some or all of the transform coefficients 207. For example, an n-bit transform coefficient may be truncated to an m-bit transform coefficient during quantization, where n is greater than m. The quantization degree may be modified by adjusting a quantization parameter (QP). For example, in case of scalar quantization, different scales may be applied to achieve finer or coarser quantization. A smaller quantization step corresponds to a finer quantization, while a larger quantization step corresponds to a coarser quantization. An appropriate quantization step size may be indicated by a quantization parameter (QP). The quantization parameter may be, for example, an index to a predefined set of appropriate quantization step sizes. For example, a smaller quantization parameter may correspond to a finer quantization (smaller quantization step size) and a larger quantization parameter may correspond to a coarser quantization (larger quantization step size), or vice versa. Quantization may include division by the quantization step size and corresponding quantization and / or inverse quantization, for example, performed by the inverse quantization unit 210, or may include multiplication by the quantization step size. In embodiments according to some standards, such as AVS, AVS2, and AVS3, the quantization parameter may be used to determine the quantization step size. In general, the quantization step size may be calculated based on the quantization parameter using a fixed-point approximation of an equation that includes a division. To restore the norm of the residual block, an additional scaling factor may be introduced into the quantization and dequantization, where the norm of the residual block may be modified due to the scale used in the fixed-point approximation of the equation of the quantization step size and the quantization parameter.In an exemplary implementation, the inverse transform and dequantization scales may be combined. Alternatively, customized quantization tables can be used and signaled from the encoder to the decoder, for example, in the bitstream. Quantization is a lossy operation, where larger quantization step sizes are more lossy.
[0148] The inverse quantization unit 210 is configured to apply the inverse quantization of the quantization unit 208 to the quantized coefficients to obtain dequantized coefficients 211, e.g., to apply an inverse quantization scheme of the quantization scheme applied by the quantization unit 208 based on or using the same quantization step size as the quantization unit 208. The dequantized coefficients 211, also referred to as dequantized residual coefficients 211, correspond to the transform coefficients 207, but are typically not identical to the transform coefficients due to losses due to quantization.
[0149] The inverse transform processing unit 212 is configured to apply an inverse transform of the transform applied by the transform processing unit 206, for example, an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST), to obtain an inverse transform block 213 in the sample domain. The inverse transform block 213 may also be referred to as an inverse transform dequantization block 213 or an inverse transform residual block 213.
[0150] The reconstruction unit 214 (e.g., summer 214) is configured to add the inverse transform block 213 (i.e., the reconstructed residual block 213) to the prediction block 265 to obtain the reconstructed block 215 in the sample domain, e.g., by adding the sample values of the reconstructed residual block 213 and the sample values of the prediction block 265.
[0151] Optionally, a buffer unit 216 (or "buffer" 216 for short), e.g., of the line buffer 216, is configured to buffer or store the reconstructed block 215 and corresponding sample values, e.g., for intra prediction. In other embodiments, the encoder may be configured to use the unfiltered reconstructed block and / or corresponding sample values stored in the buffer unit 216 for any type of estimation and / or prediction, e.g., intra prediction.
[0152] For example, one embodiment of encoder 20 may be configured such that buffer unit 216 is not only used to store reconstructed block 215 for intra prediction unit 254, but also for loop filter unit 220 (not shown in FIG. 2), and / or that buffer unit 216 and decoded picture buffer unit 230 form one buffer, for example. In other embodiments, filtered block 221 and / or blocks or samples from decoded picture buffer 230 (blocks or samples not shown in FIG. 2) are used as input or basis for intra prediction unit 254.
[0153] The loop filter unit 220 (simply referred to as “loop filter” 220) is configured to filter the reconstructed block 215 to obtain a filtered block 221 to smooth pixel transitions or improve video quality. The loop filter unit 220 is intended to represent one or more loop filters, including, for example, a deblocking filter, a sample-adaptive offset (SAO) filter, or another filter, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, or a collaborative filter. Although the loop filter unit 220 is illustrated in FIG. 2 as an in-loop filter, in another configuration, the loop filter unit 220 may be implemented as a post-loop filter. The filtered block 221 may also be referred to as a filtered reconstruction block 221. The decoded picture buffer 230 may store the reconstructed coding block after the loop filter unit 220 performs a filtering operation on the reconstructed coding block.
[0154] In one embodiment, encoder 20 (correspondingly, loop filter unit 220) may be configured to output loop filter parameters (such as sample adaptive offset information), e.g., directly or after entropy coding performed by entropy coding unit 270 or any other entropy coding unit, so that, e.g., decoder 30 can receive the same loop filter parameters and apply the same loop filter parameters for decoding.
[0155] The decoded picture buffer (DPB) 230 may be a reference picture memory that stores reference picture data for encoding the video data by the encoder 20. The DPB 230 may be formed by any one of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), and resistive RAM (RRAM)), or another type of memory device. The DPB 230 and the buffer 216 may be provided by the same memory device or separate memory devices. In one example, the decoded picture buffer (DPB) 230 is configured to store the filtered block 221. The decoded picture buffer 230 may further be configured to store another previously filtered block of the same current picture or a different picture, e.g., a previously reconstructed picture, e.g., the previously reconstructed and filtered block 221, and may provide a fully previously reconstructed, i.e., decoded picture (and corresponding reference blocks and samples) and / or a partially reconstructed current picture (and corresponding reference blocks and samples), e.g., for inter prediction. In one example, the decoded picture buffer (DPB) 230 is configured to store the reconstructed block 215 when the reconstructed block 215 is reconstructed without in-loop filtering.
[0156] The prediction processing unit 260, also referred to as block prediction processing unit 260, is configured to receive or obtain an image block 203 (the current image block 203 of the current picture 201) and reconstructed picture data, e.g. reference samples of the same (current) picture from the buffer 216 and / or reference picture data 231 of one or more previously decoded pictures from the decoded picture buffer 230, and to process such data for prediction, i.e. to provide a prediction block 265, which may be an inter prediction block 245 or an intra prediction block 255.
[0157] The mode selection unit 262 may be configured to select a prediction mode (e.g., intra prediction mode or inter prediction mode) and / or a corresponding prediction block 245 or 255 to be used as the prediction block 265 to calculate the residual block 205 and reconstruct the reconstruction block 215.
[0158] In one embodiment, mode selection unit 262 may be configured to select a prediction mode (e.g., from prediction modes supported by prediction processing unit 260) that provides the best match or smaller residual (smaller residual means better compression for transmission or storage), or that provides less signaling overhead (less signaling overhead means better compression for transmission or storage), or that considers or balances both. Mode selection unit 262 may be configured to determine the prediction mode based on rate distortion optimization (RDO), i.e., to select a prediction mode that provides the smallest rate distortion, or to select a prediction mode whose associated rate distortion at least meets a selection criterion for the prediction mode.
[0159] The prediction process performed (eg, using prediction processing unit 260) and mode selection performed (eg, using mode selection unit 262) according to an example encoder 20 are described in detail below.
[0160] As mentioned above, the encoder 20 is configured to determine or select an optimal or best prediction mode from a (predetermined) set of prediction modes, which may include, for example, intra-prediction modes and / or inter-prediction modes.
[0161] The set of intra prediction modes may include 35 different intra prediction modes, e.g., omnidirectional modes such as DC (or average) mode and planar mode, or directional modes defined in H.265, or may include 67 different intra prediction modes, e.g., omnidirectional modes such as DC (or average) mode and planar mode, or directional modes defined in the currently developing H.266.
[0162] In a possible implementation, the set of inter prediction modes depends on the available reference pictures (i.e., at least some of the decoded pictures stored in DBP 230 as described above), and other inter prediction parameters, for example, whether the entire reference picture is used or only a part of the reference picture, for example, a best-matching reference block found in a search window region surrounding the region of the current block, is used, and / or whether pixel interpolation, such as, for example, half-pixel and / or quarter-pixel interpolation, is applied. The set of inter prediction modes may include, for example, an Advanced Motion Vector Prediction (AMVP) mode and a merge mode. In a particular implementation, the set of inter prediction modes may include, in this embodiment of the present application, an improved control point-based AMVP mode and an improved control point-based merge mode. In one example, the intra prediction unit 254 may be configured to perform any combination of the inter prediction techniques described below.
[0163] In addition to the aforementioned prediction modes, skip mode and / or direct mode may also be used in this embodiment of the present application.
[0164] The prediction processing unit 260 may be further configured to partition the image block 203 into smaller block partitions or sub-blocks, e.g., using quad-tree (QT) partitioning, binary-tree (BT) partitioning, ternary tree (TT) partitioning, extended quad-tree (EQT) partitioning, or any combination thereof iteratively, and perform prediction for each block partition or sub-block, e.g., where the mode selection includes selecting a tree structure for the partitioned image block 203 and selecting a prediction mode to be applied to each block partition or sub-block.
[0165] The inter prediction unit 244 may include a motion estimation (ME) unit (not shown in FIG. 2) and a motion compensation (MC) unit (not shown in FIG. 2). The motion estimation unit is configured to receive or obtain the picture image block 203 (the current picture image block 203 of the current picture 201) and the decoded picture 231, or at least one or more previously reconstructed blocks, such as reconstructed blocks of one or more other / different previously decoded pictures 231 for motion estimation. For example, a video sequence may include the current picture and the previously decoded picture 31. In other words, the current picture and the previously decoded picture 31 may be part of or form a sequence of pictures forming a video sequence.
[0166] For example, the encoder 20 may be configured to select a reference block from multiple reference blocks of the same picture or different ones of multiple other pictures, provide the reference picture to a motion estimation unit (not shown in FIG. 2 ), and / or provide an offset (spatial offset) between the position of the reference block (coordinates X and Y) and the position of the current block as an inter-prediction parameter, also called a motion vector (MV).
[0167] The motion compensation unit is configured to obtain inter prediction parameters and perform inter prediction based on or using the inter prediction parameters to obtain inter prediction block 245. The motion compensation performed by the motion compensation unit (not shown in FIG. 2) may include fetching or generating a prediction block based on a motion / block vector determined by motion estimation (possibly performing sub-pixel accuracy interpolation). Interpolation filtering may generate additional pixel samples from known pixel samples, thereby potentially increasing the number of candidate prediction blocks that may be used to code the picture block. Upon receiving the motion vector for the PU of the current picture block, the motion compensation unit 246 may locate the prediction block to which the motion vector points in one reference picture list. The motion compensation unit 246 may further generate syntax elements related to the block and the video slice for decoding the picture block of the video slice by the decoder 30.
[0168] Specifically, the inter prediction unit 244 can send a syntax element to the entropy encoding unit 270, where the syntax element includes inter prediction parameters (e.g., index information of an inter prediction mode selected for prediction of a current block after traversing multiple inter prediction modes). In a possible application scenario, when there is only one inter prediction mode, the inter prediction parameters may not be transmitted in the syntax element. In this case, the decoder side 30 can directly use a default prediction mode for decoding. It can be understood that the inter prediction unit 244 can be configured to perform any combination of inter prediction techniques.
[0169] The intra prediction unit 254 is configured to, for example, obtain the picture block 203 (current picture block) and one or more previously reconstructed blocks of the same picture, for example reconstructed neighboring blocks, for intra estimation. For example, the encoder 20 may be configured to select an intra prediction mode from a plurality of (predetermined) intra prediction modes.
[0170] In one embodiment, the encoder 20 may be configured to select the intra prediction mode according to an optimization criterion, for example, based on a smaller residual (e.g., the intra prediction mode that provides the predicted block 255 that is most similar to the current picture block 203) or minimum bitrate distortion.
[0171] The intra prediction unit 254 is further configured to determine the intra prediction block 255 based on, for example, intra prediction parameters of the selected intra prediction mode. In either case, after selecting an intra prediction mode for the block, the intra prediction unit 254 is further configured to provide the intra prediction parameters, i.e., information indicative of the selected intra prediction mode for the block, to the entropy coding unit 270. In one example, the intra prediction unit 254 may be configured to perform any combination of intra prediction techniques.
[0172] Specifically, the intra prediction unit 254 can send a syntax element to the entropy encoding unit 270, and the syntax element includes intra prediction parameters (e.g., index information of an intra prediction mode selected for prediction of a current block after traversing multiple intra prediction modes). In a possible application scenario, when there is only one intra prediction mode, the intra prediction parameters may not be transmitted in the syntax element. In this case, the decoder side 30 can directly use the default prediction mode for decoding.
[0173] The entropy coding unit 270 is configured to apply (or bypass) an entropy coding algorithm or scheme (e.g., a variable length coding (VLC) scheme, a context adaptive VLC (CAVLC) scheme, an arithmetic coding scheme, context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding scheme or technique) to one or all of the following (the quantization coefficients 209, the inter-prediction parameters, the intra-prediction parameters, and / or the loop filter parameters) to obtain coded picture data 21, which may be output via an output 272, for example in the form of a coded bitstream 21. The coded bitstream may be transmitted to the video decoder 30, archived for later transmission, or retrieved by the video decoder 30. Entropy encoding unit 270 may be further configured to entropy encode another syntax element of the current video slice being encoded.
[0174] Other structural variations of the video encoder 20 may be configured to encode the video stream. For example, a non-transform-based encoder 20 may directly quantize the residual signal for some blocks or frames without the transform processing unit 206. In another implementation, the encoder 20 may include the quantization unit 208 and the inverse quantization unit 210 combined into a single unit.
[0175] Specifically, in this embodiment of the present application, the encoder 20 may be configured to implement the video encoding methods described in the following embodiments.
[0176] It should be understood that other structural variations of the video encoder 20 may be configured to encode the video stream. For example, for some image blocks or image frames, the video encoder 20 may directly quantize the residual signal without processing by the transform processing unit 206, and therefore without processing by the inverse transform processing unit 212. Alternatively, for some image blocks or image frames, the video encoder 20 may not generate residual data, and therefore the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, and the inverse transform processing unit 212 do not need to perform processing. Alternatively, the video encoder 20 may directly store the reconstructed image block as a reference block, without processing by the filter 220. Alternatively, the quantization unit 208 and the inverse quantization unit 210 in the video encoder 20 may be combined together. The loop filter 220 is optional, and in the case of lossless compression coding, the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, and the inverse transform processing unit 212 are optional. It should be understood that the inter prediction unit 244 and the intra prediction unit 254 can be selectively enabled according to different application scenarios.
[0177] 3 is a schematic / conceptual block diagram of an example of a decoder 30 configured to implement an embodiment of the present application. The video decoder 30 is configured to receive encoded picture data (e.g., encoded bitstream) 21, e.g., resulting from encoding by the encoder 20, to obtain a decoded picture 231. During decoding, the video decoder 30 receives video data from the video encoder 20, e.g., an encoded video bitstream representing picture blocks of encoded video slices, and associated syntax elements.
[0178] 3, the decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., summer 314), a buffer 316, a loop filter 320, a decoded picture buffer 330, and a prediction processing unit 360. The prediction processing unit 360 may include an inter prediction unit 344, an intra prediction unit 354, and a mode selection unit 362. In some examples, the video decoder 30 may perform a decoding process that is generally the reverse of the encoding process described with respect to the video encoder 20 of FIG.
[0179] The entropy decoding unit 304 is configured to entropy decode the encoded picture data 21 to obtain, e.g., quantized coefficients 309 and / or decoded coding parameters (not shown in FIG. 3), e.g., any one or all of the following: (decoded) inter-prediction parameters, intra-prediction parameters, loop filter parameters, and / or other syntax elements. The entropy decoding unit 304 is further configured to forward the inter-prediction parameters, intra-prediction parameters, and / or other syntax elements to the prediction processing unit 360. The video decoder 30 may receive the syntax elements at a video slice level and / or a video block level.
[0180] The inverse quantization unit 310 may have the same functionality as the inverse quantization unit 110. The inverse transform processing unit 312 may have the same functionality as the inverse transform processing unit 212. The reconstruction unit 314 may have the same functionality as the reconstruction unit 214. The buffer 316 may have the same functionality as the buffer 216. The loop filter 320 may have the same functionality as the loop filter 220. The decoded picture buffer 330 may have the same functionality as the decoded picture buffer 230.
[0181] Prediction processing unit 360 may include an inter prediction unit 344 and an intra prediction unit 354. The inter prediction unit 344 may be similar in function to the inter prediction unit 244, and the intra prediction unit 354 may be similar in function to the intra prediction unit 254. Prediction processing unit 360 is typically configured to perform block prediction and / or obtain a prediction block 365 from the encoded data 21, and to receive or obtain (explicitly or implicitly) prediction-related parameters and / or information regarding a selected prediction mode, for example from the entropy decoding unit 304.
[0182] If the video slice is coded as an intra-coded (I) slice, intra prediction unit 354 of prediction processing unit 360 is configured to generate a prediction block 365 of a picture block of the current video slice based on a signaled intra prediction mode and data from a previously decoded block of the current frame or picture. If the video frame is coded as an inter-coded (B or P) slice, inter prediction unit 344 (e.g., a motion compensation unit) of prediction processing unit 360 is configured to generate a prediction block 365 of a video block of the current video slice based on a motion vector and another syntax element received from entropy decoding unit 304. For inter prediction, the prediction block may be generated from one of the reference pictures in one reference picture list. Video decoder 30 may construct the reference frame lists (List 0 and List 1) using a default construction technique based on the reference pictures stored in DPB 330.
[0183] Prediction processing unit 360 is configured to determine a predictive block for a video block of a current video slice by analyzing the motion vector and another syntax element, and to generate a predictive block for a current video block being decoded using the predictive block. In one example of the present application, prediction processing unit 360 uses some of the received syntax elements to determine a prediction mode (e.g., intra prediction or inter prediction) used to code the video blocks of the video slice, an inter-prediction slice type (e.g., B slice, P slice, or GPB slice), configuration information of one or more reference picture lists of the slice, a motion vector for each inter-coded video block of the slice, an inter-prediction status for each inter-coded video block of the slice, and other information for decoding the video blocks of the current video slice. In another example of the present disclosure, the syntax elements received by video decoder 30 from the bitstream include one or more of an adaptive parameter set (APS), a sequence parameter set (SPS), a picture parameter set (PPS), or a slice header.
[0184] The inverse quantization unit 310 may be configured to inverse quantize (i.e., dequantize) the quantized transform coefficients provided to the bitstream and decoded by the entropy decoding unit 304. The inverse quantization process may include using quantization parameters calculated by the video encoder 20 for each video block in a video slice to determine a degree of quantization to apply, and similarly, to determine a degree of inverse quantization to apply.
[0185] The inverse transform processing unit 312 is configured to apply an inverse transform (eg, an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to produce residual blocks in the pixel domain.
[0186] The reconstruction unit 314 (e.g., adder 314) is configured to add the inverse transform block 313 (i.e., the reconstructed residual block 313) to the prediction block 365 to obtain the reconstructed block 315 in the sample domain, e.g., by adding sample values of the reconstructed residual block 313 and sample values of the prediction block 365.
[0187] The loop filter unit 320 (during or after the coding loop) is configured to filter the reconstructed block 315 to obtain a filtered block 321 to smooth pixel transitions or improve video quality. In one example, the loop filter unit 320 is configured to perform any combination of the filtering techniques described below. The loop filter unit 320 is intended to represent one or more loop filters, including, for example, a deblocking filter, a sample-adaptive offset (SAO) filter, or another filter, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, or a collaborative filter. Although the loop filter unit 320 is illustrated in FIG. 3 as an in-loop filter, in another configuration, the loop filter unit 320 may be implemented as a post-loop filter.
[0188] The decoded video blocks 321 of a given frame or picture are then stored in a decoded picture buffer 330, which stores reference pictures used for subsequent motion compensation.
[0189] The decoder 30 is arranged to output the decoded pictures 31 via an output 332, for example for presentation to or viewing by a user.
[0190] Other variations of the video decoder 30 may be configured to decode the compressed bitstream. For example, the decoder 30 may generate an output video stream without processing by the loop filter unit 320. For example, a non-transform-based decoder 30 may directly inverse quantize the residual signal for some blocks or frames, without the inverse transform processing unit 312. In another implementation, the video decoder 30 may include the inverse quantization unit 310 and the inverse transform processing unit 312 combined into a single unit.
[0191] Specifically, in this embodiment of the present application, the decoder 30 is configured to implement the decoding method described in the following embodiments.
[0192] It should be understood that the block partitioning operation may be performed by the prediction processing unit 360 or an independent unit (not shown). The prediction processing unit 360 may be configured to partition the image block 203 into smaller block partitions or sub-blocks, for example, by repeatedly using quad-tree (QT) partitioning, binary-tree (BT) partitioning, triple-tree (TT) partitioning, extended quad-tree (EQT) partitioning, or any combination thereof, perform prediction for each block partition or sub-block, for example (wherein the partitioning mode is determined according to a preset rule or based on a parsed syntax element used to indicate the partitioning mode), and then predict each block partition or sub-block, for example. The mode selection includes selecting a tree structure of the partitioned image block 203 and selecting a prediction mode to be applied to each block partition or sub-block.
[0193] It should be understood that other structural variations of the video decoder 30 may be configured to decode the encoded video bitstream. For example, the video decoder 30 may generate an output video stream without processing by the filter 320. Alternatively, for some image blocks or image frames, the entropy decoding unit 304 of the video decoder 30 does not obtain quantization coefficients by decoding, and thus the inverse quantization unit 310 and the inverse transform processing unit 312 do not need to perform processing. The loop filter 320 is optional. In the case of lossless compression, the inverse quantization unit 310 and the inverse transform processing unit 312 are optional. It should be understood that the inter prediction unit and the intra prediction unit may be selectively enabled according to different application scenarios.
[0194] It should be understood that with respect to the encoder 20 and the decoder 30 of the present application, the processing result of a stage may be output to the next stage after further processing, for example, after the stage of the interpolation filter processing, the motion vector derivation, the loop filter processing, etc., further operations such as clipping or shifting are performed on the processing result of the corresponding stage.
[0195] For example, the motion vectors of the control points of the current image block or the motion vectors of the sub-blocks of the current picture block derived from the motion vectors of the neighboring affine coded blocks can be further processed. This is not limited to the present application. For example, the value of the motion vector is restricted within a certain bit width. If the allowed bit width of the motion vector is bitDepth, the value of the motion vector is in the range of -2^(bitDepth-1) to 2^(bitDepth-1)-1, where the symbol "^" represents exponentiation. When bitDepth is 16, the value range is -32768 to 32767. When bitDepth is 18, the value range is -131072 to 131071. As another example, the value of the motion vector (e.g., the motion vector MV of four 4x4 sub-blocks in an 8x8 image block) is restricted so that the maximum difference between the integer parts of the MV of the four 4x4 sub-blocks does not exceed N pixels, e.g., does not exceed 1 pixel.
[0196] The motion vectors can be constrained to be within a particular bit-width using two methods:
[0197] Method 1: The most significant bit of the motion vector overflow is removed:
number
[0198] vx represents the horizontal component of the motion vector of the image block or a sub-block of the image block, vy represents the vertical component of the motion vector of the image block or a sub-block of the image block, ux and uy represent intermediate values, and bitDepth represents the bit width.
[0199] For example, the value of vx is -32769, where 32767 is derived according to the formula above. Values are stored in the computer in two's complement representation, and the two's complement representation of -32769 is 1, 0111, 1111, 1111, 1111 (17 bits), and the process performed by the computer discards the most significant bits due to overflow. Therefore, the value of vx is 0111, 1111, 1111, 1111, or 32767. This value matches the result derived by the process according to the formula.
[0200] Method 2: Clipping is performed on the motion vectors as shown in the following equation:
number
[0201] vx represents the horizontal component of the motion vector of the image block or a sub-block of the image block; vy represents the vertical component of the motion vector of the image block or a sub-block of the image block; x, y, and z correspond to the three input values of the MV clamping process clip3; and clip3 is defined to indicate clipping the value of z to the range [x, y].
number
[0202] 4 is a schematic structural diagram of a video coding device 400 (e.g., video encoding device 400 or video decoding device 400) according to an embodiment of the present application. The video coding device 400 is suitable for implementing the embodiments described herein. In an embodiment, the video coding device 400 may be a video decoder (e.g., decoder 30 of FIG. 1A) or a video encoder (e.g., encoder 20 of FIG. 1A). In another embodiment, the video coding device 400 may be one or more components of the decoder 30 of FIG. 1A or the encoder 20 of FIG. 1A.
[0203] The video coding device 400 includes an input port 410 and a receiver unit (Rx) 420 configured to receive data, a processor, logic unit, or central processing unit (CPU) 430 configured to process the data, a transmitter unit (Tx) 440 and an output port 450 configured to transmit the data, and a memory 460 configured to store the data. The video coding device 400 may further include optical-to-electrical and electrical-to-optical (EO) components coupled to the input port 410, the receiver unit 420, the transmitter unit 440, and the output port 450 for inputting or outputting optical or electrical signals.
[0204] The processor 430 is implemented by hardware and software. The processor 430 may be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. The processor 430 communicates with the input port 410, the receiver unit 420, the transmitter unit 440, the output port 450, and the memory 460. The processor 430 includes a coding module 470 (e.g., the encoding module 470 or the decoding module 470). The encoding / decoding module 470 implements the embodiments disclosed herein to implement the chroma block prediction method provided in the embodiments of the present application. For example, the encoding / decoding module 470 performs, processes, or provides various coding operations. Thus, the inclusion of the encoding / decoding module 470 provides substantial improvements in the functionality of the video coding device 400 and affects the switching of the video coding device 400 to different states. Alternatively, the encoding / decoding module 470 is implemented as instructions stored in the memory 460 and executed by the processor 430 .
[0205] Memory 460 may include one or more disks, tape drives, and solid state drives, and may be used as overflow data storage, to store programs when such programs are selectively executed, and to store instructions and data read during program execution. Memory 460 may be volatile and / or non-volatile, and may be read only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and / or static random access memory (SRAM).
[0206] FIG. 5 is a simplified block diagram of an apparatus 500 that can be used as either or both of the source device 12 and the destination device 14 of FIG. 1A according to an exemplary embodiment. The apparatus 500 can implement the technology of the present application. In other words, FIG. 5 is a schematic block diagram of an implementation of an encoding or decoding device (coding device 500 for short) according to an embodiment of the present application. The coding device 500 may include a processor 510, a memory 530, and a bus system 550. The processor and the memory are connected via the bus system. The memory is configured to store instructions. The processor is configured to execute the instructions stored in the memory. The memory of the coding device stores program codes, and the processor can call the program codes stored in the memory to perform various video encoding or decoding methods described in the present application, in particular various novel decoding methods. To avoid repetition, details will not be described here.
[0207] In this embodiment of the present application, the processor 510 may be a Central Processing Unit (abbreviated "CPU"), or the processor 510 may be another general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPBGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. The general purpose processor may be a microprocessor, or any conventional processor, etc.
[0208] The memory 530 may include a read only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may alternatively be used as the memory 530. The memory 530 may include code and data 531 that is accessed by the processor 510 via a bus 550. The memory 530 may further include an operating system 533 and application programs 535. The application programs 535 include at least one program that enables the processor 510 to perform the video encoding or decoding methods described herein, in particular the decoding methods described herein. For example, the application programs 535 may include applications 1-N, and further include a video encoding or decoding application (video coding application for short) that is used to perform the video encoding or decoding methods described herein.
[0209] The bus system 550 may include not only a data bus, but also a power bus, a control bus, a status signal bus, etc. However, for clarity of explanation, the various types of buses in the figures are marked as the bus system 550.
[0210] Optionally, coding apparatus 500 may further include one or more output devices, such as a display 570. In one example, display 570 may be a touch-sensitive display that combines a display with touch-sensing elements that operatively sense touch input. Display 570 may be coupled to processor 510 via bus 550.
[0211] The solution in the embodiment of the present application will be described in detail below.
[0212] In a video coding standard, a frame of an image is partitioned into non-overlapping coding tree units (CTUs). The size of the CTUs may be set to 64×64 (the size of the CTUs may alternatively be set to another value, for example, the size of the CTUs is increased to 128×128 or 256×256). A 64×64 CTU is a rectangular pixel matrix with 64 rows, each row including 64 pixels, each pixel including a luma component or / and a chroma component. A CTU stands for coding tree unit. An image includes multiple CTUs, and a CTU generally corresponds to a square image area, including luma and chroma pixels in the image area (or may include only luma pixels, or may include only chroma pixels). A CTU further includes syntax elements. These syntax elements indicate how to divide a CTU into at least one coding unit (CU) and how to decode each coding unit to obtain a reconstructed image.
[0213] CU stands for coding unit. A CU generally corresponds to an A×B rectangular region and includes A×B luma pixels and chroma pixels corresponding to the A×B luma pixels. A represents the width of the rectangle, and B represents the height of the rectangle, where A and B may be the same or different. The values of A and B are generally squares of integers, for example, 256, 128, 64, 32, 16, 8, and 4. A decoding process can be performed on the coding unit to obtain a reconstructed image of the A×B rectangular region. The decoding process generally includes processes such as prediction, dequantization, and inverse transform to generate a predicted image and a residual. The predicted image and the residual are added to obtain a reconstructed image.
[0214] A quad-tree (QT) is a tree structure. A node can be divided into four child nodes. In video coding standards, a quad-tree-based CTU partitioning mode is used. The CTU serves as the root node, and each node corresponds to a square region. Specifically, the square region is divided into four square regions with the same size (the length and width of the square region obtained by the division are half the length and width of the region before division, respectively). As shown in FIG. 6(a), each region corresponds to one node. There are cases where a node cannot be further divided (in which case the region corresponding to the node is a CU), or the node is further divided into lower-level nodes by QT, BT, TT, or EQT.
[0215] A binary tree (BT) is a tree structure. A node can be split into two child nodes. The split into two child nodes can be performed in one of two ways: (1) horizontal binary tree splitting: split the region corresponding to the node into two regions (upper region and lower region) having the same size, as shown in FIG. 6(b), where each region corresponds to one node; or (2) vertical binary tree splitting: split the region corresponding to the node into two regions (left region and right region) having the same size, as shown in FIG. 6(c), where each region corresponds to one node. In the binary tree split coding method, a node in the binary tree structure may not be split (in which case the region corresponding to the node is a CU), or the node may be further split into lower level nodes by BT, TT, or EQT.
[0216] A triple-tree (TT for short) is a tree structure. A node may be split into three child nodes. In existing triple-tree based coding schemes, a node in a triple-tree structure may not be split, or the node may be split into three lower level nodes. The split into three nodes can be performed in one of two ways: (1) horizontal triple-tree splitting: as shown in FIG. 6(d), the region corresponding to the node is split into three regions (top region, middle region, bottom region), each region corresponds to one node, and the heights of the three regions are 1 / 4, 1 / 2, and 1 / 4 of the node height, respectively; or (2) vertical triple-tree splitting: as shown in FIG. 6(e), the region corresponding to the node is split into three regions (left region, middle region, and right region), each region corresponds to one node, and the widths of the three regions are 1 / 4, 1 / 2, and 1 / 4 of the node height, respectively. In the ternary tree partitioning coding method, a node in the ternary tree structure may not be partitioned (in which case the region corresponding to that node is a CU), or the node may be further partitioned into lower level nodes by BT, TT, or EQT.
[0217] An Extended Quad-Tree (EQT) is an "I"-shaped partitioning structure. One node can be partitioned into four child nodes. The partitioning into three nodes can be done in two ways: (1) Horizontal Quad-Tree Partitioning: As shown in Fig. 6(f), the region corresponding to the node is partitioned into three regions (top region, middle region, and bottom region), each of which corresponds to one node. The heights of the top region, left middle region, right middle region, and bottom region are 1 / 4, 1 / 2, 1 / 2, and 1 / 4 of the node height, respectively, and the widths of the left middle region and the right middle region are 1 / 2 and 1 / 2 of the node height, respectively. or (2) vertical quadtree division: as shown in Fig. 6(g), divide the region corresponding to the node into three regions (left region, upper middle region, lower middle region, and right region), each region corresponding to one node, the widths of the left region, middle region, and right region are 1 / 4, 1 / 2, 1 / 2, and 1 / 4 of the node height, respectively, and the widths of the upper middle region and lower middle region are 1 / 2 and 1 / 2 of the node height, respectively. In the extended quadtree coding scheme, the nodes in the extended quadtree structure may not be divided, or the nodes are further divided into lower level nodes by BT, TT, or EQT.
[0218] Video decoding is the process of recovering a video bitstream into a reconstructed image by following certain syntax rules and processing methods.
[0219] Video encoding is the process of compressing a sequence of images into a bitstream.
[0220] Video coding is a general term for video encoding and video decoding. The Chinese translation of video coding is the same as the Chinese translation of video encoding.
[0221] VTM is a new codec reference software developed by JVET.
[0222] In a video coding standard, an image frame is partitioned into non-overlapping coding tree units (CTUs). The size of a CTU may be set to 64×64 (the size of a CTU may alternatively be set to another value, for example, the size of a CTU is increased to 128×128 or 256×256). A 64×64 CTU is a rectangular pixel matrix with 64 rows, each row includes 64 pixels, and each pixel includes a luma component or / and a chroma component.
[0223] A quad-tree (QT for short) based CTU splitting method is used, where the CTU acts as the root node of the quad-tree, and the CTU is recursively split into several leaf nodes in the quad-tree splitting mode. One node corresponds to one image region. If the node is not split, the node is called a leaf node, and the image region corresponding to the node forms one CU. If the node is further split, the image region corresponding to the node is split into four regions with the same size (the length and width of the four regions are half of the length and width of the region before splitting, respectively), and each region corresponds to one node. It is necessary to decide separately whether these nodes are further split. Whether the node is further split is indicated by a split flag split_cu_flag in the bitstream and corresponding to the node. The quad-tree depth (qtDepth) of the root node is 0, and the quad-tree depth of the child node is the quad-tree depth of the parent node plus 1. For simplicity, in the following, the size and shape of a node is the size and shape of the image region corresponding to the node.
[0224] More specifically, in the case of a 64×64 CTU node (quadtree depth is 0), the 64×64 CTU node cannot be split based on the split_cu_flag corresponding to the 64×64 CTU node, and the CTU node forms one 64×64 CU; or the 64×64 CTU node is split into four 32×32 nodes (quadtree depth is 1). Each of the four 32×32 nodes may or may not be further split based on the split_cu_flag corresponding to the node. Further splitting of one 32×32 node generates four 16×16 nodes (quadtree depth is 2). By analogy, the splitting continues until all nodes cannot be further split, and such a CTU is split into a group of CUs. The minimum size of a CU is identified in the SPS, for example, 8×8 represents the minimum size of a CU. In the recursive splitting process described above, if the size of a node is equal to the minimum CU size, then by default the node will not be split any further and there is no need to include the split flag of the node in the bitstream.
[0225] After the parsing finds that the node is a leaf node, the leaf node is a CU, and the coding information corresponding to the CU (e.g., information such as prediction mode and transform coefficients of the CU, including, for example, coding_unit() syntax structure) is further parsed. Then, based on the coding information, a decoding process such as prediction, dequantization, inverse transform, loop filter process, etc. is performed on the CU, and a reconstructed image corresponding to the CU is generated. The CTU can be divided into a group of CUs with appropriate sizes based on the features of the local image using a quadtree structure. For example, smooth regions are divided into larger CUs, and texture-rich regions are divided into smaller CUs.
[0226] The partitioning mode, in which a CTU is partitioned into groups of CUs, corresponds to a coding tree. The particular coding tree to be used for a CTU is usually determined based on the rate distortion optimization (RDO) technique of the encoder. The encoder attempts to use multiple CTU partitioning modes, with each partitioning mode corresponding to a rate distortion cost (RD cost). The encoder compares the RD costs of the various partitioning modes that were attempted to be used, and uses the partitioning mode with the smallest RD cost as the optimal partitioning mode of the CTU for the actual coding of the CTU. The CTU partitioning mode that the encoder attempted to use should comply with the partitioning rules specified by the decoder so that the decoder can correctly identify the CTU partitioning mode.
[0227] In screen content video, images usually contain the same content. For example, in an image containing numbers or graphics, the same numbers or graphics may be found around the current block, as shown in FIG. 7. Therefore, if a referenceable block is found around the current block during the coding of the current block, the reconstructed pixels of the block are directly referenced. In this way, the coding compression rate is greatly improved. Intra Block Copy (IBC) technique is an intra prediction technique used to search for the same block in the current screen content. For example, the syntax element pred_mode_ibc_flag in Table 2 can be used to indicate whether the IBC prediction mode is used for the current coding unit.
[0228] Based on the quad-tree division, a binary-tree (abbreviated as BT) division mode and an extended quad-tree (abbreviated as EQT) division mode can further be used.
[0229] A node is split into two child nodes by a binary tree split. Specifically, there are two binary tree split modes: (1) Horizontal binary tree partitioning: As shown in Fig. 6(b), the region corresponding to a node is partitioned into two regions with the same size (i.e., the width is unchanged and the height is half the height of the region before partitioning), and each region corresponds to one node; and (2) Vertical Binary Tree Splitting: As shown in Figure 6(c), the region corresponding to the node is split into two regions (the left region and the right region) that have the same size (i.e., the height does not change and the width becomes half of the width of the region before the split).
[0230] A node is split into four child nodes by an extended quadtree split. Specifically, there are two extended quadtree split modes: (1) Horizontal quadtree division: As shown in Fig. 6(f), the region corresponding to a node is divided into three regions (top region, middle region, and bottom region), each region corresponding to one node, and the heights of the top region, left middle region, right middle region, and bottom region are 1 / 4, 1 / 2, 1 / 2, and 1 / 4 of the node height, respectively, and the widths of the left middle region and the right middle region are 1 / 2 and 1 / 2 of the node height, respectively; and (2) Vertical quadtree division: As shown in Figure 6(g), the region corresponding to a node is divided into three regions (left region, upper middle region, lower middle region, and right region), each of which corresponds to one node. The widths of the left region, middle region, and right region are 1 / 4, 1 / 2, 1 / 2, and 1 / 4 of the node height, respectively, and the widths of the upper middle region and the lower middle region are 1 / 2 and 1 / 2 of the node height, respectively.
[0231] The QT plus BT / EQT split mode means that a node in the first level coding tree can be split into child nodes only by QT, a leaf node in the first level coding tree is a root node of a second level coding tree, a node in the second level coding tree can be split into child nodes by BT or EQT, and a leaf node in the second level coding tree is a coding unit. It should be noted that when the BT or EQT split mode is used for a leaf node, only the BT or EQT split mode is used for the leaf node, and the QT split mode is not allowed to be used for the leaf node.
[0232] Alternatively, based on the quad-tree division, a binary-tree (abbreviated as BT) division mode and a triple-tree (abbreviated as TT) division mode can be further used.
[0233] A node is split into two child nodes by a binary tree split. Specifically, there are two binary tree split modes: (1) Horizontal binary tree partitioning: As shown in Fig. 6(b), the region corresponding to a node is partitioned into two regions with the same size (i.e., the width is unchanged and the height is half the height of the region before partitioning), and each region corresponds to one node; and (2) Vertical Binary Tree Splitting: As shown in Figure 6(c), the region corresponding to the node is split into two regions (the left region and the right region) that have the same size (i.e., the height does not change and the width becomes half of the width of the region before the split).
[0234] A node is split into two child nodes by a ternary tree split. Specifically, there are two binary tree split modes: (1) Horizontal ternary tree division: As shown in Fig. 6(d), the region corresponding to a node is divided into three regions (upper region, middle region, and lower region), each region corresponds to one node, and the heights of the upper region, middle region, and lower region are 1 / 4, 1 / 2, and 1 / 4 of the height of the node, respectively; and (2) Vertical ternary tree division: As shown in Figure 6(e), the region corresponding to a node is divided into three regions (left region, middle region, and right region), each of which corresponds to one node, and the widths of the left region, middle region, and right region are 1 / 4, 1 / 2, and 1 / 4 of the height of the node, respectively.
[0235] The QT plus BT / TT partitioning mode, briefly called QT-BTT, means that a node in the first-level coding tree can be partitioned into child nodes by QT only, a leaf node in the first-level coding tree is the root node of the second-level coding tree, a node in the second-level coding tree can be partitioned into child nodes by one of four partitioning modes (horizontal binary tree partition, vertical binary tree partition, horizontal ternary tree partition, vertical ternary tree partition), and a leaf node in the second-level coding tree is a coding unit.
[0236] A part of the syntax structure of the CU level may be shown in Table 1. If the current node is not further divided into child nodes, the current node is a coding unit, and the predictive blocks of the coding unit are parsed based on the following syntax structure.
[0237] skip_flag represents a flag of the skip mode. When the value of skip_flag is 1, it indicates that the skip mode is used for the current CU, or when the value of skip_flag is 0, it indicates that the skip mode is not used for the current CU.
[0238] merge_flag represents a flag of the direct mode. When the value of merge_flag is 1, it indicates that the merge mode is used for the current CU, or when the value of merge_flag is 0, it indicates that the merge mode is not used for the current CU.
[0239] cu_pred_mode represents a prediction mode flag of a coding unit. When the value of cu_pred_mode is 1, it indicates that an intra prediction mode is used for the current prediction unit, or when the value of cu_pred_mode is 0, it indicates that a common inter prediction mode is used for the current prediction unit. [Table 1]
[0240] A portion of the CU level syntax structure may alternatively be shown in Table 2. Table 2 is merely an example. The meaning of cu_skip_flag in Table 2 is the same as the meaning of skip_flag in Table 1, and the meaning of pred_mode_flag in Table 2 is the same as the meaning of cu_pred_mode in Table 1.
[0241] cu_skip_flag represents a flag of the skip mode. When the value of cu_skip_flag is 1, it indicates that the skip mode is used for the current CU, or when the value of cu_skip_flag is 0, it indicates that the skip mode is not used for the current CU.
[0242] general_merge_flag represents a flag of the merge mode. When the value of general_merge_flag is 1, it indicates that the merge mode is used for the current CU, or when the value of general_merge_flag is 0, it indicates that the merge mode is not used for the current CU.
[0243] pred_mode_flag represents a prediction mode flag of a coding unit. When the value of pred_mode_flag is 1, it indicates that an intra prediction mode is used for the current coding unit, or when the value of pred_mode_flag is 0, it indicates that a common inter prediction mode is used for the current coding unit. When the value of pred_mode_flag is 1, CuPredMode[x0][y0] is MODE_INTRA. When the value of pred_mode_flag is 0, CuPredMode[x0][y0] is MODE_INTER.
[0244] When the value of pred_mode_ibc_flag is 1, it indicates that IBC prediction mode is used for the current coding unit, or when the value of pred_mode_ibc_flag is 0, it indicates that IBC prediction mode is not used for the current coding unit. When the value of pred_mode_ibc_flag is 1, CuPredMode [x0] [y0] is MODE_IBC.
[0245] CuPredMode [x0][y0] represents the prediction mode of the current coding unit, and (x0, y0) represents the position of the current coding unit in the current image. [Table 2-1] [Table 2-2]
[0246] By dividing a node having a size of 8×M (or M×8) by vertical binary tree division (or horizontal binary tree division), two child nodes each having a size of 4×M (or M×4) are generated. Similarly, by dividing a node having a size of 16×M (or M×16) by vertical extended quad tree division (or horizontal extended quad tree division), four child nodes each having a size of 4×M (or M×4) and one child node each having a size of 8×M (or N×8) are generated. Similarly, by dividing a node having a size of 16×M (or M×16) by vertical ternary tree division (or horizontal ternary tree division), two child nodes each having a size of 4×M (or M×4) and one child node each having a size of 8×M (or N×8) are generated. In the case of the YUV4:2:0 data format, the resolution of the chroma components is 1 / 2 of the resolution of the luma components. In other words, one 4×M node includes one 4×M luma block and two 2×(M / 2) chroma blocks. For hardware decoder, the cost of processing small blocks (especially small blocks with a size of 2×2, 2×4, or 2×8) is relatively high. However, small blocks with a size of 2×2 or 2×4 are generated in such a split mode. This is disadvantageous to the implementation of hardware decoder. It is relatively complicated for hardware decoder to process small blocks. The complexity is particularly reflected in the following three aspects:
[0247] (1) Intra prediction: To increase processing speed, hardware is generally designed to process 16 pixels at a time during intra prediction. However, small blocks with sizes such as 2×2, 2×4, or 4×2 contain less than 16 pixels. As a result, the processing performance of intra prediction is reduced.
[0248] (2) Coefficient coding: In HEVC, transform coefficient coding is performed based on a coefficient group (CG) containing 16 coefficients. However, small blocks with sizes such as 2×2, 2×4, or 4×2 contain 4 or 8 transform coefficients. As a result, to support coefficient coding of these small blocks, a coefficient group containing 4 or 8 coefficients needs to be added. As a result, the implementation complexity increases.
[0249] (3) Inter prediction: Small block inter prediction has a relatively high requirement on data bandwidth, which affects the decoding process speed.
[0250] When a child node includes a chroma block whose side length is 2 in the child node generated by further dividing the node in the division mode, the luma block included in the child node is further divided in the division mode, and the chroma block included in the child node is not further divided. In this way, a chroma block whose side length is 2 is not generated. This reduces the maximum throughput rate of the decoder, which is favorable for the implementation of the decoder. Furthermore, a method for determining a chroma block prediction mode based on a luma block prediction mode is provided, thereby effectively improving the coding efficiency.
[0251] The image prediction method provided in this application may be used in the video encoder 18 or the video decoder 24 shown in Fig. 8. Please note that one or more steps in some of the following embodiments are performed only in the video decoder 24, and are specifically described in the corresponding parts below.
[0252] In the following, the image prediction method in the present application is described in detail using specific embodiments. It should be noted that the following several specific embodiments can be combined with each other, and the same or similar content is not repeatedly described in different embodiments.
[0253] 9 is a schematic flowchart of a first image prediction method according to an embodiment of the present application. Referring to FIG. 9, the image prediction method provided in this embodiment includes the following steps:
[0254] Step 101: Get the partition mode of the current node.
[0255] In this embodiment, the split information of the current node is first analyzed, and the split information is used to indicate whether to split or not to split the current node. If the split information indicates to split the current node, the split mode of the current node is obtained. The split mode of the current node includes at least one of quadtree split, vertical binary tree split, horizontal binary tree split, vertical ternary tree split, and horizontal ternary tree split. Of course, there may be another split mode, which is not particularly limited in this embodiment.
[0256] The partitioning information of the current node may be transmitted in the bitstream. The partitioning information of the current node may be parsed from a corresponding syntax element in the bitstream, and a specific partitioning mode may be determined. Alternatively, the partitioning mode of the current node may be determined according to another preset rule, which is not particularly limited in this embodiment.
[0257] In this embodiment, when the split information related to the current node and obtained by parsing is used to indicate splitting the current node, the split information specifically includes the split mode of the luma block included in the current node and / or the split mode of the chroma block included in the current node. The split mode of the luma block included in the current node may be the same as or different from the split mode of the chroma block included in the current node. This is not particularly limited in this embodiment. For example, the split information is used to indicate that quadtree splitting is used for both the luma block and the chroma block of the current node. Alternatively, the split information is used to indicate that quadtree splitting is used for the luma block of the current node and vertical binary tree splitting is used for the chroma block of the current node.
[0258] Step 102: Determine whether an image block having a preset size is obtained by dividing the current node in the division mode.
[0259] The image block having a preset size may be a luma block having a size less than a threshold, which may be a number of luma samples, such as 128, 64, or 32, or a number of chroma samples, such as 32, 16, or 8. The size of the current node may be equal to or greater than the threshold.
[0260] If it is determined that an image block having a preset size is obtained by dividing the current node in the division mode, then step 103 is executed. If it is determined that an image block having a preset size is not obtained by dividing the current node in the division mode, then step 104 is executed.
[0261] Step 103: Perform intra prediction or inter prediction for all coding blocks covered by the current node.
[0262] It should be noted that the current node in this embodiment can be understood as an image region or image block corresponding to the node to be processed or the node to be split. All coding blocks covered by the current node can be understood as all coding blocks located in the region corresponding to the current node. All coding blocks in this embodiment include luma coding blocks and chroma coding blocks obtained by splitting or not splitting the current node. Alternatively, the coding blocks can be coding units.
[0263] Optionally, the intra prediction performed may be a general intra prediction mode (intra mode) or an intra block copy (IBC) mode.
[0264] Optionally, if the slice type in which the current node is located is of intra type, then instead of inter prediction, intra prediction is performed for all coding blocks covered by the current node.
[0265] In an embodiment, the step of performing intra prediction on all coding blocks covered by the current node comprises: In a split mode, the method may include the steps of: splitting a luma block included in the current node to obtain a luma block obtained by the splitting, and performing intra prediction on the luma block obtained by the splitting; and using a chroma block included in the current node as a chroma coding block, and performing intra prediction on the chroma coding block.
[0266] In other words, when it is determined to perform intra prediction for all coding blocks of the current node, the luma block of the current node is split in a luma block splitting mode to obtain N luma coding tree nodes; and the chroma block of the current node is not split to obtain one chroma coding block (chroma CB for short).
[0267] The N luma coding tree nodes may be restricted to not be further split, or this is not restricted. If the luma coding tree node is further split, the split mode of the luma coding tree node is analyzed for recursive splitting. If the luma coding tree node is not further split, the luma coding tree node corresponds to one luma coding block (luma CB for short). Intra prediction is performed on the luma CB to obtain a luma prediction block corresponding to the luma CB.
[0268] Intra prediction is performed on the chroma CB to obtain a chroma prediction block corresponding to the chroma CB. The chroma prediction block and the chroma CB have the same size.
[0269] In an embodiment, the step of performing inter prediction on all coding blocks covered by the current node includes: In a split mode, the method may include the steps of: splitting a luma block included in the current node to obtain a luma block obtained by the splitting, and performing inter prediction on the luma block obtained by the splitting; and using a chroma block included in the current node as a chroma coding block, and performing inter prediction on the chroma coding block.
[0270] In other words, when it is determined to perform inter prediction for all coding blocks of the current node, the luma block of the current node is split in a luma block splitting mode to obtain N luma coding tree nodes; and the chroma block of the current node is not split to obtain one chroma coding block (chroma CB for short).
[0271] In this embodiment, when it is determined to perform intra prediction for all coding blocks of the current node, or when it is determined to perform inter prediction for all coding blocks of the current node, the luma blocks included in the current node are split in the split mode of the current node, and the chroma blocks of the current node are not further split. According to the above method, small chroma blocks on which intra prediction is performed are not generated, so that the case of performing intra prediction on small chroma blocks is avoided.
[0272] In an embodiment, the step of performing inter prediction on all coding blocks covered by the current node includes: The method may include: in a split mode, splitting a luma block included in the current node to obtain a luma block obtained by the splitting, and performing inter prediction on the luma block obtained by the splitting; and in a split mode, splitting a chroma block included in the current node to obtain a chroma block obtained by the splitting, and performing inter prediction on the chroma block obtained by the splitting.
[0273] In other words, when it is determined to perform inter prediction on all coding blocks of the current node, the luma block of the current node is divided in a luma block division mode to obtain N luma coding tree nodes; and the chroma block of the current node is divided in a chroma block division mode to obtain M chroma coding tree nodes. N and M are positive integers, and N and M may be the same or different. The N luma coding tree nodes and the M chroma coding tree nodes may be restricted not to be further divided, or this is not restricted. When no further division is performed, the N luma coding tree nodes correspond to the N luma CBs of the current node, and the M chroma coding tree nodes correspond to the M chroma CBs of the current node. Inter prediction is performed on the N luma CBs to obtain corresponding luma prediction blocks, and inter prediction is performed on the M chroma CBs to obtain corresponding chroma prediction blocks.
[0274] Optionally, in the case where inter prediction is performed for all coding blocks covered by the current node, the step of performing inter prediction for all coding blocks covered by the current node comprises: The method may include: obtaining a partition sub mode of a child node of the current node, the child node including a luma block and a chroma block; determining whether a luma block having a first preset size is obtained by partitioning the child node of the current node in the partition sub mode; and if it is determined that a luma block having the first preset size is obtained by partitioning the child node of the current node in the partition sub mode, partitioning the child node of the current node in a partition mode other than the partition sub mode to obtain a corresponding coding block and performing inter prediction on the corresponding coding block; or using the child node of the current node as a coding block and performing inter prediction on the child node of the current node.
[0275] In other words, if a luma block having a first preset size (4×4) is generated by splitting a child node of the current node in the split submode, the split submode of the child node is not allowed, or further splitting of the child node is not allowed, or the child node is split in a split mode other than the split submode. For example, if the size of the current node is 8×8 and two 8×4 (or 4×8) nodes are generated by horizontal binary tree splitting (or vertical binary tree splitting), further splitting of the 8×4 (or 4×8) node generates a 4×4 block. In this case, further splitting of the 8×4 (or 4×8) node is not allowed.
[0276] Step 104: Split the current node with the split mode of the current node, where the prediction modes used for all coding blocks covered by the current node are not restricted.
[0277] Specifically, the luma block of the current node is split with the partition mode of the luma block of the current node, and the chroma block of the current node is split with the partition mode of the chroma block of the current node.
[0278] Please note that "the prediction modes used for all coding blocks covered by the current node are not restricted" in step 104 can be understood as follows: all coding blocks covered by the current node may not be predicted with the same prediction mode, i.e., the prediction mode of each coding block is analyzed, and each coding block is predicted with the prediction mode obtained by the analysis.
[0279] Optionally, after step 103 or step 104, the method further includes:
[0280] Step 105: Analyze the prediction blocks and residual information of all coding blocks covered by the current node.
[0281] Step 106: Decode each coding block to obtain a reconstructed signal of the image block corresponding to the current node.
[0282] It should be noted that the above two steps may be used in the video decoder 24 shown in FIG.
[0283] The prediction block includes a prediction mode (indicating an intra prediction mode or a non-intra prediction mode), an intra prediction mode, an inter prediction mode, motion information, etc. The motion information may include information such as a prediction direction (forward, backward, or bidirectional direction), a reference frame index, and a motion vector.
[0284] The residual information includes coded block flags (cbf), transform coefficients, transform type (DCT-2, DCT-7, or DCT-8, etc.) The transform type may be DCT-2 by default.
[0285] When the current node is restricted to perform only intra prediction for all coding blocks covered by the current node, the step of analyzing the prediction block of the luma CB obtained by splitting the current node includes the steps of: setting the value of skip_flag, the value of merge_flag, and the value of cu_pred_mode to 0, 0, and 1 by default (i.e., skip_flag, merge_flag, and cu_pred_mode are all absent in the bitstream), or setting the value of skip_flag and the value of cu_pred_mode to 0 and 1 by default (i.e., skip_flag and cu_pred_mode are both absent in the bitstream); and analyzing information about the intra prediction mode of the luma CB. The step of analyzing the prediction block of the chroma CB obtained by splitting the current node includes the step of analyzing the intra prediction mode of the chroma CB. The intra prediction mode of chroma CB can be analyzed according to the following methods: (1) analyzing syntax elements in the bitstream to obtain the intra prediction mode; or (2) directly setting the intra prediction mode to one intra prediction mode in a set of chroma intra prediction modes, such as a linear model mode, a chroma derived mode (DM) mode, or an IBC mode.
[0286] When restricted to performing only inter prediction for all coding blocks covered by the current node, the step of analyzing the prediction mode of the CU obtained by splitting the current node includes the steps of analyzing skip_flag and / or merge_flag, setting the value of cu_pred_mode to 0 by default, and analyzing inter prediction blocks such as merge index, inter direction, reference index, motion vector predictor index, and motion vector difference.
[0287] skip_flag is a flag of skip mode. When the value of skip_flag is 1, it indicates that the skip mode is used for the current CU, or when the value of skip_flag is 0, it indicates that the skip mode is not used for the current CU. merge_flag is a flag of merge mode. When the value of merge_flag is 1, it indicates that the merge mode is used for the current CU, or when the value of merge_flag is 0, it indicates that the merge mode is not used for the current CU. cu_pred_mode represents a prediction mode flag of a coding unit. When the value of cu_pred_mode is 1, it indicates that intra prediction is performed for the current prediction unit, or when the value of cu_pred_mode is 0, it indicates that common inter prediction (identifying information such as inter direction in the bitstream, reference index, motion vector predictor index, and motion vector difference) is performed for the current prediction unit.
[0288] Optionally, the step of analyzing the prediction block of the luma CB obtained by splitting the current node when the current node is restricted to perform only inter prediction for all coding blocks covered by the current node includes the steps of analyzing skip_flag or / and merge_flag, setting the value cu_pred_mode to 0 by default, and analyzing the inter prediction block such as merge index, inter direction, reference index, motion vector predictor index, and motion vector difference. The motion information of each 4×4 sub-block in the luma CB is derived based on the inter prediction block obtained by the analysis. It is not necessary to analyze the prediction block of the chroma CB obtained by splitting the current node when the current node is restricted to perform only inter prediction for all coding blocks covered by the current node. The chroma CB is split into 2×2 chroma sub-blocks (the split mode can be split mode S). The motion information of each 2×2 chroma sub-block is the motion information of the 4×4 luma region corresponding to each 2×2 chroma sub-block. In the above-mentioned split mode, small chroma blocks on which intra prediction is performed are not generated, and transform blocks having a size less than 16 pixels are not generated, thus overcoming the above-mentioned inter prediction problems and coefficient coding problems.
[0289] Optionally, when the current node is restricted to perform only inter prediction for all coding blocks covered by the current node, there is no need to analyze the prediction block of chroma CB obtained by splitting the current node. The chroma prediction block and the chroma coding block have the same size. The motion information of the chroma CB is the motion information of a preset position of the luma area corresponding to the chroma CB (e.g., the middle, the bottom right corner, or the top left corner of the luma area). In the above split mode, no small chroma block is generated on which intra prediction is performed, no small transform block is generated, and no small chroma block is generated on which inter prediction is performed.
[0290] Please note that in this embodiment, the intra prediction mode is a prediction mode in which a predictor of a coding block is generated using spatial domain reference pixels of the image in which the coding block is located. For example, the intra prediction mode is a direct current mode (DC mode), a planar mode, an angular mode, a template matching mode, or an IBC mode. The inter prediction mode is a prediction mode in which a predictor of a coding block is generated using temporal domain reference pixels of a reference image of the coding block. For example, the inter prediction mode is a skip mode, a merge mode, or an advanced motion vector prediction (AMVP) mode (also called a common inter mode).
[0291] According to the prediction block of the coding block, inter prediction or intra prediction is performed on each coding block to obtain an inter prediction image or intra prediction image of the coding block. Then, according to the residual information of each coding block, dequantization and inverse transform operations are performed on the transform coefficients to obtain a residual image, and the residual image is added to the prediction image of the corresponding region to generate a reconstructed image.
[0292] Optionally, in a possible embodiment, the image block having a preset size includes a luma block having a first preset size, and therefore step 102 includes: The method includes a step of determining whether a luma block having a first preset size is obtained by splitting the current node in a splitting mode of the luma block of the current node, and the luma block having the first preset size is a luma block having a pixel size of 4x4.
[0293] When a luma block having a first preset size is obtained by splitting the luma block of the current node in the split mode of the luma block of the current node, then step 103 includes a step of performing intra prediction for all coding blocks covered by the current node.
[0294] If a luma block having a first preset size is not obtained by splitting the luma block of the current node in the partition mode of the luma block of the current node, then step 104 includes splitting the luma block of the current node in the partition mode of the luma block of the current node and splitting the chroma block of the current node in the partition mode of the chroma block of the current node, and the prediction modes used for all coding blocks covered by the current node are not restricted.
[0295] Optionally, in another possible implementation, the image block having the preset size includes a chroma block having a second preset size, and thus step 102 includes: The method includes a step of determining whether a chroma block having a second preset size is obtained by splitting the current node in a chroma block splitting mode of the current node, and the chroma block having the second preset size is a chroma block having a pixel size of 2x2, 2x4, or 4x2.
[0296] When a chroma block having a second preset size is obtained by splitting the chroma block of the current node in the split mode of the chroma block of the current node, then step 103 includes performing intra prediction or inter prediction on all coding blocks covered by the current node.
[0297] If a chroma block having a second preset size is not obtained by splitting the chroma block of the current node in the split mode of the chroma block of the current node, then step 104 includes splitting the chroma block of the current node in the split mode of the chroma block of the current node and splitting the luma block of the current node in the split mode of the luma block of the current node, and the prediction modes used for all coding blocks covered by the current node are not restricted.
[0298] According to the image prediction method provided in this embodiment, a partition mode of a current node is obtained, and it is determined whether an image block having a preset size is obtained by partitioning the current node in the partition mode of the current node, and the image block includes a luma block or a chroma block. If an image block having a preset size is obtained by partitioning the current node in the partition mode of the current node, intra prediction or inter prediction is performed on all coding blocks covered by the current node. According to the above method, since intra prediction or inter prediction is performed on all coding blocks of the current node, parallel processing can be performed on all coding blocks of the current node. This improves the processing performance of image prediction and increases the coding processing speed.
[0299] Based on the embodiment shown in Figure 9, a process of determining whether a luma block having a first preset size is obtained by splitting a current node with a division mode of the luma block of the current node according to an image prediction method provided in the following embodiment is described in detail. A set of conditions for determining whether a luma block having a first preset size is obtained is specifically disclosed.
[0300] 10 is a schematic flowchart of a second image prediction method according to an embodiment of the present application. As shown in FIG. 10, the image prediction method provided in this embodiment includes the following steps:
[0301] Step 201: Get the partition mode of the current node.
[0302] Specifically, the division information of the current node is analyzed. If the division information indicates that the luma block of the current node is divided, the division mode of the luma block of the current node is further determined. The division mode of the luma block includes at least one of quadtree division, vertical binary tree division, horizontal binary tree division, vertical ternary tree division, and horizontal ternary tree division. Of course, there may be another division mode, which is not particularly limited in this embodiment.
[0303] Step 202: According to the size of the current node and the division mode of the current node, determine whether a luma block having a first preset size is obtained by dividing the current node in the division mode.
[0304] The luma block having the first preset size may be a luma block having a pixel size of 4x4 or 8x8.
[0305] If it is determined that a luma block having a first preset size is obtained by dividing the current node in the division mode, step 203 is executed. If it is determined that a luma block having a first preset size is not obtained by dividing the current node in the division mode, step 204 is executed.
[0306] Specifically, based on the size of the current node and the partition mode of the luma block of the current node, it is determined whether a luma block having a first preset size can be obtained by partitioning the current node in the partition mode of the luma block.
[0307] In this embodiment, the size of the current node can be understood as the pixel size of the image block corresponding to the current node. The size of the current node can be determined based on the width and height of the image block corresponding to the current node, or based on the area of the image block corresponding to the current node, or based on the number of luma pixels of the image block corresponding to the current node. For example, the current node includes 128 luma pixels, which can describe that the area of the current node is 128, and can describe that the product of the width and height of the current node is 128.
[0308] The step of determining, based on the size of the current node and the luma block partition mode of the current node, that a luma block having a first preset size is obtained by partitioning the current node in the luma block partition mode, is performed based on one or more conditions of a first set of the following: (1) The current node contains M1 pixels, and the partitioning mode of the current node is quadtree partitioning, e.g., M1 is 64; (2) The current node contains M2 pixels, and the partitioning mode of the current node is ternary tree partitioning, e.g., M2 is 64; (3) The current node contains M3 pixels, and the partitioning mode of the current node is binary tree partitioning, e.g., M3 is 32; (4) The current node contains 64 luma pixels, and ternary tree partitioning (vertical ternary tree partitioning or horizontal ternary tree partitioning) or quad-tree partitioning is used for the current node; or, the current node contains 32 luma pixels, and binary tree partitioning (vertical binary tree partitioning or horizontal binary tree partitioning) is used for the current node; (5) the width of the current node is four times the second threshold, the height of the current node is equal to the second threshold, and the splitting mode of the current node is vertical ternary tree splitting; (6) the width of the current node is equal to the second threshold, the height of the current node is four times the second threshold, and the splitting mode of the current node is horizontal ternary tree splitting; (7) the width of the current node is twice the second threshold, the height of the current node is equal to the second threshold, and the splitting mode of the current node is vertical binary tree splitting; (8) the height of the current node is twice the second threshold, the width of the current node is equal to the second threshold, and the splitting mode of the current node is horizontal binary tree splitting; or (9) The width and / or height of the current node is twice the second threshold, and the partitioning mode of the current node is quadtree partitioning.
[0309] In the first set above, the width of the current node is the width of the luma block corresponding to the current node, and the height of the current node is the height of the luma block corresponding to the current node. In a particular implementation, for example, the second threshold value may be 4.
[0310] The first set above is applicable to YUV4:2:0 or YUV4:2:2 video data formats.
[0311] Optionally, when the luma block having the first preset size is a luma block having a pixel size of 4×4, determining whether a luma block having the first preset size is obtained by splitting the current node in a split mode based on the size of the current node and the split mode of the current node may be performed based on one of the following conditions: (1) The number of samples in the luma block of the current node is 64, and the partitioning mode is quadtree partitioning; (2) The number of samples in the luma block of the current node is 64 and the partitioning mode is ternary tree partitioning; or (3) The number of samples in the luma block of the current node is 32, and the partitioning mode is binary tree partitioning.
[0312] The number of samples in the luma block of the current node is the number of luma pixels (pixel size) of the image block corresponding to the current node.
[0313] Step 203: Perform intra prediction for all coding blocks covered by the current node.
[0314] In an alternative method of step 203, intra prediction may be performed only on the coding block of the luma block having the first preset size, and the prediction mode of another coding block covered by the current node is not restricted.
[0315] The step of performing intra prediction on all coding blocks covered by the current node includes: In a split mode, the method may include the steps of: splitting a luma block included in the current node to obtain a luma block obtained by the splitting, and performing intra prediction on the luma block obtained by the splitting; and using a chroma block included in the current node as a chroma coding block, and performing intra prediction on the chroma coding block.
[0316] In other words, when it is determined to perform intra prediction for all coding blocks of the current node, the luma block of the current node is split in a luma block splitting mode to obtain N luma coding tree nodes; and the chroma block of the current node is not split to obtain one chroma coding block (chroma CB for short).
[0317] The N luma coding tree nodes may be restricted not to be further split, or this is not restricted. If the luma coding tree node is further split, the split mode of the luma coding tree node is analyzed for recursive splitting. If the luma coding tree node is not further split, the luma coding tree node corresponds to one luma coding block (luma CB for short). Intra prediction is performed on the luma CB to obtain a luma prediction block corresponding to the luma CB.
[0318] Intra prediction is performed on the chroma CB to obtain a chroma prediction block corresponding to the chroma CB. The chroma prediction block and the chroma CB have the same size.
[0319] Optionally, the step of performing intra prediction for all coding blocks covered by the current node comprises: The method may include using a luma block included in the current node as a luma coding block and performing intra prediction on the luma coding block; and using a chroma block included in the current node as a chroma coding block and performing intra prediction on the chroma coding block. In other words, both the luma block and the chroma block of the current node cannot be further divided.
[0320] Step 204: Split the current node with the split mode of the current node, where the prediction modes used for all coding blocks covered by the current node are not restricted.
[0321] Step 204 in this embodiment is the same as step 104 in the embodiment shown in Figure 9. For details, please refer to the above embodiment. The details will not be described again here.
[0322] Optionally, after step 203 or step 204, the method further includes:
[0323] Step 205: Analyze the prediction blocks and residual information of all coding blocks covered by the current node.
[0324] Step 206: Decode each coding block to obtain a reconstructed signal of the image block corresponding to the current node.
[0325] Step 205 and step 206 in this embodiment are the same as step 105 and step 106 in the embodiment shown in Figure 9. For details, please refer to the above embodiment. The details will not be described again here.
[0326] According to the image prediction method provided in this embodiment, a partition mode of a current node is obtained; whether a luma block having a first preset size is obtained by partitioning the current node in the partition mode of the luma block is determined based on the size of the current node and the partition mode of the luma block; and if it is determined that a luma block having a first preset size is obtained by partitioning the current node in the partition mode of the luma block, intra prediction is performed on all coding blocks covered by the current node. According to the above method, intra prediction is performed on all coding blocks of the current node, so that parallel processing can be performed on all coding blocks of the current node. This improves the processing performance of image prediction and increases the coding processing speed.
[0327] Figure 11 is a schematic flow chart of a third image prediction method according to an embodiment of the present application. Based on the embodiment shown in Figure 10, as shown in Figure 11, it should be noted that the solution shown in Figure 11 can be used for a video data format of YUV4:2:0 or YUV4:2:2, or for a video data format of only YUV4:2:0. When it is determined that a luma block having a first preset size is not obtained by splitting the current node in the split mode, step 204 may include the following steps:
[0328] Step 2041: Determine whether a chroma block having a second preset size is obtained by splitting the current node in the split mode.
[0329] If it is determined that a chroma block having a second preset size is obtained by splitting the current node in the split mode, then step 2042 is executed. If it is determined that a chroma block having a second preset size is not obtained by splitting the current node in the split mode, then step 2043 is executed.
[0330] Specifically, step 2041 includes determining, based on the size of the current node and the division mode of the chroma block, whether a chroma block having a second preset size is obtained by dividing the current node with the division mode of the chroma block. The chroma block having the second preset size may be a chroma block having a pixel size of 2×2, 2×4, or 4×2.
[0331] The step of determining, based on the size of the current node and the chroma block splitting mode, that a chroma block having a second preset size is obtained by splitting the current node in the chroma block splitting mode is performed based on one or more conditions of a second set of the following:
[0332] When the video data format is YUV4:2:2, the second set includes the following: (1) The size of the chroma block of at least one child node of the current node is 2×2, 2×4, or 4×2; (2) The width or height of the chroma block of at least one child node of the current node is 2; (3) The current node contains 64 luma pixels, and the splitting mode of the current node is a ternary split or a quaternary split; (4) The current node contains 32 luma pixels, and the splitting mode of the current node is a binary split or a ternary split; or (5) The area of the current node (i.e., the product of the width and the height) is S, where S / 2 < th1, and the splitting mode of the current node is a vertical binary split or a horizontal binary split; or, the area of the current node (i.e., the product of the width and the height) is S, where S / 4 < th1, and the splitting mode of the current node is a vertical ternary split, a horizontal ternary split, or a quaternary split. The threshold th1 is 32.
[0333] When the video data format is YUV4:2:0, the second set includes the following: (1) The size of the chroma block of at least one child node of the current node is 2×2, 2×4, or 4×2; (2) The width or height of the chroma block of at least one child node of the current node is 2; (3) The current node contains 128 luma pixels, and a ternary split is used for the current node; or, the current node contains 64 luma pixels, and a binary split, a quaternary split, or a ternary split is used for the current node; (4) The current node contains 256 luma pixels, and a ternary split or a quaternary split is used for the node; or, the current node contains 128 luma pixels, and a binary split is used for the node; (5) The current node contains N1 luma pixels, and a ternary split is used for the current node, where N1 is 64, 128, or 256; (6) The current node contains N2 luma pixels, and quadtree splitting is used for the current node, where N2 is 64 or 256; (7) The current node contains N3 luma pixels, and binary tree splitting is used for the current node, where N3 is 64, 128, or 256; or (8) The area of the current node (i.e., the product of width and height) is S, where S / 2 < th1, and the splitting mode of the current node is vertical binary tree splitting or horizontal binary tree splitting; or, the area of the current node (i.e., the product of width and height) is S, where S / 4 < th1, and the splitting mode of the current node is vertical ternary tree splitting, horizontal ternary tree splitting, or quadtree splitting. The threshold th1 is 64.
[0334] Optionally, the luma block having the first preset size can be a 4×4 luma block. When the luma block having the first preset size is a 4×4 luma block, the chroma block having the second preset size can be the pixel size of a 2×4 chroma block or the pixel size of a 4×2 chroma block (excluding 2×2 chroma blocks).
[0335] Optionally, the luma block having the first preset size can be a 4×4 luma block. When the luma block having the first preset size is a 4×4 luma block, the chroma block having the second preset size can be a 4×8 luma block or an 8×4 luma block (excluding 4×4 luma blocks).
[0336] Optionally, when the chroma block having the second preset size is a 2×4 chroma block, a 4×2 chroma block, a 4×8 luma block, or an 8×4 luma block, the step of determining whether the chroma block having the second preset size is obtained by splitting the current node in the splitting mode can be executed based on one of the following conditions: (1) The number of samples in the luma block of the current node is 64 and the partitioning mode is binary tree partitioning; or (2) The number of samples in the luma block of the current node is 128, and the partitioning mode is ternary tree partitioning.
[0337] Step 2042: Perform intra prediction or inter prediction on all coding blocks covered by the current node.
[0338] Whether intra prediction or inter prediction is performed for all coding blocks covered by the current node may be determined according to the following method.
[0339] Method 1: Analyze the prediction mode status flag of the current node; if the value of the prediction mode status flag is a first value, perform inter prediction for all coding blocks covered by the current node; or, if the value of the prediction mode status flag is a second value, perform intra prediction for all coding blocks covered by the current node.
[0340] In this method, the prediction mode used for all coding blocks covered by the current node is actually determined based on a flag in a syntax table. Specifically, a prediction mode status flag cons_pred_mode_flag is parsed from the bitstream. When the first value of cons_pred_mode_flag is set to 0, it indicates that inter prediction is performed for all coding blocks obtained by splitting or not splitting the current node, and when the second value of cons_pred_mode_flag is set to 1, it indicates that intra prediction is performed for all coding blocks obtained by splitting or not splitting the current node. Optionally, when the first value of cons_pred_mode_flag is set to 1, it indicates that inter prediction is performed for all coding blocks obtained by splitting or not splitting the current node, and when the second value of cons_pred_mode_flag is set to 0, it indicates that intra prediction is performed for all coding blocks obtained by splitting or not splitting the current node. The meaning of cons_pred_mode_flag may alternatively be indicated by another flag (e.g., mode_cons_flag), which is not limited in this embodiment.
[0341] cons_pred_mode_flag may be a syntax element that needs to be parsed during block division. When parsing the syntax element, the prediction mode flag cu_pred_mode of the coding unit of the coding unit in the coverage area of the current node may not be parsed further, and the value of cu_pred_mode is a default value corresponding to the value of cons_pred_mode_flag.
[0342] The syntax element cons_pred_mode_flag is semantically described as follows: if the value of cons_pred_mode_flag is 0, it indicates that only inter prediction is performed for the coding units covered by the current node; or if the value of cons_pred_mode_flag is 1, it indicates that only intra prediction is performed for the coding blocks covered by the current node.
[0343] If the current node is located in an intra-image region (i.e., the type or slice_type of the image where the current node is located is intra-type or I-type) and the use of IBC mode is allowed, the value of cu_pred_mode is derived as 1, and the value of cu_pred_mode is obtained without the need to parse the bitstream. If the current node is located in an intra-image region but the use of IBC mode is not allowed, the value of cu_pred_mode is derived as 1, the value of cu_skip_flag is 0, and the value of cu_pred_mode is obtained without the need to parse the bitstream.
[0344] If the current node is located in an inter-picture region (i.e., the type or slice_type of the picture in which the current node is located is inter-type or B-type), the value of cu_pred_mode is derived as 0, and the value of cu_pred_mode is obtained without the need to parse the bitstream.
[0345] Because the reference pixel of IBC prediction is the reconstructed pixel of the current image, IBC prediction mode can be regarded as intra prediction mode.Therefore, in this embodiment of the present application, intra prediction can include IBC mode.In other words, in this embodiment of the present application, IBC mode, common intra-intra prediction mode, or IBC mode + common intra-intra prediction mode can be used for intra prediction.Therefore, in this embodiment of the present application, intra prediction can be understood as non-inter prediction.
[0346] Optionally, the slice type in which the current node is located is not of Intra type.
[0347] Method 2: If the prediction mode used for any coding block covered by the current node is inter prediction, then inter prediction is performed for all coding blocks covered by the current node; or, if the prediction mode used for any coding block covered by the current node is intra prediction, then intra prediction is performed for all coding blocks covered by the current node.
[0348] In this way, the prediction mode used for all coding blocks covered by the current node is actually determined based on the prediction mode used for any coding blocks covered by the current node.
[0349] Optionally, the arbitrary coding block is the first coding block of all coding blocks covered by the current node in decoding order. Specifically, the prediction mode of the first coding block B0 in the area of the current node is analyzed, and in this embodiment, the prediction mode of the first coding block B0 is not limited. If the analysis finds that the prediction mode of the coding block B0 is intra prediction, intra prediction is performed for all coding blocks covered by the current node. If the analysis finds that the prediction mode of the coding block B0 is inter prediction, inter prediction is performed for all coding blocks covered by the current node.
[0350] It should be noted that the steps performed according to Method 1 and Method 2 above may be used in the video decoder 24 shown in FIG.
[0351] Step 2043: Split the current node with the split mode of the current node, where the prediction modes used for all coding blocks covered by the current node are not restricted.
[0352] Optionally, after step 203 or step 2042 or step 2043, the method further comprises the following steps:
[0353] Step 205: Analyze the prediction blocks and residual information of all coding blocks covered by the current node.
[0354] Step 206: Decode each coding block to obtain a reconstructed signal of the image block corresponding to the current node.
[0355] Step 205 and step 206 in this embodiment are the same as step 105 and step 106 in the embodiment shown in Figure 9. For details, please refer to the above embodiment. The details will not be described again here.
[0356] According to the image prediction method provided in this embodiment, a partition mode of a current node is obtained; whether a luma block having a first preset size is obtained by partitioning the current node in the partition mode of the luma block is determined based on the size of the current node and the partition mode of the luma block; if it is determined that a luma block having a first preset size is obtained by partitioning the current node in the partition mode of the luma block, it is further determined whether a chroma block having a second preset size is obtained by partitioning the current node in the partition mode of the chroma block; and if it is determined that a chroma block having a second preset size is obtained by partitioning the current node in the partition mode of the chroma block, intra prediction or inter prediction is performed on all coding blocks covered by the current node. According to the above method, since intra prediction or inter prediction is performed on all coding blocks of the current node, parallel processing can be performed on all coding blocks of the current node. This improves the processing performance of image prediction and increases the coding processing speed.
[0357] In the following, the image prediction method provided in the embodiment shown in FIG. 11 will be described with reference to two specific examples.
[0358] The first example is applicable to either YUV4:2:0 or YUV4:2:2 video data formats, or is only applicable to YUV4:2:0 video data format.
[0359] The image prediction method of this example includes the following steps.
[0360] Step 1: Get the partition mode of the current node.
[0361] Step 2: Determine whether the area of the current node and the splitting mode of the current node satisfy at least one of the following Condition A: (1) The area of the current node is equal to 32, and the splitting mode of the current node is vertical binary tree splitting or horizontal binary tree splitting; or (2) The area of the current node is equal to 64, and the splitting mode of the current node is vertical ternary tree splitting, horizontal ternary tree splitting, or quadtree splitting.
[0362] When the area of the current node and the splitting mode of the current node satisfy at least one of the conditions in Condition A, Step 3 is executed.
[0363] Step 3: Limit the execution of intra prediction for all coding blocks covered by the current node.
[0364] Optionally, the value of cons_pred_mode_flag is set to 1.
[0365] When the area of the current node and the splitting mode of the current node do not satisfy Condition A, Step 4 is executed.
[0366] Step 4: Determine whether the area of the current node and the splitting mode of the current node satisfy at least one of the following Condition B: (1) The area S of the current node satisfies S / 2 < th1, and the splitting mode of the current node is vertical binary tree splitting or horizontal binary tree splitting; or (2) The area S of the current node satisfies S / 4 < th1, and the splitting mode of the current node is vertical ternary tree splitting, horizontal ternary tree splitting, or quadtree splitting.
[0367] The threshold th1 is related to the video data format. For example, when the video data format is YUV4:2:0, the threshold th1 is 64; or when the video data format is YUV4:2:2, the threshold th1 is 32.
[0368] If the area of the current node and the partitioning mode of the current node satisfy at least one of the conditions in condition B, step 5 is executed.
[0369] Step 5: Analyze the flag cons_pred_mode_flag from the bitstream, and determine to perform inter prediction or intra prediction for all coding units in the coverage area of the current node based on the value of cons_pred_mode_flag.
[0370] If the area of the current node and the partitioning mode of the current node do not satisfy condition B, step 6 is performed.
[0371] Step 6: Split the current node with the split mode of the current node, where the prediction modes used for all coding blocks covered by the current node are not restricted.
[0372] Optionally, after step 6, the method further comprises the steps of:
[0373] Step 7: Analyze the prediction blocks and residual information of all coding blocks covered by the current node.
[0374] Step 8: Decode each coding block to obtain the reconstructed signal of the image block corresponding to the current node.
[0375] The second example is applicable to a video data format of YUV4:2:0.
[0376] The image prediction method of this example includes the following steps.
[0377] Step 1: Get the partition mode of the current node.
[0378] Step 2: Determine whether the area of the current node and the partitioning mode of the current node satisfy condition C. The area of the current node is equal to 64, and the partitioning mode of the current node is horizontal ternary tree partitioning, vertical ternary tree partitioning, or quadtree partitioning.
[0379] If the area of the current node and the partitioning mode of the current node satisfy condition C, step 3 is executed.
[0380] Step 3: Intra prediction is performed for all coding units within the coverage area of the current node.
[0381] Optionally, the value of cons_pred_mode_flag is set to 1.
[0382] If the area of the current node and the partitioning mode of the current node do not satisfy condition C, step 4 is performed.
[0383] Step 4: Determine whether the area of the current node and the partitioning mode of the current node satisfy at least one of the following conditions D: (1) the area of the current node is equal to 64, and the partitioning mode of the current node is horizontal binary tree partitioning or vertical binary tree partitioning; or (2) The area of the current node is equal to 128, and the partitioning mode of the current node is horizontal ternary tree partitioning or vertical ternary tree partitioning.
[0384] If the area of the current node and the partitioning mode of the current node satisfy at least one of the conditions D, step 5 is executed.
[0385] Step 5: Analyze the flag cons_pred_mode_flag from the bitstream, and determine to perform inter prediction or intra prediction for all coding units in the coverage area of the current node based on the value of cons_pred_mode_flag.
[0386] If the area of the current node and the partitioning mode of the current node do not satisfy condition D, step 6 is performed.
[0387] Step 6: Split the current node with the split mode of the current node, where the prediction modes used for all coding blocks covered by the current node are not restricted.
[0388] Optionally, after step 6, the method further comprises the steps of:
[0389] Step 7: Analyze the prediction blocks and residual information of all coding blocks covered by the current node.
[0390] Step 8: Decode each coding block to obtain the reconstructed signal of the image block corresponding to the current node.
[0391] 12 is a schematic flowchart of a fourth image prediction method according to an embodiment of the present application. As shown in FIG. 12, the image prediction method provided in this embodiment includes the following steps:
[0392] Step 301: Get the partition mode of the current node.
[0393] Specifically, the split information of the current node is analyzed. If the split information indicates to split the chroma block of the current node, the split mode of the chroma block of the current node is further determined. The split mode of the chroma block includes at least one of quadtree split, vertical binary tree split, horizontal binary tree split, vertical ternary tree split, and horizontal ternary tree split. Of course, there may be another split mode, which is not particularly limited in this embodiment.
[0394] Step 302: Based on the size of the current node and the split mode of the current node, determine whether a chroma block having a second preset size is obtained by splitting the current node in the split mode.
[0395] If it is determined that a chroma block having a second preset size is obtained by splitting the current node in the split mode, then step 303 is executed. If it is determined that a chroma block having a second preset size is not obtained by splitting the current node in the split mode, then step 304 is executed.
[0396] Step 302 in this embodiment is the same as step 2041 in the embodiment shown in Figure 11. For details, please refer to the above embodiment. The details will not be described again here.
[0397] Optionally, step 302 may include determining, based on the size of the current node and the partitioning mode of the current node, whether a luma block having a third preset size is obtained by partitioning the current node in the partitioning mode.
[0398] Optionally, the luma block having the third preset size may be a 4x4, 4x8, or 8x4 luma block.
[0399] Optionally, the step of determining whether a chroma block having a second preset size is obtained by splitting the current node in the split mode may be performed based on one of the following conditions: (1) The number of samples in the luma block of the current node is 64, and the partitioning mode is quadtree partitioning; (2) The number of samples in the luma block of the current node is 64, and the partitioning mode is ternary tree partitioning; (3) The number of samples in the luma block of the current node is 32, and the partitioning mode is binary tree partitioning; (4) The number of samples in the luma block of the current node is 64, and the partitioning mode is binary tree partitioning; or (5) The number of samples in the luma block of the current node is 128, and the partitioning mode is ternary tree partitioning.
[0400] Optionally, the chroma block having the second preset size may be a chroma block having a pixel size of 2×4 or 4×2 (except for the chroma block having a pixel size of 2×2). Similarly, the luma block having the third preset size may be a luma block having a pixel size of 4×8 or 8×4 (except for the luma block having a pixel size of 4×4). Correspondingly, the step of determining whether a chroma block having a second preset size is obtained by splitting the current node in the split mode may be performed based on one of the following conditions: (1) The number of samples in the luma block of the current node is 64 and the partitioning mode is binary tree partitioning; or (2) The number of samples in the luma block of the current node is 128, and the partitioning mode is ternary tree partitioning.
[0401] Step 303: Perform intra prediction or inter prediction on all coding blocks covered by the current node.
[0402] Whether to perform intra prediction or inter prediction for all coding blocks covered by the current node can be determined based on step 2042 of the embodiment shown in Figure 11. For details, please refer to the above embodiment. The details will not be described again here.
[0403] Step 304: Split the current node with the split mode of the current node, where the prediction modes used for all coding blocks covered by the current node are not restricted.
[0404] Step 304 in this embodiment is the same as step 104 in the embodiment shown in Figure 9. For details, please refer to the above embodiment. The details will not be described again here.
[0405] Optionally, after step 303 or step 304, the method further comprises the following steps:
[0406] Step 305: Analyze the prediction blocks and residual information of all coding blocks covered by the current node.
[0407] Step 306: Decode each coding block to obtain a reconstructed signal of the image block corresponding to the current node.
[0408] Step 305 and step 306 in this embodiment are the same as step 105 and step 106 in the embodiment shown in Figure 9. For details, please refer to the above embodiment. The details will not be described again here.
[0409] According to the image prediction method provided in this embodiment, a partition mode of a current node is obtained, and whether a chroma block having a second preset size is obtained by partitioning the current node in the partition mode is determined based on the size of the current node and the partition mode of the current node; and when it is determined that a chroma block having a second preset size is obtained by partitioning the current node in the partition mode, intra prediction or inter prediction is performed on all coding blocks covered by the current node. According to the above method, since intra prediction or inter prediction is performed on all coding blocks of the current node, parallel processing can be performed on all coding blocks of the current node. This improves the processing performance of image prediction and increases the coding processing speed.
[0410] Figure 13 is a schematic flowchart of a fifth image prediction method according to an embodiment of the present application. Based on the embodiment shown in Figure 12, if it is determined that a chroma block having a second preset size is obtained by splitting the current node in a split mode as shown in Figure 13, step 304 may include the following steps:
[0411] Step 3041: Determine whether a luma block having a first preset size is obtained by splitting the current node in a split mode.
[0412] If it is determined that a luma block having a first preset size is obtained by splitting the current node in the split mode, then step 3042 is executed. If it is determined that a luma block having a first preset size is not obtained by splitting the current node in the split mode, then step 3043 is executed.
[0413] Specifically, according to the size of the current node and the division mode of the luma block, it is determined whether a luma block having a first preset size is obtained by dividing the current node in the division mode of the luma block. The luma block having the first preset size is a luma block having a pixel size of 4×4. The specific determination process is the same as the process in step 202 of the embodiment shown in Fig. 10, and can refer to the above embodiment. The details are not described again here.
[0414] Step 3042: Perform intra prediction for all coding blocks covered by the current node.
[0415] Step 3043: Split the current node with the split mode of the current node, where the prediction modes used for all coding blocks covered by the current node are not restricted.
[0416] Optionally, after step 303 or step 3042 or step 3043, the method further comprises the following steps:
[0417] Step 305: Analyze the prediction blocks and residual information of all coding blocks covered by the current node.
[0418] Step 306: Decode each coding block to obtain a reconstructed signal of the image block corresponding to the current node.
[0419] Step 305 and step 306 in this embodiment are the same as step 105 and step 106 in the embodiment shown in Figure 9. For details, please refer to the above embodiment. The details will not be described again here.
[0420] According to the image prediction method provided in this embodiment, a split mode of a current node is obtained; whether a chroma block having a second preset size is obtained by splitting the current node in the split mode is determined based on the size of the current node and the split mode of the current node; if it is determined that a chroma block having a second preset size is not obtained by splitting the current node in the split mode, it is further determined whether a luma block having a first preset size is obtained by splitting the current node in the split mode; and if it is determined that a luma block having a first preset size is obtained by splitting the current node in the split mode, intra prediction is performed on all coding blocks covered by the current node. According to the above method, intra prediction or inter prediction is performed on all coding blocks of the current node, so that parallel processing can be performed on all coding blocks of the current node. This improves the processing performance of image prediction and increases the coding processing speed.
[0421] Figure 14 is a schematic flow chart of a sixth image prediction method according to an embodiment of the present application. The image prediction method provided in this embodiment is used in the video encoder 18 shown in Figure 8. As shown in Figure 14, the method of this embodiment includes the following steps:
[0422] Step 401: Get the partition mode of the current node.
[0423] The type of the slice in which the current node is located is type B or type P. It should be understood that if the type of the slice in which the current node is located is type I, by default, intra prediction needs to be performed on all coding blocks covered by the current node.
[0424] The decoder side can obtain the partition mode of the current node by analyzing the bitstream or based on the obtained parameters. The encoder side generally first determines the partition modes allowed for the current node, and then determines the optimal partition mode as the partition mode of the current node according to a rate-distortion optimization (RDO) method. This step belongs to the prior art, and the details are not described here.
[0425] Step 402: Determine whether the size of the current node and the split mode of the current node satisfy one of the first preset conditions.
[0426] The first preset condition may include: (1) The number of samples in the luma block of the current node is 64, and the partitioning mode of the current node is quadtree partitioning; (2) The number of samples in the luma block of the current node is 64, and the partitioning mode of the current node is ternary tree partitioning; or (3) The number of samples in the luma block of the current node is 32, and the partitioning mode of the current node is binary tree partitioning.
[0427] The number of samples of the luma block of the current node is the number of luma pixels (pixel size) of the image block corresponding to the current node. The number of samples may be obtained based on the product of the width and height of the current node.
[0428] In another embodiment, the first preset condition further comprises the following condition (4): (4) A luma block with a preset size is obtained by splitting the current node in a split mode, where the preset size is 4×4 or 8×8.
[0429] It should be noted that a luma block having a first preset size (4x4 or 8x8) and a chroma block having a second preset size (2x4 or 4x2) can be obtained if one of the first preset conditions is met.
[0430] In this step, if the size of the current node and the partition mode of the current node satisfy one of the first preset conditions, step 403 is executed. If the size of the current node and the partition mode of the current node do not satisfy any of the first preset conditions, step 404 is executed.
[0431] Step 403: Perform intra prediction for all coding blocks covered by the current node.
[0432] Optionally, when it is determined to perform intra prediction for all coding blocks covered by the current node, there is no need to write the value of mode_constraint_flag into the bitstream, and the value of mode_constraint_flag may be set to 1. Correspondingly, the decoder side may also derive the value of mode_constraint_flag to be 1 following the same method.
[0433] Step 404: Determine whether the size of the current node and the split mode of the current node satisfy one of the second preset conditions.
[0434] The second preset condition includes: (1) The number of samples in the luma block of the current node is 64, and the partitioning mode of the current node is vertical binary tree partitioning or horizontal binary tree partitioning; or (2) The number of samples of the luma block of the current node is 128, and the partitioning mode of the current node is vertical ternary tree partitioning or horizontal ternary tree partitioning.
[0435] In another embodiment, the second preset condition further comprises the following condition (3): (3) A chroma block with a preset size is obtained by splitting the current node in a split mode, where the preset size is 2×4 or 4×2.
[0436] In this step, if the size of the current node and the partitioning mode of the current node satisfy one of the second preset conditions, step 405 is executed. If the size of the current node and the partitioning mode of the current node do not satisfy any of the second preset conditions, step 406 is executed.
[0437] Step 405: Perform intra prediction or inter prediction for all coding blocks covered by the current node.
[0438] In this embodiment, the prediction mode used for all coding blocks of the current node can be determined in several ways:
[0439] In one embodiment, if the type of the image or slice where the current node is located is I type, the encoder or encoder side determines that only intra prediction (non-inter prediction) is applicable to all coding blocks of the current node. Optionally, the encoder or encoder side does not need to write mode_constraint_flag into the bitstream and may set the value of mode_constraint_flag to 1.
[0440] If the type of the image or slice where the current node is located is not I-type, the encoder or encoder side determines the value of mode_constraint_flag according to a rate-distortion optimization (RDO) method or another method.
[0441] The RDO method is specifically as follows: The encoder calculates the rate-distortion cost (RD cost) generated when performing inter prediction and intra prediction for all coding blocks covered by the current node separately, compares the rate-distortion cost values generated for the two prediction modes, and determines the prediction mode with the smaller rate-distortion cost as the final prediction mode. If the prediction mode with the smaller rate-distortion cost is intra prediction, the encoder sets the value of mode_constraint_flag to 1. If the prediction mode with the smaller rate-distortion cost is inter prediction, the encoder sets the value of mode_constraint_flag to 0, and writes the value of mode_constraint_flag into the bitstream.
[0442] For example, the encoder first calculates the RD cost generated when performing inter prediction on all coding blocks covered by the current node, and then calculates the RD cost generated when performing intra prediction. If there is no residual when performing inter prediction on all coding blocks covered by the current node (e.g., skip mode is used), the encoder determines that inter prediction is performed on all coding blocks covered by the current node, sets the value of mode_constraint_flag to 0, and does not need to calculate the RD cost generated when performing intra prediction. Alternatively, the encoder can first calculate the RD cost generated when performing intra prediction on all coding blocks covered by the current node, and then calculate the RD cost generated when performing inter prediction, and determine the prediction mode with the smaller RD cost as the final prediction mode.
[0443] Note that if the current node is located in an intra image region (i.e., the type or slice_type of the image where the current node is located is intra type or I type) and the use of IBC mode is allowed, the value of pred_mode_flag defaults to 1. If the current node is located in an intra image region but the use of IBC mode is not allowed, the value of pred_mode_flag defaults to 1 and the value of cu_skip_flag defaults to 0 (indicating that skip mode is not used for the current block). If the current node is located in an intra image region (i.e., the type or slice_type of the image where the current node is located is intra type or I type), the value of mode_constraint_flag defaults to 1.
[0444] In one embodiment, the decoder side or decoder may analyze a prediction mode status flag (mode_constraint_flag) of the current node; if the value of the prediction mode status flag is a first value, perform inter prediction for all coding blocks belonging to the current node; or, if the value of the prediction mode status flag is a second value, perform intra prediction for all coding blocks belonging to the current node.
[0445] Step 406: Split the current node with the split mode of the current node, where the prediction modes used for all coding blocks covered by the current node are not restricted.
[0446] Optionally, after step 403, step 405 or step 406, the method may further include the following steps:
[0447] Step 407: Determine whether the luma block and the chroma block of the current node are further divided according to the prediction mode of the current node.
[0448] When it is determined that only intra prediction is performed for all coding blocks of the current node, the luma block included in the current node is divided in a division mode to obtain a luma block obtained by division, and intra prediction is performed for the luma block obtained by division; and the chroma block included in the current node is used as a chroma coding block, and intra prediction is performed for the chroma coding block. In other words, when it is determined that intra prediction is performed for all coding blocks of the current node, the luma block of the current node is divided in a division mode of the luma block to obtain N luma coding tree nodes; and the chroma block of the current node is not divided, and one chroma coding block (chroma CB for short) is obtained. The N luma coding tree nodes may be restricted not to be further divided, or this is not restricted. When the luma coding tree node is further divided, the division mode of the luma coding tree node is analyzed for recursive division. When the luma coding tree node is not further divided, the luma coding tree node corresponds to one luma coding block (luma CB for short). Intra prediction is performed on the luma CB to obtain a luma prediction block corresponding to the luma CB. Intra prediction is performed on the chroma CB to obtain a chroma prediction block corresponding to the chroma CB. The chroma prediction block and the chroma CB have the same size.
[0449] When it is determined to perform only intra prediction for all coding blocks of the current node, in another embodiment, the luma block and the chroma block included in the current node are split in a split mode to obtain a node obtained by splitting. When the coding tree node is not further split, the coding tree node corresponds to one coding unit and includes a luma coding unit and a chroma coding unit. Intra prediction is performed on the luma coding unit and the chroma coding unit, where the luma coding unit and the chroma coding unit are obtained by splitting.
[0450] When it is determined to perform only inter prediction for all coding blocks of the current node, in one embodiment, the luma block included in the current node is divided in a partition mode to obtain a luma block obtained by the partition, and inter prediction is performed on the luma block obtained by the partition; and the chroma block included in the current node is divided in a partition mode to obtain a chroma block obtained by the partition, and inter prediction is performed on the chroma block obtained by the partition. In other words, when it is determined to perform inter prediction for all coding blocks of the current node, the luma block of the current node is divided in a luma block partition mode to obtain N luma coding tree nodes; and the chroma block of the current node is divided in a chroma block partition mode to obtain M chroma coding tree nodes. N and M are positive integers, and N and M may be the same or different. The N luma coding tree nodes and the M chroma coding tree nodes may be restricted not to be further divided, or this is not restricted. When no further splitting is performed, the N luma coding tree nodes correspond to the N luma CBs of the current node, and the M chroma coding tree nodes correspond to the M chroma CBs of the current node. Inter prediction is performed on the N luma CBs to obtain corresponding luma prediction blocks, and inter prediction is performed on the M chroma CBs to obtain corresponding chroma prediction blocks.
[0451] When it is determined to perform only inter prediction for all coding blocks of the current node, in one embodiment, the luma block included in the current node is divided in a partition mode to obtain a luma block obtained by the partition, and inter prediction is performed on the luma block obtained by the partition; and the chroma block included in the current node is divided in a partition mode to obtain a chroma block obtained by the partition, and inter prediction is performed on the chroma block obtained by the partition. In other words, when it is determined to perform inter prediction for all coding blocks of the current node, the luma block of the current node is divided in a luma block partition mode to obtain N luma coding tree nodes; and the chroma block of the current node is divided in a chroma block partition mode to obtain M chroma coding tree nodes. N and M are positive integers, and N and M may be the same or different. The N luma coding tree nodes and the M chroma coding tree nodes may be restricted not to be further divided, or this is not restricted. When no further splitting is performed, the N luma coding tree nodes correspond to the N luma CBs of the current node, and the M chroma coding tree nodes correspond to the M chroma CBs of the current node. Inter prediction is performed on the N luma CBs to obtain corresponding luma prediction blocks, and inter prediction is performed on the M chroma CBs to obtain corresponding chroma prediction blocks.
[0452] In particular, when only inter prediction is performed for all coding blocks of the current node, a child node is obtained by dividing the current node in a division mode, and the child node needs to be further divided, and a luma block having a preset size is obtained by dividing in the division mode of the child node, where, for example, the preset size is 4×4 (i.e., both the width and the height are 4), and the division mode of the child node is not allowed, or further division of the child node is not allowed. Specifically, when only inter prediction is restricted to be performed for a node, and the number of luma samples of the node is 32 (i.e., the product of the width and height of the node is 32), binary tree division (including horizontal binary tree division or vertical binary tree division) is not allowed for the node. When only inter prediction is restricted to be performed for a node, and the number of luma samples of the node is 64 (i.e., the product of the width and height of the node is 64), ternary tree division (including horizontal ternary tree division or vertical ternary tree division) is not allowed for the node. Such a determination method is applicable to both YUV4:2:0 and YUV4:2:2 video data formats.
[0453] For example, if the size of the current node is 8x8 and two 8x4 (or 4x8) nodes are generated by horizontal binary tree splitting (or vertical binary tree splitting), further splitting the 8x4 (or 4x8) node will generate a 4x4 block. In this case, vertical binary tree splitting (or horizontal binary tree splitting) is not allowed for the 8x4 (or 4x8) node, or further splitting of the 8x4 (or 4x8) node is not allowed. As another example, if the quantity of samples of the luma block of the current node is 128 and the splitting mode is horizontal ternary tree splitting or vertical ternary tree splitting, the quantity of samples of the luma block may be determined to be 64. However, if the horizontal ternary tree splitting or vertical ternary tree splitting of the child node is used for a luma node with a quantity of samples of 64, a 4x4 luma block is obtained. Therefore, when the restriction is that only inter prediction is performed, no horizontal or vertical ternary tree partitioning is permitted for nodes of quantity 64 samples, or any further partitioning of nodes of quantity 64 samples is permitted.
[0454] Step 408: Predict the coding block of the CU obtained by splitting the current node to obtain a predictor of the coding block.
[0455] If only intra prediction is performed for all coding blocks of the current node, the encoder or encoder side determines an optimal intra prediction mode to be used for the current coding block according to a rate-distortion optimization (RDO) method or another method; then predicts the current coding block using the corresponding intra prediction mode to obtain a predictor for the current block.
[0456] If only inter prediction is performed for all coding blocks of the current node, the encoder or encoder side determines an optimal inter prediction mode to be used for the current coding block according to a rate-distortion optimization method or another method; then predicts the current coding block using the corresponding inter prediction mode to obtain a predictor for the current block.
[0457] Furthermore, the encoder assigns values to CU depth-related syntax elements, and writes the values of each syntax element into the bitstream according to the criteria of the CU-level syntax definition. For example, when only intra prediction is performed for all coding blocks of the current node, the value of pred_mode_flag is set to 1, and pred_mode_flag is not written into the bitstream. That is, pred_mode_flag does not exist in the bitstream. When it is determined that only intra prediction is performed for all coding blocks of the current node and IBC mode is not used, the value of cu_skip_flag (or skip_flag) is 0, and cu_skip_flag is not written into the bitstream. Otherwise, after determining the value of cu_skip_flag, cu_skip_flag needs to be written into the bitstream, and the cu_skip_flag is transmitted to the decoder side.
[0458] When only inter prediction is performed for all coding blocks of the current node, the value of pred_mode_flag is set to 0, and pred_mode_flag is not written to the bitstream. That is, pred_mode_flag is not present in the bitstream. When the value of pred_mode_ibc_flag is set to 0, pred_mode_ibc_flag is not written to the bitstream. That is, pred_mode_ibc_flag is not present in the bitstream.
[0459] When only intra prediction is performed for all coding blocks of the current node, the decoder or the decoder side determines an optimal intra prediction mode to be used for the current coding block by analyzing the bitstream or based on the obtained parameters; and then predicts the current coding block using the corresponding intra prediction mode to obtain a predictor for the current block.
[0460] When only inter prediction is performed for all coding blocks of the current node, the decoder or the decoder side determines the optimal inter prediction mode to be used for the current coding block by analyzing the bitstream or based on the obtained parameters; and then predicts the current coding block using the corresponding inter prediction mode to obtain a predictor for the current block.
[0461] Step 409: Obtain the reconstructed signal of the image block at the current node.
[0462] After obtaining the prediction information by intra prediction or inter prediction, the encoder or encoder side obtains residual information by subtracting the corresponding prediction information (or predictor) from the pixel value of the pixel in the current coding block, transforms the residual information using a method such as Discrete Cosine Transform (DCT), obtains a bitstream by quantization and entropy coding, and transmits the residual information to the decoder side. After adding the prediction signal and the reconstructed residual signal, the encoder side needs to further perform a filtering operation to obtain a reconstructed signal, and uses the reconstructed signal as a reference signal for subsequent encoding. In particular, when the skip mode is used for the coding block, there is no residual information, there is no need to perform a transformation at the encoder side, and the predictor becomes the final reconstruction value.
[0463] After adding the prediction signal and the reconstructed residual signal, the decoder or decoder side needs to further perform a filtering operation to obtain a reconstructed signal. Furthermore, the decoder side uses the obtained reconstructed signal as a reference signal for subsequent encoding. In particular, when the skip mode is used for the coding block, there is no residual information, the decoder or decoder side does not need to perform a transform, and the predictor becomes the final reconstructed value.
[0464] For example, according to the prediction block of the CU, an inter prediction process or an intra prediction process is performed on each CU to obtain an inter prediction image or an intra prediction image of the CU. Then, according to the residual information of each CU, a dequantization and inverse transform process is performed on the transform coefficient to obtain a residual image, and the residual image is added to the prediction image of the corresponding region to generate a reconstructed image.
[0465] In this embodiment, the image prediction method is described from the viewpoint of the video decoder side. The video decoder determines whether to perform intra prediction or inter prediction for all coding blocks of the current node according to the size of the current node and the partition mode of the current node, so that it can perform parallel processing for all coding blocks of the current node. This improves the processing performance of image prediction and increases the decoding processing speed.
[0466] The image prediction method provided in one embodiment is used for the video encoder 18 and / or the video decoder 24 shown in Figure 8. This embodiment includes the following steps.
[0467] Step 501: Get the partition mode of the current node.
[0468] Step 501 in this embodiment is the same as step 101 in the embodiment shown in FIG. 9, and the details will not be described again here.
[0469] Step 502: Derive the value of the variable modeTypeCondition according to the following method.
[0470] The value of modeTypeCondition is a first value, e.g., 0, if one or more of the following first preset conditions are true: (1) The type of the image or slice where the current node is located is I type (slice_type == I), and the value of qtbtt_dual_tree_intra_flag is 1; (2) The prediction mode type of the current node is only intra prediction or inter prediction, i.e., is restricted to only perform inter prediction or intra prediction (non-inter prediction); or (3) The chroma format is a monochrome format or a 4:4:4 format. For example, the value of chroma_format_idc is 0 or 3.
[0471] In another embodiment, the first preset condition further includes the following condition (4): (4) The chroma format is monochrome format, 4:4:4 format, or 4:2:2 format, for example, the value of chroma_format_idc is 0, 3, or 2.
[0472] Otherwise, the value of modeTypeCondition is a second value, e.g., 1, if one or more of the following second preset conditions are true: (1) The product of the width and height of the luma block of the current node is 64, and the partitioning mode of the current node is quadtree partitioning; (2) The product of the width and height of the luma block of the current node is 64, and the partitioning mode of the current node is horizontal ternary tree partitioning or vertical ternary tree partitioning; or (3) The product of the width and height of the luma block of the current node is 32, and the partitioning mode of the current node is horizontal binary tree partitioning or vertical binary tree partitioning.
[0473] Otherwise, if one or more of the following predefined third conditions are true and the chroma format is 4:2:0 (the value of chroma_format_idc is 1), the value of modeTypeCondition is derived according to the following formula: 1 + (slice_type != I ? 1 : 0): (1) The product of the width and height of the luma block of the current node is 64, and the partitioning mode of the current node is horizontal binary tree partitioning or vertical binary tree partitioning; or (2) The product of the width and height of the luma block of the current node is 128, and the partitioning mode of the current node is horizontal ternary tree partitioning or vertical ternary tree partitioning.
[0474] Note that Table 3 shows the correspondence between chroma formats and chroma_format_idc. [Table 3]
[0475] In monochrome sampling, there is no chroma component format, only a set of luma components.
[0476] In 4:2:0 sampling, the width of the two chroma components is half the width of the corresponding luma component, and the height of the chroma components is half the height of the luma components.
[0477] In 4:2:2 sampling, the height of the two chroma components is the same as the height of the corresponding luma component, and the width of the chroma components is half the width of the corresponding luma component.
[0478] For 4:4:4 sampling, the height and width of the two chroma components depend on the value of separate_colour_plane_flag: if the value of separate_colour_plane_flag is equal to 0, the width and height of the two chroma components are the same as the width and height of the corresponding luma component, respectively. Otherwise (value of separate_colour_plane_flag is equal to 1), the three components are coded separately as monochrome sample images.
[0479] separate_colour_plane_flag equal to 1 specifies that the three colour components of the 4:4:4 chroma format are coded separately. separate_colour_plane_flag equal to 0 specifies that the colour components are not coded separately.
[0480] qtbtt_dual_tree_intra_flag equal to 1 specifies that for an I-slice, each CTU is divided into coding units with 64x64 luma samples using an implicit quadtree partitioning, and these coding units are the root of two separate coding_tree syntax structures for luma and chroma. qtbtt_dual_tree_intra_flag equal to 0 specifies that separate coding_tree syntax structures are not used for I-slice. When qtbtt_dual_tree_intra_flag is not present, it is inferred to be equal to 0.
[0481] Step 503: Determine the prediction mode types of all coding units of the current node according to the value of modeTypeCondition.
[0482] Specifically, when the value of modeTypeCondition is 1, intra prediction (MODE_INTRA) is restricted to be performed for all coding units of the current node. When the value of modeTypeCondition is 2, the value of the syntax element mode_constraint_flag is parsed from the bitstream. When the value of mode_constraint_flag is 0, inter prediction (MODE_INTER) is performed for all coding units of the current node. When the value mode_constraint_flag is 1, intra prediction (non-inter prediction / MODE_INTRA) is performed for all coding units of the current node.
[0483] Otherwise, the prediction mode type of all coding units of the current node is not restricted and is the same as the prediction mode type of the current node.
[0484] Step 504: Determine whether the chroma block and the luma block corresponding to the current node are further divided to obtain chroma coding units and luma coding units.
[0485] (Step 504 is the same as step 407).
[0486] When it is determined to perform only intra prediction for all coding blocks of the current node, the luma block included in the current node is divided in a division mode to obtain a luma block obtained by division, and intra prediction is performed on the luma block obtained by division; and the chroma block included in the current node is used as a chroma coding block, and intra prediction is performed on the chroma coding block. In other words, when it is determined to perform intra prediction for all coding blocks of the current node, the luma block of the current node is divided in a division mode of the luma block to obtain N luma coding tree nodes; and the chroma block of the current node is not divided, and one chroma coding block (chroma CB for short) is obtained. The N luma coding tree nodes may be restricted not to be further divided, or this is not restricted. When the luma coding tree node is further divided, the division mode of the luma coding tree node is analyzed for recursive division. When the luma coding tree node is not further divided, the luma coding tree node corresponds to one luma coding block (luma CB for short). Intra prediction is performed on the luma CB to obtain a luma prediction block corresponding to the luma CB. Intra prediction is performed on the chroma CB to obtain a chroma prediction block corresponding to the chroma CB. The chroma prediction block and the chroma CB have the same size.
[0487] When it is determined to perform only intra prediction for all coding blocks of the current node, in another embodiment, the luma block and the chroma block included in the current node are split in a split mode to obtain a node obtained by splitting. When the coding tree node is not further split, the coding tree node corresponds to one coding unit and includes a luma coding unit and a chroma coding unit. Intra prediction is performed on the luma coding unit and the chroma coding unit, and the luma coding unit and the chroma coding unit are obtained by splitting.
[0488] When it is determined to perform only inter prediction for all coding blocks of the current node, in one embodiment, the luma block included in the current node is divided in a partition mode to obtain a luma block obtained by the partition, and inter prediction is performed on the luma block obtained by the partition; and the chroma block included in the current node is divided in a partition mode to obtain a chroma block obtained by the partition, and inter prediction is performed on the chroma block obtained by the partition. In other words, when it is determined to perform inter prediction for all coding blocks of the current node, the luma block of the current node is divided in a luma block partition mode to obtain N luma coding tree nodes; and the chroma block of the current node is divided in a chroma block partition mode to obtain M chroma coding tree nodes. N and M are positive integers, and N and M may be the same or different. The N luma coding tree nodes and the M chroma coding tree nodes may be restricted not to be further divided, or this is not restricted. When no further splitting is performed, the N luma coding tree nodes correspond to the N luma CBs of the current node, and the M chroma coding tree nodes correspond to the M chroma CBs of the current node. Inter prediction is performed on the N luma CBs to obtain corresponding luma prediction blocks, and inter prediction is performed on the M chroma CBs to obtain corresponding chroma prediction blocks.
[0489] When it is determined to perform only inter prediction for all coding blocks of the current node, in one embodiment, the luma block included in the current node is divided in a partition mode to obtain a luma block obtained by the partition, and inter prediction is performed on the luma block obtained by the partition; and the chroma block included in the current node is divided in a partition mode to obtain a chroma block obtained by the partition, and inter prediction is performed on the chroma block obtained by the partition. In other words, when it is determined to perform inter prediction for all coding blocks of the current node, the luma block of the current node is divided in a luma block partition mode to obtain N luma coding tree nodes; and the chroma block of the current node is divided in a chroma block partition mode to obtain M chroma coding tree nodes. N and M are positive integers, and N and M may be the same or different. The N luma coding tree nodes and the M chroma coding tree nodes may be restricted not to be further divided, or this is not restricted. When no further splitting is performed, the N luma coding tree nodes correspond to the N luma CBs of the current node, and the M chroma coding tree nodes correspond to the M chroma CBs of the current node. Inter prediction is performed on the N luma CBs to obtain corresponding luma prediction blocks, and inter prediction is performed on the M chroma CBs to obtain corresponding chroma prediction blocks.
[0490] In particular, when only inter prediction is performed for all coding blocks of the current node, a child node is obtained by dividing the current node in a division mode, and the child node needs to be further divided, and a luma block having a preset size is obtained by dividing in the division mode of the child node, for example, the preset size is 4×4 (i.e., both the width and the height are 4), and the division mode of the child node is not allowed, or further division of the child node is not allowed. Specifically, when only inter prediction is restricted to be performed for a node, and the number of luma samples of the node is 32 (i.e., the product of the width and height of the node is 32), binary tree division (including horizontal binary tree division or vertical binary tree division) is not allowed for the node. When only inter prediction is restricted to be performed for a node, and the number of luma samples of the node is 64 (i.e., the product of the width and height of the node is 64), ternary tree division (including horizontal ternary tree division or vertical ternary tree division) is not allowed for the node. Such a determination method is applicable to both YUV4:2:0 and YUV4:2:2 video data formats.
[0491] For example, if the size of the current node is 8x8 and two 8x4 (or 4x8) nodes are generated by horizontal binary tree splitting (or vertical binary tree splitting), further splitting the 8x4 (or 4x8) node will generate a 4x4 block. In this case, vertical binary tree splitting (or horizontal binary tree splitting) is not allowed for the 8x4 (or 4x8) node, or further splitting of the 8x4 (or 4x8) node is not allowed. As another example, if the quantity of samples of the luma block of the current node is 128 and the partitioning mode is horizontal ternary tree splitting or vertical ternary tree splitting, the quantity of samples of the luma block may be determined to be 64. However, if the horizontal ternary tree splitting or vertical ternary tree splitting of the child node is used for a luma node with a quantity of samples of 64, a 4x4 luma block is obtained. Therefore, when restricted to performing only inter prediction, no horizontal or vertical ternary tree partitioning is allowed for nodes of quantity 64 samples, or any further partitioning of nodes of quantity 64 samples is allowed.
[0492] Step 505: Parse the coding unit to obtain prediction mode information.
[0493] Based on the type of prediction mode of the coding unit, the syntax elements related to intra prediction or inter prediction are analyzed to obtain the final prediction mode of the coding unit, and a predictor is obtained by performing prediction in the corresponding prediction mode.
[0494] If the current node is located in an intra-image region (i.e., the type or slice (slice_type) of the image where the current node is located is intra-type or I-type) and the use of IBC mode is allowed, the value of cu_pred_mode is derived as 1, and the value of cu_pred_mode is obtained without the need to parse the bitstream. If the current node is located in an intra-image region but the use of IBC mode is not allowed, the value of cu_pred_mode is derived as 1, the value of cu_skip_flag is 0, and the value of cu_pred_mode is obtained without the need to parse the bitstream.
[0495] If the current node is located in an inter-picture region (i.e., the type or slice_type of the picture in which the current node is located is inter-type or B-type), the value of cu_pred_mode is derived as 0, and the value of cu_pred_mode is obtained without the need to parse the bitstream.
[0496] Step 506: Decode each coding block to obtain a reconstructed signal of the image block corresponding to the current node.
[0497] For example, according to the prediction block of the CU, an inter prediction process or an intra prediction process is performed on each CU to obtain an inter prediction image or an intra prediction image of the CU. Then, according to the residual information of each CU, a dequantization and inverse transform process is performed on the transform coefficient to obtain a residual image, and the residual image is added to the prediction image of the corresponding region to generate a reconstructed image.
[0498] Encoder side support:
[0499] The image prediction method provided in one embodiment is used in a video encoder 18 shown in Figure 8. This embodiment includes the following steps.
[0500] Step 601: Get the partition mode of the current node.
[0501] Step 601 in this embodiment is the same as step 501, and the details are not described again here.
[0502] Step 602: Derive the value of the variable modeTypeCondition according to the following method.
[0503] The value of modeTypeCondition is a first value, e.g., 0, if one or more of the following first preset conditions are true: (1) The type of the image or slice where the current node is located is I type (slice_type == I), and the value of qtbtt_dual_tree_intra_flag is 1; (2) The prediction mode type of the current node is only intra prediction or inter prediction, i.e., it is restricted to only perform inter prediction or intra prediction (non-inter prediction); or (3) Chroma format is Monochrome format or 4:4:4 format, where, for example, the value of chroma_format_idc is 0 or 3.
[0504] In another embodiment, the first preset condition further comprises the following condition (4): (4) The chroma format is monochrome format, 4:4:4 format, or 4:2:2 format, where, for example, the value of chroma_format_idc is 0, 3, or 2.
[0505] Otherwise, the value of modeTypeCondition is a second value, e.g., 1, if one or more of the following second preset conditions are true: (1) The product of the width and height of the luma block of the current node is 64, and the partitioning mode of the current node is quadtree partitioning; (2) The product of the width and height of the luma block of the current node is 64, and the partitioning mode of the current node is horizontal ternary tree partitioning or vertical ternary tree partitioning; or (3) The product of the width and height of the luma block of the current node is 32, and the partitioning mode of the current node is horizontal binary tree partitioning or vertical binary tree partitioning.
[0506] Otherwise, if one or more of the following predefined third conditions are true and the chroma format is 4:2:0 (the value of chroma_format_idc is 1), the value of modeTypeCondition is derived according to the following formula: 1 + (slice_type != I ? 1 : 0): (1) The product of the width and height of the luma block of the current node is 64, and the partitioning mode of the current node is horizontal binary tree partitioning or vertical binary tree partitioning; or (2) The product of the width and height of the luma block of the current node is 128, and the partitioning mode of the current node is horizontal ternary tree partitioning or vertical ternary tree partitioning.
[0507] Step 603: Determine the prediction mode types of all coding units of the current node based on the value of modeTypeCondition.
[0508] Specifically, all coding units of the current node are restricted from performing intra prediction (MODE_INTRA) when the value of modeTypeCondition is 1. Optionally, the value of mode_constraint_flag is set to 1.
[0509] When the value of modeTypeCondition is 2, the value of the syntax element mode_constraint_flag is determined using the RDO method. For example, the RD cost generated when performing inter prediction for all coding units of the current node is calculated first; then, the RD cost generated when performing intra prediction is calculated; then, when there is no residual when performing inter prediction for all coding units of the current node (e.g., skip mode is used), it is determined to perform inter prediction for all coding units of the current node, the value of mode_constraint_flag is set to 0, and there is no need to calculate the RD cost generated in the case of intra prediction. Alternatively, the RD cost generated when performing intra prediction for all coding units of the current node may be calculated first; then, the RD cost generated when performing inter prediction is calculated; then, the prediction mode with the smaller RD cost is determined as the final prediction mode.
[0510] Otherwise, the prediction mode type of all coding units of the current node is not restricted and is the same as the prediction mode type of the current node.
[0511] In particular, if the current node is located in an intra image region (i.e., the type or slice of the image (slice_type) of the current node is intra type or I type) and the use of IBC mode is allowed, then the value of pred_mode_flag defaults to 1. If the current node is located in an intra image region but the use of IBC mode is not allowed, then the value of pred_mode_flag defaults to 1 and the value of cu_skip_flag is 0.
[0512] Step 604: Determine the partition mode of the chroma block and the partition mode of the luma block corresponding to the current node to obtain a chroma coding unit and a luma coding unit.
[0513] (Step 604 is the same as step 504).
[0514] Step 605: Predict the coding block of the CU obtained by splitting the current node to obtain a predictor of the coding block.
[0515] If only intra prediction is performed for all coding blocks of the current node, the encoder side determines an optimal intra prediction mode to be used for the current coding block according to a rate-distortion optimization (RDO) method or another method; then predicts the current coding block using the corresponding intra prediction mode to obtain a predictor for the current block.
[0516] If only inter prediction is performed for all coding blocks of the current node, the encoder side determines the optimal inter prediction mode to be used for the current coding block according to a rate-distortion optimization method or another method; then predicts the current coding block using the corresponding inter prediction mode to obtain a predictor for the current block.
[0517] In addition, the encoder assigns values to CU depth-related syntax elements, and writes the values of each syntax element into the bitstream according to the criteria of the CU-level syntax definition. For example, when only intra prediction is performed for all coding blocks of the current node, the value of pred_mode_flag is set to 1, and pred_mode_flag is not written into the bitstream. That is, pred_mode_flag does not exist in the bitstream. When it is determined that only intra prediction is performed for all coding blocks of the current node and IBC mode is not used, the value of cu_skip_flag (or skip_flag) is 0, and cu_skip_flag is not written into the bitstream. Otherwise, after determining the value of cu_skip_flag, cu_skip_flag needs to be written into the bitstream, and the cu_skip_flag is transmitted to the decoder side.
[0518] When only inter prediction is performed for all coding blocks of the current node, the value of pred_mode_flag is set to 0, and pred_mode_flag is not written to the bitstream. That is, pred_mode_flag is not present in the bitstream. When the value of pred_mode_ibc_flag is set to 0, pred_mode_ibc_flag is not written to the bitstream. That is, pred_mode_ibc_flag is not present in the bitstream.
[0519] Step 606: Obtain the reconstructed signal of the image block at the current node.
[0520] After obtaining the prediction information by intra prediction or inter prediction, the encoder obtains the residual information by subtracting the corresponding prediction information (or predictor) from the pixel value of the pixel in the current coding block, transforms the residual information using a method such as Discrete Cosine Transform (DCT), obtains a bitstream by quantization and entropy coding, and transmits the residual information to the decoder. After adding the prediction signal and the reconstructed residual signal, the encoder needs to further perform a filtering operation to obtain a reconstructed signal, and uses the reconstructed signal as a reference signal for subsequent encoding. In particular, when the skip mode is used for the coding block, there is no residual information, there is no need to perform a transform at the encoder side, and the predictor is the final reconstructed value.
[0521] 15 is a schematic structural diagram of the functions of an image prediction device according to an embodiment of the present application. As shown in FIG. 15, the image prediction device 40 provided in this embodiment includes: an acquisition module 41 configured to acquire a partitioning mode of a current node; a determining module 42 configured to determine whether an image block having a preset size is obtained by dividing a current node in a division mode, the image block including a luma block or a chroma block; and an execution module 43 configured to perform intra prediction or inter prediction on all coding blocks covered by the current node when it is determined that an image block having a preset size is obtained by dividing the current node in the division mode.
[0522] Optionally, the image block having the preset size includes a luma block having a first preset size, and the determination module 42 is particularly configured to determine, based on the size of the current node and the partitioning mode of the current node, whether the luma block having the first preset size is obtained by partitioning the current node in the partitioning mode.
[0523] Optionally, when it is determined that a luma block having a first preset size is obtained by splitting the current node in the split mode, the execution module 43 is particularly configured to perform intra prediction for all coding blocks covered by the current node.
[0524] Optionally, when a luma block having a first preset size is not obtained by splitting the current node in the split mode, the determining module 42 is further configured to determine whether a chroma block having a second preset size is obtained by splitting the current node in the split mode; When it is determined that a chroma block having a second preset size is obtained by splitting the current node in the split mode, the execution module 43 is specifically configured to perform intra prediction or inter prediction for all coding blocks covered by the current node.
[0525] Optionally, the image block having the preset size includes a chroma block having a second preset size, and the determination module 42 is particularly configured to determine, based on the size of the current node and the splitting mode of the current node, whether the chroma block having the second preset size is obtained by splitting the current node in the splitting mode.
[0526] Optionally, when it is determined that a chroma block having a second preset size is obtained by splitting the current node in the split mode, the execution module 43: Analyzing the prediction mode status flag of the current node; and It is particularly configured to: perform inter prediction for all coding blocks covered by the current node when the value of the prediction mode status flag is a first value; or perform intra prediction for all coding blocks covered by the current node when the value of the prediction mode status flag is a second value.
[0527] Optionally, when it is determined that a chroma block having a second preset size is obtained by splitting the current node in the split mode, the execution module 43 is specifically configured to perform inter prediction for all coding blocks covered by the current node if the prediction mode used for any coding block covered by the current node is inter prediction; or to perform intra prediction for all coding blocks covered by the current node if the prediction mode used for any coding block covered by the current node is intra prediction.
[0528] Optionally, the given coding block is the first coding block among all coding blocks covered by the current node in decoding order.
[0529] Optionally, when it is determined that a chroma block having a second preset size is obtained by splitting the current node in the split mode, the execution module 43: Determining whether a luma block having a first preset size is obtained by splitting the current node in a split mode; and It is particularly configured to perform intra prediction for all coding blocks covered by the current node when it is determined that a luma block having a first preset size is obtained by splitting the current node in the split mode.
[0530] Optionally, when it is determined that a luma block having the first preset size is not obtained by splitting the current node in the split mode, the execution module 43: Parsing the prediction mode status flag of the current node: and It is particularly configured to: perform inter prediction for all coding blocks covered by the current node when the value of the prediction mode status flag is a first value; or perform intra prediction for all coding blocks covered by the current node when the value of the prediction mode status flag is a second value.
[0531] Optionally, when it is determined that a luma block having the first preset size is not obtained by splitting the current node in the split mode, the execution module 43: It is particularly configured to: perform inter prediction for all coding blocks covered by the current node if the prediction mode used for any coding block covered by the current node is inter prediction; or perform intra prediction for all coding blocks covered by the current node if the prediction mode used for any coding block covered by the current node is intra prediction.
[0532] Optionally, the execution module 43 In a split mode, splitting a luma block included in the current node to obtain a luma block obtained by splitting, and performing intra prediction on the luma block obtained by splitting; and using a chroma block included in the current node as a chroma coding block, and performing intra prediction on the chroma coding block; or The pixel shift register is particularly configured to: in a split mode, split a luma block included in the current node to obtain a luma block obtained by the splitting, and perform inter prediction on the luma block obtained by the splitting; and in a split mode, split a chroma block included in the current node to obtain a chroma block obtained by the splitting, and perform inter prediction on the chroma block obtained by the splitting.
[0533] Optionally, the execution module 43 In a split mode, splitting a luma block included in the current node to obtain a luma block obtained by splitting, and performing intra prediction on the luma block obtained by splitting; and using a chroma block included in the current node as a chroma coding block, and performing intra prediction on the chroma coding block; or In a split mode, the pixel shift register is particularly configured to: split a luma block included in the current node to obtain a luma block obtained by the splitting, and perform inter prediction on the luma block obtained by the splitting; and use a chroma block included in the current node as a chroma coding block, and perform inter prediction on the chroma coding block.
[0534] Optionally, when inter prediction is performed for all coding blocks covered by the current node, the obtaining module 41 is further configured to obtain a partitioning sub-mode of a child node of the current node, where the child node includes a luma block and a chroma block; and The determining module 42 is further configured to determine whether a luma block having a first preset size is obtained by splitting a child node of the current node in the split submode; and When it is determined that a luma block having a first preset size is obtained by splitting the child node of the current node in the split sub-mode, the execution module 43 is particularly configured to: split the child node of the current node in a split mode other than the split sub-mode to obtain a corresponding coding block and perform inter prediction on the corresponding coding block; or use the child node of the current node as a coding block and perform inter prediction on the coding block.
[0535] The image prediction device provided in this embodiment of the present application can implement the technical solutions in the above-mentioned method embodiments, and the implementation principles and their technical effects are similar, and the details are not described again here.
[0536] Figure 16 is a schematic structural diagram of the hardware of a video encoding device according to an embodiment of the present application. As shown in Figure 16, the video encoding device 50 provided in this embodiment includes a processor 51 and a memory 52 configured to store executable instructions of the processor 51. The processor 51 can execute the image prediction method corresponding to the video encoding device in the above-mentioned method embodiment. The implementation principle and its technical effect are similar, and the details are not described again here.
[0537] Optionally, the memory 52 may be separate or integrated with the processor 51 .
[0538] In the case where the memory 52 is a separate component from the processor 51 , the video encoding device 50 further includes a bus 53 configured to connect the memory 52 and the processor 51 .
[0539] Figure 17 is a schematic structural diagram of the hardware of a video decoding device according to an embodiment of the present application. As shown in Figure 17, the video decoding device 60 provided in this embodiment includes a processor 61 and a memory 62 configured to store executable instructions of the processor 61. The processor 61 can execute the image prediction method corresponding to the video decoding device in the above-mentioned method embodiment. The implementation principle and its technical effect are similar, and the details are not described again here.
[0540] Optionally, memory 62 may be separate or integrated with processor 61 .
[0541] In the case where the memory 62 is a separate component from the processor 61 , the video decoding device 60 further includes a bus 63 configured to connect the memory 62 and the processor 61 .
[0542] Figure 18 is a schematic structural diagram of an image prediction system according to an embodiment of the present application. As shown in Figure 18, the image prediction system provided in this embodiment includes a video acquisition device 70, a video encoding device 50 of the embodiment shown in Figure 16, a video decoding device 60 of the embodiment shown in Figure 17, and a display device 80.
[0543] The video encoding device 50 is connected to both a video acquisition device 70 and a video decoding device 60 , which is connected to a display device 80 .
[0544] Specifically, the video encoding device 50 receives video or image information transmitted by the video acquisition device 70. The video encoding device 50 may perform an image prediction method corresponding to the video encoding device 50 in the above-mentioned method embodiments. The video encoding device 50 transmits the encoded video or image information to the video decoding device 60. The video decoding device 60 may perform an image prediction method corresponding to the video decoding device 60 in the above-mentioned method embodiments. The video decoding device 60 transmits the decoded video or image information to the display device 80 for display.
[0545] The image prediction system provided in this embodiment of the present application includes a video encoding device that can implement the above-mentioned method embodiment and a video decoding device that can implement the above-mentioned method embodiment. The implementation principle and its technical effect are similar, and the details are not described again here.
[0546] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, which is executed by a processor to perform the steps in the above-mentioned method embodiments.
[0547] An embodiment of the present application further provides a video decoding method, the method comprising: Obtaining a split mode of a current node, where the current node includes a luma block and a chroma block; determining whether a small chroma block is obtained by further dividing the current node in a division mode of the current node, where the small chroma block is a chroma block having a pixel quantity equal to or less than a first preset value, or the small chroma block is a block-by-block chroma block having a pixel quantity equal to or less than a second preset value; When a small chroma block is obtained by further dividing the current node in the division mode of the current node, performing inter prediction or intra prediction on a coding block obtained by dividing the current node serving as a root node to obtain prediction information of the coding block obtained by division;
[0548] Optionally, the step of performing inter prediction on coding blocks obtained by splitting the current node acting as a root node comprises the step of performing inter prediction on all coding blocks obtained by splitting the current node acting as a root node; or The step of performing intra prediction on coding blocks obtained by dividing the current node that serves as the root node includes a step of performing intra prediction on all coding blocks obtained by dividing the current node that serves as the root node.
[0549] Optionally, performing inter prediction on coding blocks obtained by splitting the current node serving as a root node includes performing inter prediction on all small chroma blocks obtained by splitting the current node serving as a root node; or The step of performing intra prediction on a coding block obtained by dividing the current node serving as the root node includes a step of performing intra prediction on all small chroma blocks obtained by dividing the current node serving as the root node.
[0550] Optionally, performing inter prediction on a coding block obtained by splitting the current node serving as a root node comprises performing inter prediction on a coding unit obtained by splitting the current node serving as a root node; or The step of performing intra prediction on a coding block obtained by dividing the current node that functions as a root node includes the step of performing intra prediction on a coding unit obtained by dividing the current node that functions as a root node.
[0551] Optionally, the step of performing inter prediction or intra prediction on a coding block obtained by dividing the current node serving as a root node comprises: analysing the node prediction mode flag (cons_pred_mode_flag) of the current node; and performing inter prediction on a coding block obtained by dividing the current node that serves as a root node when the value of the node prediction mode flag is a first value; or The method includes the step of: performing intra prediction on a coding block obtained by dividing the current node that serves as the root node when the value of the node prediction mode flag is a second value.
[0552] Optionally, the step of performing inter prediction or intra prediction on a coding block obtained by dividing the current node serving as a root node comprises: performing inter prediction on a coding block obtained by splitting the current node serving as a root node, if the prediction mode used for any coding block obtained by splitting the current node serving as a root node is inter prediction; or The method includes a step of performing intra prediction on a coding block obtained by splitting the current node serving as the root node, when the prediction mode used for any coding block obtained by splitting the current node serving as the root node is intra prediction.
[0553] Optionally, the first preset value is 2 or 4, or the second preset value is 16, 8, or 32.
[0554] Optionally, the step of performing intra prediction on a coding block obtained by splitting the current node serving as a root node comprises: In a node split mode, splitting a luma block included in the current node to obtain a luma coding block; performing intra prediction on the luma coding block; Using the chroma block included in the current node as a chroma coding block, and performing intra prediction on the chroma coding block;
[0555] Optionally, the step of performing inter prediction or intra prediction on a coding block obtained by dividing the current node serving as a root node comprises: In a node split mode, splitting a luma block included in the current node to obtain a luma coding block; performing inter prediction or intra prediction on the luma coding block; Using the chroma block included in the current node as a chroma coding block, and performing inter prediction or intra prediction on the chroma coding block;
[0556] Optionally, the step of using the chroma block included in the current node as a chroma coding block and performing inter prediction or intra prediction on the chroma coding block includes: using the chroma coding block as a chroma prediction block and performing intra prediction on the chroma prediction block; or The method includes a step of: dividing a chroma coding block to obtain a chroma prediction block; and performing inter prediction on the chroma prediction block obtained by the division.
[0557] An embodiment of the present application further provides a video decoding method, the method comprising: Obtaining a split mode of a current node, where the current node includes a luma block and a chroma block; determining whether a luma block having a preset size is obtained by further dividing the current node in a division mode of the current node; When a luma block having a predetermined size is obtained by further dividing the current node in the division mode of the current node, performing intra prediction on all coding blocks obtained by dividing the current node serving as the root node, and obtaining prediction blocks of the coding blocks obtained by division.
[0558] Optionally, this method When a luma block having a preset size is not obtained by further dividing the current node in the division mode of the current node, determining whether a small chroma block is obtained by further dividing the current node in the division mode of the current node, where the small chroma block is a chroma block having a pixel quantity equal to or less than a first preset value, or the small chroma block is a block-by-block chroma block having a pixel quantity equal to or less than a second preset value; The method further includes a step of performing inter prediction or intra prediction on a coding block obtained by splitting the current node serving as a root node to obtain a prediction block of the coding block obtained by splitting when a small chroma block is obtained by further splitting the current node in the splitting mode of the current node.
[0559] Optionally, the step of performing inter prediction on coding blocks obtained by splitting the current node acting as a root node comprises the step of performing inter prediction on all coding blocks obtained by splitting the current node acting as a root node; or The step of performing intra prediction on coding blocks obtained by dividing the current node that serves as the root node includes a step of performing intra prediction on all coding blocks obtained by dividing the current node that serves as the root node.
[0560] Optionally, performing inter prediction on coding blocks obtained by splitting the current node serving as a root node includes performing inter prediction on all small chroma blocks obtained by splitting the current node serving as a root node; or The step of performing intra prediction on a coding block obtained by dividing the current node serving as the root node includes a step of performing intra prediction on all small chroma blocks obtained by dividing the current node serving as the root node.
[0561] Optionally, performing inter prediction on a coding block obtained by splitting the current node serving as a root node comprises performing inter prediction on a coding unit obtained by splitting the current node serving as a root node; or The step of performing intra prediction on a coding block obtained by dividing the current node that functions as a root node includes the step of performing intra prediction on a coding unit obtained by dividing the current node that functions as a root node.
[0562] Optionally, the step of performing inter prediction or intra prediction on a coding block obtained by dividing the current node serving as a root node comprises: Analyzing a node prediction mode flag (cons_pred_mode_flag) of the current node; performing inter prediction on a coding block obtained by dividing the current node serving as a root node when the value of the node prediction mode flag is a first value; When the value of the node prediction mode flag is a second value, performing intra prediction on a coding block obtained by dividing the current node that serves as the root node.
[0563] Optionally, the step of performing inter prediction or intra prediction on a coding block obtained by dividing the current node serving as a root node comprises: performing inter prediction on a coding block obtained by splitting the current node serving as a root node, when a prediction mode used for any coding block obtained by splitting the current node serving as a root node is inter prediction; The method includes a step of performing intra prediction on a coding block obtained by splitting the current node serving as the root node when the prediction mode used for any coding block obtained by splitting the current node serving as the root node is intra prediction.
[0564] Optionally, the first preset value is 2 or 4, or the second preset value is 16, 8, or 32.
[0565] Optionally, the step of performing intra prediction on a coding block obtained by splitting the current node serving as a root node comprises: In a node split mode, splitting a luma block included in the current node to obtain a luma coding block; performing intra prediction on the luma coding block; Using the chroma block included in the current node as a chroma coding block, and performing intra prediction on the chroma coding block;
[0566] Optionally, the step of performing inter prediction or intra prediction on a coding block obtained by dividing the current node serving as a root node comprises: In a node split mode, splitting a luma block included in the current node to obtain a luma coding block; performing inter prediction or intra prediction on the luma coding block; Using the chroma block included in the current node as a chroma coding block, and performing inter prediction or intra prediction on the chroma coding block;
[0567] Optionally, the step of using the chroma block included in the current node as a chroma coding block and performing inter prediction or intra prediction on the chroma coding block includes: using the chroma coding block as a chroma prediction block and performing intra prediction on the chroma prediction block; or The method includes a step of dividing a chroma coding block to obtain a chroma prediction block, and performing inter prediction on the chroma prediction block obtained by the division.
[0568] Optionally, the step of performing inter prediction on a coding block obtained by splitting the current node serving as a root node comprises: splitting the current node with a split mode of the current node to obtain a child node of the current node; Obtaining a split sub-mode of a child node of a current node, the child node including a luma block and a chroma block; determining whether a luma block having a preset size is obtained by further dividing a child node of the current node in a division sub-mode; When a luma block having a predetermined size is obtained by further dividing a child node of the current node in the division sub-mode, the child node of the current node is divided in a division mode other than the division sub-mode to obtain a corresponding coding unit, and inter prediction is performed on the corresponding coding unit; or the child node of the current node is used as a coding unit, and inter prediction is performed on the coding unit.
[0569] Optionally, the pre-set size can be 4x4, 4x8, 8x4, 2x4, or 4x2.
[0570] An embodiment of the present application further provides a video decoding method, the method comprising: Obtaining a split mode of a current node, where the current node includes a luma block and a chroma block; When a prediction mode used for all coding blocks obtained by dividing the current node serving as a root node is an inter prediction mode, dividing the current node in a division mode of the current node to obtain a child node of the current node; Obtaining a split sub-mode of a child node of a current node, the child node including a luma block and a chroma block; determining whether a luma block having a preset size is obtained by further dividing a child node of the current node in a division sub-mode; When a luma block having a predetermined size is obtained by further dividing a child node of the current node in the division sub-mode, the child node of the current node is divided in a division mode other than the division sub-mode to obtain a corresponding coding unit, and inter prediction is performed on the corresponding coding unit; or the child node of the current node is used as a coding unit, and inter prediction is performed on the coding unit.
[0571] The first video decoding method provided in this embodiment of the present application relates to a block division mode in video decoding. The video data format in this embodiment is YUV4:2:0 format. The same method can also be used for data in YUV4:2:2 format.
[0572] Step 1: Analyze the split mode S of node A. If node A is to be further split, step 2 is performed. If the current node is not to be further split into child nodes, the current node corresponds to one coding unit, and information about the coding unit is analyzed.
[0573] The partitioning mode of node A may be at least one of quadtree partitioning, vertical binary tree partitioning, horizontal binary tree partitioning, vertical ternary tree partitioning, and horizontal ternary tree partitioning. The partitioning mode may be another partitioning mode. This is not limited to the present application. Information about the partitioning mode of the current node may be transmitted in the bitstream. The partitioning mode of the current node can be obtained by parsing the corresponding syntax element in the bitstream. Alternatively, the partitioning mode of the current node may be determined according to a preset rule. This is not limited to the present application.
[0574] Step 2: Determine whether the chroma block of at least one child node B in a child node obtained by splitting node A in split mode S is a small block (by determining whether the width and height of node A, and / or the split mode of node A, and / or the width and height of node B satisfy at least one of the following conditions). If the chroma block of at least one child node B in a child node obtained by splitting node A is a small block, steps 3 to 6 are executed.
[0575] Specifically, whether a chroma block of at least one child node B of node A is a small block may be determined according to one of the following methods: (1) If the size of the chroma block of at least one child node B of node A is 2×2, 2×4, or 4×2, then the chroma block of at least one child node B of node A is a small block; (2) If the width or height of a chroma block of at least one child node B of node A is 2, then the chroma block of at least one child node B of node A is a small block; (3) A chroma block of at least one child node B of node A is a small block when node A contains 128 luma pixels and ternary tree partitioning is used for node A, or when node A contains 64 luma pixels and binary tree partitioning, quad tree partitioning, or ternary tree partitioning is used for node A; (4) When node A contains 256 luma pixels and ternary tree partitioning or quad-tree partitioning is used for node A, or when node A contains 128 luma pixels and binary tree partitioning is used for node A, the chroma block of at least one child node B of node A is a small block; (5) When node A contains N1 luma pixels and ternary tree partitioning is used for node A (N1 is 64, 128, or 256), the chroma block of at least one child node B of node A is a small block; (6) If node A contains N2 luma pixels and quadtree partitioning is used for node A (N2 is 64 or 256), the chroma block of at least one child node B of node A is a small block; or (7) If node A contains N3 luma pixels and binary tree partitioning is used for node A (N3 is 64, 128, or 256), the chroma block of at least one child node B of node A is a small block.
[0576] Note that node A contains 128 luma pixels, which can also be described as the area of the current node being 128, or the product of the width and height of node A being 128. Details will not be described here.
[0577] Step 3: Restrict node A from performing intra prediction for all coding units in the coverage area of node A, or restrict node A from performing inter prediction for all coding units in the coverage area of node A. Since intra prediction or inter prediction is performed for all coding units in the coverage area of node A, small block parallel processing can be implemented by hardware, thereby improving coding performance.
[0578] Whether to perform intra prediction or inter prediction for all coding units within the coverage area of node A may be determined according to one of the following methods.
[0579] Method 1: Determine whether intra- or inter-prediction is performed based on a flag in the syntax table.
[0580] If the chroma blocks of at least one child node B obtained by splitting node A with split mode S are small blocks (and the chroma blocks of node A are not small blocks), the flag cons_pred_mode_flag is parsed from the bitstream. If the value of cons_pred_mode_flag is 0, it indicates that inter prediction is performed for all coding units in the coverage area of node A, or if the value of cons_pred_mode_flag is 1, it indicates that intra prediction is performed for all coding units in the coverage area of node A. cons_pred_mode_flag may be a syntax element that needs to be parsed during block splitting. If the syntax element is parsed, the cu_pred_mode of the coding units in the coverage area of node A may not be parsed, and the value of cu_pred_mode is a default value corresponding to the value of cons_pred_mode_flag.
[0581] When only intra prediction mode is used for the child nodes of node A, for example, when node A is located in an intra image (i.e., the type of the image where node A is located is intra type or I type), or when node A is located in an intra image and IBC technology is not used in the sequence, the value of cons_pred_mode_flag is 1 by default, that is, cons_pred_mode_flag is not present in the bitstream. IBC technology may belong to inter prediction or intra prediction.
[0582] Method 2: Based on the prediction mode of the first node in node A's region, determine whether intra prediction or inter prediction is performed.
[0583] The prediction mode of the first coding unit B0 in the area of node A (the prediction mode of the first coding unit B0 is not limited) is analyzed. If the prediction mode of the first coding unit B0 is intra prediction, intra prediction is performed for all coding units in the coverage area of node A. If the prediction mode of the first coding unit B0 is inter prediction, inter prediction is performed for all coding units in the coverage area of node A.
[0584] Step 4: Determine the partition mode of the chroma blocks and the partition mode of the luma blocks of node A based on the prediction modes used for the coding units in the coverage area of node A.
[0585] When intra prediction mode is used for all coding units in the coverage area of node A, the luma block of node A is split with a split mode S to obtain N luma coding tree nodes. The chroma block of node A is not split any further and corresponds to one chroma coding block (chroma CB for short). The N luma coding tree nodes may be restricted to not be split any further or this is not restricted. When the luma child node is further split, the split mode of the luma child node is analyzed for recursive splitting. When the luma coding tree node is not split any further, the luma coding tree node corresponds to one luma coding block (luma CB for short). The chroma transform block and the chroma coding block corresponding to the chroma CB are the same size, and the chroma prediction block and the chroma coding block are the same size.
[0586] If inter prediction mode is used for all coding units in the coverage area of node A, the luma blocks and chroma blocks of node A are further split into N coding tree nodes in a split mode S, where the coding tree nodes include luma blocks and chroma blocks. The N coding tree nodes may or may not be further split. In the absence of further splitting, the N coding tree nodes correspond to coding units that include luma blocks and chroma blocks.
[0587] Step 5: Analyze the prediction information and residual information of the CU obtained by splitting node A.
[0588] The prediction information includes a prediction mode (indicating an intra prediction mode or a non-intra prediction mode), an intra prediction mode, an inter prediction mode, motion information, etc. The motion information may include information such as a prediction direction (forward, backward, or bidirectional), a reference index, and a motion vector.
[0589] The residual information includes coded block flags (cbf), transform coefficients, transform type (DCT-2, DCT-7, or DCT-8, etc.) The transform type may be DCT-2 by default.
[0590] When all CUs obtained by splitting node A are restricted to perform only intra prediction, the step of analyzing prediction information of luma CB obtained by splitting node A includes the steps of: setting the value of skip_flag, the value of merge_flag, and the value of cu_pred_mode to 0, 0, and 1 by default, respectively (i.e., skip_flag, merge_flag, and cu_pred_mode are all absent in the bitstream), or setting the value of skip_flag and the value of cu_pred_mode to 0 and 1 by default, respectively (i.e., skip_flag and cu_pred_mode are both absent in the bitstream); and analyzing information on intra prediction mode of luma CB. The step of analyzing prediction information of chroma CB obtained by splitting node A includes the step of analyzing intra prediction mode of chroma CB. The intra prediction mode of chroma CB can be analyzed according to the following method: (1) Parsing syntax elements in the bitstream to obtain intra-prediction modes; or (2) In a set of chroma intra-prediction modes, such as a linear model mode, a chroma derived mode (DM) mode, and an IBC mode, the intra-prediction mode is directly set to one intra-prediction mode.
[0591] When all CUs obtained by splitting node A are restricted to performing only inter prediction, the step of analyzing the prediction mode of the CU obtained by splitting node A includes the steps of analyzing skip_flag and / or merge_flag, setting the cu_pred_mode value to 0 by default, and analyzing inter prediction information such as merge index, inter direction, reference index, motion vector predictor inde...
Claims
1. 1. A method of image prediction, the method comprising: obtaining a partition mode of a current node, the current node being an image block in a coding tree unit of a current image; determining whether the current node satisfies a preset condition based on the partitioning mode of the current node and the size of the current node; and when it is determined that the current node satisfies the preset condition, performing prediction on a coding block belonging to the current node using a same prediction mode to obtain a predictor of the coding block belonging to the current node, wherein the prediction mode is intra prediction or inter prediction. method.
2. The step of determining whether the current node satisfies the preset condition includes: The method of claim 1 , comprising determining whether the current node satisfies the preset condition based on the split mode of the current node, the size of the current node, and a chroma format of the current node.
3. The step of performing the prediction on the coding block belonging to the current node comprises: analyzing a prediction mode status flag of the current node; 3. The method of claim 1 or 2, further comprising: performing inter prediction on the coding block belonging to the current node when the value of the prediction mode status flag is a first value; or performing intra prediction on the coding block belonging to the current node when the value of the prediction mode status flag is a second value.
4. The step of performing inter prediction on the coding block belonging to the current node includes: splitting the current node with the split mode of the current node to obtain a child node of the current node; determining disallowed split modes for the child nodes of the current node based on the size of the child nodes of the current node; determining a block partitioning policy for the child node of the current node based on the partitioning modes that are not permitted for the child node of the current node; and obtaining a coding block corresponding to the child node of the current node and performing inter prediction on the corresponding coding block according to the block partitioning policy of the child node of the current node.
5. 5. The method of claim 1, wherein the size of the current node is determined based on a size of a coding tree node that corresponds to the current node and the partitioning mode used to obtain the current node.
6. The preset conditions include: The method of claim 1 , wherein the number of samples of a luma block of the current node is 64, and the partitioning mode is a binary tree partitioning.
7. The preset conditions include: The method of claim 1 , wherein the number of samples of a luma block of the current node is 128, and the partitioning mode is ternary tree partitioning.
8. The method of claim 2 , wherein the chroma format of the current node is YUV 4:2:0 or YUV 4:2:
2.
9. The preset conditions include: the number of samples of the luma block of the current node is 64, the partitioning mode is binary tree partitioning, and the chroma format is 4:2:0; or The method of claim 2 , comprising at least one of: a number of samples of a luma block of the current node is 128; the partitioning mode is ternary tree partitioning; and the chroma format is 4:2:
0.
10. 1. A video coding apparatus, comprising: A processor; a memory coupled to the processor for storing executable instructions; The instructions, when executed by the processor, cause the processor to perform operations, including: obtaining a partition mode of a current node, the current node being an image block in a coding tree unit of a current image; determining whether the current node satisfies a preset condition based on the partitioning mode of the current node and the size of the current node; and When it is determined that the current node satisfies the preset condition, performing prediction on a coding block belonging to the current node using a same prediction mode to obtain a predictor of the coding block belonging to the current node, where the prediction mode is intra prediction or inter prediction. Device.
11. Determining whether the current node satisfies the preset condition includes: The apparatus of claim 10 , further comprising: determining whether the current node satisfies the preset condition based on the split mode of the current node, the size of the current node, and a chroma format of the current node.
12. Performing prediction on the coding block belonging to the current node includes: analyzing a prediction mode status flag of the current node; The device of claim 10 or 11, further comprising: performing inter prediction on the coding block belonging to the current node when the value of the prediction mode status flag is a first value; or performing intra prediction on the coding block belonging to the current node when the value of the prediction mode status flag is a second value.
13. Performing inter prediction on the coding block belonging to the current node includes: splitting the current node with the split mode of the current node to obtain a child node of the current node; determining disallowed split modes for the child nodes of the current node based on a size of the child nodes of the current node; determining a block partitioning policy for the child node of the current node based on the partitioning modes that are not permitted for the child node of the current node; and obtaining a coding block corresponding to the child node of the current node and performing inter prediction on the corresponding coding block according to the block partitioning policy of the child node of the current node.
14. 14. The apparatus of claim 10, wherein the size of the current node is determined based on a size of a coding tree node that corresponds to the current node and the partitioning mode used to obtain the current node.
15. The preset conditions include: The apparatus of claim 10 , further comprising: a number of samples of a luma block of the current node being 64; and the partitioning mode being a binary tree partitioning.
16. The preset conditions include: The apparatus of claim 10 , wherein the number of samples of a luma block of the current node is 128, and the partitioning mode is ternary tree partitioning.
17. The apparatus of claim 11 , wherein the chroma format of the current node is YUV 4:2:0 or YUV 4:2:
2.
18. The preset conditions include: the number of samples of the luma block of the current node is 64, the partitioning mode is binary tree partitioning, and the chroma format is 4:2:0; or 12. The apparatus of claim 11, further comprising at least one of: a number of samples of a luma block of the current node is 128; the partitioning mode is ternary tree partitioning; and the chroma format is 4:2:
0.
19. 1. A non-transitory storage medium containing an encoded bitstream, the encoded bitstream comprising: obtaining a partition mode of a current node, the current node being an image block in a coding tree unit of a current image; determining whether the current node satisfies a preset condition based on the partitioning mode of the current node and the size of the current node; and When it is determined that the current node satisfies the preset condition, performing prediction on a coding block belonging to the current node using the same prediction mode to obtain a predictor of the coding block belonging to the current node, where the prediction mode is intra prediction or inter prediction. Non-transitory storage media.
20. 10. A video encoding device comprising a processor and a memory configured to store executable instructions for the processor, the processor executing the method according to any one of claims 1 to 9. Video encoding device.
21. 10. A video decoding apparatus comprising a processor and a memory configured to store executable instructions for the processor, the processor executing the method of any one of claims 1 to 9. Video Decoding Device.
22. 22. An image prediction system, comprising: a video acquisition device, the video encoding device of claim 20, the video decoding device of claim 21, and a display device, the video encoding device being connected to both the video acquisition device and the video decoding device, and the video decoding device being connected to the display device. Image prediction system.
Citation Information
Cited By
Wind turbine, method of acquiring information, and wind turbine management system
EP4715202A1
