Video coding and decoding method and device and storage medium

Abstract

An apparatus for video decoding includes circuitry configured to receive syntax elements from a bitstream of an encoded video, the syntax elements being included in a parameter set or a picture header and indicating a minimum luma coding block size; verifying whether the minimum luma coding block size is within a range of allowable minimum luma coding block sizes, the range having an upper limit less than a maximum allowable coding tree unit (CTU) size; and decoding the reference parameter set or an encoded picture including a picture header in the encoded video based on the minimum luma encoding block size. The upper limit of the range of allowable minimum luma coding block sizes may be a predefined maximum allowable minimum luma coding block size.

Description

[0001] Cross-references

[0002] This application claims priority to U.S. Patent Application No. 17 / 078,302, filed on October 23, 2020, “RANGE OF MINIMUM CODING BLOCK SIZE IN VIDEO CODING,” which claims priority to U.S. Provisional Application No. 62 / 928,150, filed on October 30, 2019, “METHODS ON RANGE OF MINIMUM CODING BLOCK SIZE.” Technical Field

[0003] This application describes embodiments that generally relate to video encoding and decoding. Background Art

[0004] The background description provided herein is intended to present the background of the present application as a whole. The extent to which the work of the presently named inventors described in the background section and various aspects of this specification is performed does not indicate that it is prior art at the time of filing this application, and it is never explicitly or implicitly admitted that it is prior art for this application.

[0005] Video encoding and decoding are enabled by inter-picture prediction techniques with motion compensation. An uncompressed digital video may include a series of pictures, each picture having spatial dimensions such as 1920×1080 luminance samples and associated chrominance samples. The series of pictures has a fixed or variable picture rate (also informally referred to as a frame rate), such as 60 pictures per second or 60Hz. Uncompressed video has specific bitrate requirements. For example, 1080p60 4:2:0 video (1920x1080 luminance sample resolution, 60Hz frame rate) with 8 bits per sample requires close to 1.5Gbit / s bandwidth. One hour of such video requires more than 600GB of storage space.

[0006] One goal of video encoding and decoding is to reduce redundant information in the input video signal through compression. Video compression can help reduce the bandwidth and / or storage space requirements mentioned above, in some cases by two or more orders of magnitude. Both lossless and lossy compression, as well as combinations of the two, can be used. Lossless compression refers to techniques that reconstruct an exact replica of the original signal from the compressed original signal. When lossy compression is used, the reconstructed signal may not be exactly the same as the original signal, but the distortion between the original and the reconstructed signal is small enough that the reconstructed signal can be used for the intended application. Lossy compression is widely used in video. The amount of distortion allowed depends on the application. For example, users of some consumer streaming applications may be able to tolerate higher distortion than users of television applications. The achievable compression ratio reflects that higher allowed / tolerant distortion results in higher compression ratios.

[0007] Video encoders and decoders may utilize techniques from several broad categories including, for example, motion compensation, transforms, quantization, and entropy coding and decoding.

[0008] Video codec techniques may include techniques known as intra-frame coding. In intra-frame coding, sample values ​​are represented without reference to samples or other data from previously reconstructed reference pictures. In some video codecs, pictures are spatially subdivided into blocks of samples. When all sample blocks are encoded in intra-frame mode, the picture may be an intra-frame picture. Intra-frame pictures and their derivatives (such as independent decoder refresh pictures) can be used to reset the decoder state, and can therefore be used as the first picture in an encoded video stream and video session, or as a still image. Samples of intra-frame blocks can be exposed to transformation, and transform coefficients can be quantized before entropy coding and decoding. Intra-frame prediction can be a technique for minimizing sample values ​​in the pre-transform domain. In some cases, the smaller the DC value after transformation, and the smaller the AC coefficient, the fewer bits are required to represent the entropy coded block at a given quantization step size.

[0009] Conventional intra-frame codecs, such as those known from, for example, the MPEG-2 generation of codecs, do not use intra-frame prediction. However, some newer video compression techniques include techniques that attempt to use intra-frame prediction from, for example, surrounding sample data and / or metadata that is obtained during encoding / decoding of spatially adjacent data blocks and precedes the data blocks in decoding order. Such techniques are hereinafter referred to as "intra-frame prediction" techniques. Note that, in at least some cases, intra-frame prediction uses only reference data from the current picture being reconstructed, and not reference data from reference pictures.

[0010] There can be many different forms of intra-frame prediction. When more than one such technique can be used in a given video codec technique, the technique used can be encoded in the intra-frame prediction mode. In some cases, a mode can have sub-modes and / or parameters, and these sub-modes and / or parameters can be encoded separately or included in the mode codeword. The codeword to be used for a given mode / sub-mode / parameter combination may have an impact on the coding efficiency gain through intra-frame prediction, and the same is true for the entropy coding and decoding technique that converts the codeword into a bitstream.

[0011] Some mode of intra prediction was introduced with H.264, improved in H.265, and further improved in newer codecs such as joint exploration model (JEM), versatile video coding (VVC), and benchmark set (BMS). The predictor block can be formed using neighboring sample values ​​belonging to samples that are already available. Sample values ​​of neighboring samples are copied into the predictor block according to the direction. A reference to the direction in use can be encoded in the bitstream, or it can be predicted itself.

[0012] The mapping of the intra-frame prediction direction bits representing directions in the coded video bitstream can vary depending on the video coding technique, and can range from simple direct mapping of prediction directions to intra-frame prediction modes to codewords, to complex adaptive schemes involving the most probable mode, and similar techniques, for example. However, in all cases, there may be certain directions in the video content that are statistically less likely to occur than certain other directions. Since the goal of video compression is to reduce redundancy, in well-working video coding techniques, those unlikely directions will be represented by a larger number of bits than more likely directions.

[0013] Motion compensation may be a lossy compression technique and may involve a technique where a block of sample data from a previously reconstructed picture or part of a reconstructed picture (reference picture) is used for prediction of a newly reconstructed picture or part of a picture after being spatially shifted in the direction indicated by a motion vector (hereinafter MV). In some cases, the reference picture may be the same as the picture currently being reconstructed. The MV may have two dimensions, X and Y, or three dimensions, where the third dimension indicates the reference picture in use (the latter may indirectly be the temporal dimension).

[0014] In some video compression techniques, the MV applied to a region of sample data can be predicted based on other MVs, such as those MVs associated with another region of sample data that is spatially adjacent to the region being reconstructed and that precede the MV in decoding order. Doing so can greatly reduce the amount of data required to encode the MV, thereby eliminating redundant information and increasing the amount of compression. MV prediction can be done effectively, for example, when encoding an input video signal derived from a camera (called natural video), there is a statistical probability that regions larger than the region to which a single MV applies will move in a similar direction, so in some cases, similar motion vectors derived from MVs of neighboring regions can be used for prediction. This results in the MV found for a given region being similar or identical to the MV predicted from the surrounding MVs, and after entropy coding, it can be represented with fewer bits than the number of bits used when the MV is directly encoded. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., MV) derived from the original signal (i.e., sample stream). In other cases, MV prediction itself may be lossy, for example due to rounding errors when calculating predicted values ​​based on several surrounding MVs.

[0015] H.265 / HEVC (ITU-T Rec. H.265, "High Efficiency Video Coding", December 2016) describes various MV prediction mechanisms. Among the various MV prediction mechanisms provided by H.265, this article describes a technique hereinafter referred to as "spatial merging".

[0016] refer to Figure 1 , the current block (101) includes samples found by the encoder during motion search, which can be predicted based on the previous block that has been spatially shifted by the same amount. The MV is not encoded directly, but is derived from metadata associated with one or more reference pictures, such as the most recent (in decoding order) reference picture, by using the MV associated with any of the five surrounding samples. The five surrounding samples are represented by A0, A1 and B0, B1, B2 (from 102 to 106), respectively. In H.265, MV prediction can use the prediction value of the same reference picture being used by the neighboring blocks. Summary of the invention

[0017] Various aspects of the present disclosure provide methods and apparatus for video encoding / decoding. In some examples, an apparatus for video decoding includes a circuit. The circuit is used to receive a first syntax element from a bitstream of an encoded video, the first syntax element being contained in a parameter set or a picture header, which in some examples may indicate a minimum luminance coding block size. The circuit further verifies whether the minimum luminance coding block size is within a range of allowable minimum luminance coding block sizes, the range having an upper limit less than a maximum allowable coding tree unit CTU size; and based on the minimum luminance coding block size, decodes an encoded picture in the encoded video. In some examples, the encoded picture references a parameter set, including a picture header. In some embodiments, the upper limit of the range of allowable minimum luminance coding block sizes may be a predefined maximum allowable minimum luminance coding block size.

[0018] In some embodiments, the circuit is further configured to receive a second syntax element from the bitstream of the encoded video, the second syntax element indicating a CTU size and included in a parameter set. When the CTU size is larger than a predefined maximum allowable minimum luma coding block size, the predefined maximum allowable minimum luma coding block size is used as the upper limit of the range of allowable minimum luma coding block sizes. When the CTU size is smaller than the predefined maximum allowable minimum luma coding block size, the CTU size is used as the upper limit of the range of allowable minimum luma coding block sizes.

[0019] In some embodiments, the first syntax element indicates a binary logarithmic value of the minimum luma coding block size minus 2; the circuit is further configured to verify whether the binary logarithmic value of the minimum luma coding block size minus 2 is in the range of 0 to N (including the end values), where N is an integer and N+2 represents the binary logarithmic value of a predefined maximum allowable minimum luma coding block size.

[0020] In some embodiments, the first syntax element indicates a binary logarithmic value of the minimum luma coding block size minus 2; the circuit is further configured to verify whether the binary logarithmic value of the minimum luma coding block size minus 2 is in the range of 0 to Min(N, log2_ctu_size_minus5+3) (including end values), where N is an integer, N+2 represents the binary logarithmic value of the predefined maximum allowable minimum luma coding block size, and log2_ctu_size_minus5+5 represents the binary logarithmic value of the CTU size of the encoded video. In some embodiments, N is equal to 4.

[0021] In some embodiments, the first syntax element indicates a binary logarithmic value of the minimum luma coding block size minus 2; the circuit is further configured to verify whether the binary logarithmic value of the minimum luma coding block size is greater than Min(N+2,log2_ctu_size), where N is an integer, N+2 represents a binary logarithmic value of a predefined maximum allowable minimum luma coding block size, and log2_ctu_size represents a binary logarithmic value of a CTU size of the encoded video. In some embodiments, N is equal to 4.

[0022] In some embodiments, the first syntax element indicates a binary logarithmic value of the minimum luma coding block size minus 2; and verifies whether the binary logarithmic value of the minimum luma coding block size minus 2 is in the range of 0 to log2_ctu_size_minus5+M (including end values), where M is an integer, log2_ctu_size_minus5+5 represents the binary logarithmic value of the CTU size of the encoded video, and log2_ctu_size_minus5+M+2 represents the binary logarithmic value of the predefined maximum allowable minimum luma coding block size.

[0023] Various aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions, which, when executed by a computer for video decoding, causes the computer to perform a method for video decoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Other features, properties and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0025] Figure 1 is a schematic diagram of an exemplary current block and its surrounding spatial merging candidates.

[0026] Figure 2 is a schematic diagram of a simplified block diagram of a communication system according to an embodiment.

[0027] Figure 3 is a schematic diagram of a simplified block diagram of a communication system according to an embodiment.

[0028] Figure 4 is a schematic diagram of a simplified block diagram of a decoder according to an embodiment.

[0029] Figure 5 is a schematic diagram of a simplified block diagram of an encoder according to an embodiment.

[0030] Figure 6 A block diagram of an encoder according to another embodiment is shown.

[0031] Figure 7A block diagram of a decoder according to another embodiment is shown.

[0032] Figure 8 The vertical and horizontal positions of 4:2:0 luma and chroma samples in a picture are shown in FIG.

[0033] Fig. 9 The vertical and horizontal positions of 4:2:2 luma and chroma samples in a picture are shown in FIG.

[0034] Fig.10 The vertical and horizontal positions of 4:4:4 luma and chroma samples in a picture are shown in FIG.

[0035] Fig.11 An example of a picture divided into coding tree units CTUs is shown in FIG.

[0036] Fig.12 An example of a picture divided into tiles and raster scan slices is shown.

[0037] Fig.13 An example of a picture divided into tiles and rectangular strips is shown.

[0038] Fig.14 Examples of pictures divided into tiles, bricks, and rectangular strips are shown.

[0039] Fig.15 Shows the segmentation pattern in a multi-type tree structure.

[0040] Fig.16 Partition signaling in a quadtree with a nested multi-type tree coding tree structure is shown.

[0041] Fig.17 An example of a quadtree with a nested multi-type tree coding block structure is shown.

[0042] Fig.18 An example showing that ternary tree partitioning for a 128x128 coding block is not performed is shown.

[0043] Fig.19 The diagram shows the redundant partitioning patterns for binary tree partitioning and ternary tree partitioning.

[0044] Fig. 20 Examples of ternary and binary tree partitions that are not allowed are shown.

[0045] Fig.21 A flowchart is shown that outlines the process according to an embodiment of the present disclosure.

[0046] Fig. 22 A schematic diagram of a computer device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0047] I. Video Encoder and Decoder Systems

[0048] Figure 2 2 is a simplified block diagram of a communication system (200) according to an embodiment disclosed in the present application. The communication system (200) includes a plurality of terminal devices, which can communicate with each other through, for example, a network (250). For example, the communication system (200) includes a first pair of terminal devices (210) and a terminal device (220) interconnected through the network (250). Figure 2 In an embodiment, the terminal device (210) and the terminal device (220) perform unidirectional data transmission. For example, the terminal device (210) may encode video data (e.g., a video picture stream collected by the terminal device (210)) for transmission to another terminal device (220) via the network (250). The encoded video data is transmitted in the form of one or more encoded video code streams. The terminal device (220) may receive the encoded video data from the network (250), decode the encoded video data to restore the video data, and display the video picture based on the restored video data. Unidirectional data transmission is more common in applications such as media services.

[0049] In another embodiment, the communication system (200) includes a second pair of terminal devices (230) and (240) that perform bidirectional transmission of encoded video data, which can occur, for example, during a video conference. For bidirectional data transmission, each of the terminal devices (230) and (240) can encode video data (e.g., a video picture stream collected by the terminal device) for transmission to the other terminal device (230) and (240) through the network (250). Each of the terminal devices (230) and (240) can also receive the encoded video data transmitted by the other terminal device (230) and (240), and can decode the encoded video data to restore the video data, and can display the video picture on an accessible display device based on the restored video data.

[0050] exist Figure 2In the embodiment of the present invention, the terminal device (210), the terminal device (220), the terminal device (230) and the terminal device (240) may be a server, a personal computer and a smart phone, but the principles disclosed in the present application may not be limited thereto. The embodiments disclosed in the present application are applicable to laptop computers, tablet computers, media players and / or dedicated video conferencing equipment. The network (250) represents any number of networks that transmit encoded video data between the terminal devices (210), the terminal devices (220), the terminal devices (230) and the terminal devices (240), including, for example, wired (wired) and / or wireless communication networks. The communication network (250) may exchange data in circuit switching and / or packet switching channels. The network may include a telecommunications network, a local area network, a wide area network and / or the Internet. For the purpose of this discussion, unless explained below, the architecture and topology of the network (250) may be irrelevant to the operation disclosed in the present application.

[0051] As an example, Figure 3 The video encoder and video decoder are shown in a streaming environment. The subject matter disclosed in this application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0052] The streaming system may include an acquisition subsystem (313), which may include a video source (301) such as a digital camera, which creates an uncompressed video picture stream (302). In an embodiment, the video picture stream (302) includes samples taken by a digital camera. Compared to the encoded video data (304) (or the encoded video bitstream), the video picture stream (302) is depicted as a thick line to emphasize the high data volume of the video picture stream, and the video picture stream (302) can be processed by an electronic device (320), which includes a video encoder (303) coupled to the video source (301). The video encoder (303) may include hardware, software, or a combination of hardware and software to implement or implement various aspects of the disclosed subject matter as described in more detail below. Compared to the video picture stream (302), the encoded video data (304) (or the encoded video bitstream (304)) is depicted as a thin line to emphasize the lower amount of data of the encoded video data (304) (or the encoded video bitstream (304)), which can be stored on the streaming server (305) for future use. One or more streaming client subsystems, such as Figure 3The client subsystem (306) and the client subsystem (308) in the video transmission system can access the streaming server (305) to retrieve the copy (307) and the copy (309) of the encoded video data (304). The client subsystem (306) may include, for example, a video decoder (310) in the electronic device (330). The video decoder (310) decodes the incoming copy (307) of the encoded video data and generates an output video picture stream (311) that can be presented on a display (312) (e.g., a display screen) or another presentation device (not depicted). In some streaming transmission systems, the encoded video data (304), the video data (307), and the video data (309) (e.g., a video bitstream) can be encoded according to certain video encoding / compression standards. Examples of these standards include ITU-T H.265. In an embodiment, the video coding standard under development is informally referred to as Versatile Video Coding (VVC), and the present application can be used in the context of the VVC standard.

[0053] It should be noted that the electronic device (320) and the electronic device (330) may include other components (not shown). For example, the electronic device (320) may include a video decoder (not shown), and the electronic device (330) may also include a video encoder (not shown).

[0054] Figure 4 4 is a block diagram of a video decoder (410) according to an embodiment disclosed in the present application. The video decoder (410) may be provided in an electronic device (430). The electronic device (430) may include a receiver (431) (e.g., a receiving circuit). The video decoder (410) may be used to replace Figure 3 A video decoder (310) of an embodiment.

[0055] The receiver (431) may receive one or more encoded video sequences to be decoded by the video decoder (410); in the same or another embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the other encoded video sequences. The encoded video sequence may be received from a channel (401), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (431) may receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that may be forwarded to their respective consuming entities (not shown). The receiver (431) may separate the encoded video sequence from the other data. To prevent network jitter, a buffer memory (415) may be coupled between the receiver (431) and the entropy decoder / parser (420) (hereinafter referred to as "parser (420)"). In some applications, the buffer memory (415) is part of the video decoder (410). In other cases, the buffer memory (415) may be provided external to the video decoder (410) (not shown). In other cases, a buffer memory (not shown) is provided outside the video decoder (410) to prevent network jitter, for example, and another buffer memory (415) may be configured inside the video decoder (410) to handle broadcast timing, for example. When the receiver (431) receives data from a storage / forward device with sufficient bandwidth and controllability or from an isochronous network, it may not be necessary to configure the buffer memory (415), or the buffer memory may be made smaller. Of course, in order to use on a service packet network such as the Internet, a buffer memory (415) may also be required, and the buffer memory may be relatively large and have an adaptive size, and may be at least partially implemented in an operating system or a similar element (not shown) outside the video decoder (410).

[0056] The video decoder (410) may include a parser (420) to reconstruct symbols (421) from the encoded video sequence. The types of symbols include information used to manage the operation of the video decoder (410) and potential information used to control a display device such as a display device (412) (e.g., a display screen) that is not part of the electronic device (430) but can be coupled to the electronic device (430), such as Figure 4As shown in . The control information for the display device may be a parameter set fragment (not indicated) of Supplemental Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (420) may parse / entropy decode the received coded video sequence. The encoding of the coded video sequence may be performed according to a video coding technique or standard, and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, and the like. The parser (420) may extract a subgroup parameter set of at least one subgroup of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. The subgroup may include a Group of Pictures (GOP), a picture, a tile, a slice, a macroblock, a Coding Unit (CU), a block, a Transform Unit (TU), a Prediction Unit (PU), and the like. The parser (420) may also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, and so on.

[0057] The parser (420) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (415), thereby creating symbols (421).

[0058] Depending on the type of coded video picture or portion of coded video picture (e.g., inter- and intra-pictures, inter- and intra-blocks) and other factors, the reconstruction of the symbol (421) may involve multiple different units. Which units are involved and how they are involved may be controlled by subgroup control information parsed by the parser (420) from the coded video sequence. For the sake of brevity, such subgroup control information flow between the parser (420) and the multiple units below is not described.

[0059] In addition to the functional blocks already mentioned, the video decoder (410) can be conceptually subdivided into several functional units as described below. In a practical embodiment operating under commercial constraints, many of these units interact closely with each other and can be integrated with each other. However, for the purpose of describing the disclosed subject matter, the conceptual subdivision into the following functional units is appropriate.

[0060] The first unit is a sealer / inverse transform unit (451). The sealer / inverse transform unit (451) receives quantized transform coefficients as symbols (421) from the parser (420) and control information, including which transform method to use, block size, quantization factor, quantization scaling matrix, etc. The sealer / inverse transform unit (451) can output a block including sample values, which can be input into an aggregator (455).

[0061] In some cases, the output samples of the scaler / inverse transform unit (451) may belong to an intra-coded block; that is, a block that does not use predictive information from a previously reconstructed picture, but may use predictive information from a previously reconstructed portion of the current picture. Such predictive information may be provided by an intra-picture prediction unit (452). In some cases, the intra-picture prediction unit (452) generates surrounding blocks of the same size and shape as the block being reconstructed using reconstructed information extracted from a current picture buffer (458). For example, the current picture buffer (458) buffers a partially reconstructed current picture and / or a fully reconstructed current picture. In some cases, the aggregator (455) adds the prediction information generated by the intra-prediction unit (452) to the output sample information provided by the scaler / inverse transform unit (451) on a per-sample basis.

[0062] In other cases, the output samples of the scaler / inverse transform unit (451) may belong to an inter-frame coded and potentially motion compensated block. In this case, the motion compensated prediction unit (453) may access the reference picture memory (457) to extract samples for prediction. After the extracted samples are motion compensated according to the symbols (421), these samples may be added to the output of the scaler / inverse transform unit (451) (in this case referred to as residual samples or residual signals) by the aggregator (455) to generate output sample information. The acquisition of the predicted samples by the motion compensated prediction unit (453) from the address in the reference picture memory (457) may be controlled by a motion vector, and the motion vector is provided to the motion compensated prediction unit (453) in the form of the symbols (421), for example, including X, Y and reference picture components. Motion compensation may also include interpolation of sample values ​​extracted from the reference picture memory (457) when using sub-sample accurate motion vectors, motion vector prediction mechanisms, and the like.

[0063] The output samples of the aggregator (455) may be employed by various loop filtering techniques in a loop filter unit (456). The video compression techniques may include in-loop filter techniques controlled by parameters included in the encoded video sequence (also referred to as the encoded video bitstream) and available to the loop filter unit (456) as symbols (421) from the parser (420). However, in other embodiments, the video compression techniques may also be responsive to meta-information obtained during decoding of a previous (in decoding order) portion of an encoded picture or encoded video sequence, as well as to previously reconstructed and loop filtered sample values.

[0064] The output of the loop filter unit (456) may be a sample stream that may be output to a display device (412) and stored in a reference picture memory (457) for subsequent inter-picture prediction.

[0065] Once fully reconstructed, certain coded pictures may be used as reference pictures for future prediction. For example, once the coded picture corresponding to the current picture is fully reconstructed, and the coded picture is identified as a reference picture (e.g., by the parser (420)), the current picture buffer (458) may become part of the reference picture memory (457), and a new current picture buffer may be reallocated before starting to reconstruct a subsequent coded picture.

[0066] The video decoder (410) may perform decoding operations according to a predetermined video compression technique, such as in the ITU-T H.265 standard. The encoded video sequence may conform to the syntax specified by the video compression technique or standard used in the sense that the encoded video sequence follows the syntax of the video compression technique or standard and the profile recorded in the video compression technique or standard. Specifically, the profile may select certain tools from all the tools available in the video compression technique or standard as the only tools available for use under the profile. For compliance, the complexity of the encoded video sequence is also required to be within the range defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum picture size, the maximum frame rate, the maximum reconstruction sampling rate (measured in, for example, mega samples per second), the maximum reference picture size, etc. In some cases, the limits set by the hierarchy may be further limited by the Hypothetical Reference Decoder (HRD) specification and metadata of the HRD buffer management signaled in the encoded video sequence.

[0067] In an embodiment, the receiver (431) may receive additional (redundant) data along with the encoded video. The additional data may be part of the encoded video sequence. The additional data may be used by the video decoder (410) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0068] Figure 5 1 is a block diagram of a video encoder (503) according to an embodiment disclosed in the present application. The video encoder (503) is provided in an electronic device (520). The electronic device (520) includes a transmitter (540) (e.g., a transmission circuit). The video encoder (503) can be used to replace Figure 3 A video encoder (303) in an embodiment.

[0069] The video encoder (503) can be used to obtain the video source (501) (not Figure 5 In another embodiment, the video source (501) is a part of the electronic device (520) to receive video samples, and the video source can collect video images to be encoded by the video encoder (503). In another embodiment, the video source (501) is a part of the electronic device (520).

[0070] The video source (501) may provide a source video sequence in the form of a digital video sample stream to be encoded by the video encoder (503), wherein the digital video sample stream may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source (501) may be a storage device storing previously prepared videos. In a video conferencing system, the video source (501) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of separate pictures that are given motion when viewed sequentially. The pictures themselves may be constructed as a spatial pixel array, wherein each pixel may include one or more samples depending on the sampling structure, color space, etc. used. The relationship between pixels and samples may be easily understood by those skilled in the art. The following description focuses on the samples.

[0071] According to an embodiment, the video encoder (503) may encode and compress pictures of a source video sequence into an encoded video sequence (543) in real time or under any other time constraints required by the application. Implementing an appropriate encoding speed is a function of the controller (550). In some embodiments, the controller (550) controls other functional units as described below and is functionally coupled to these units. For the sake of simplicity, couplings are not shown in the figure. The parameters set by the controller (550) may include rate control related parameters (picture skipping, quantizer, lambda value of rate distortion optimization technology, etc.), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (550) can be used to have other suitable functions that are related to the video encoder (503) optimized for a certain system design.

[0072] In some embodiments, the video encoder (503) operates in a coding loop. As a simple description, in embodiments, the coding loop may include a source encoder (530) (e.g., responsible for creating symbols, such as a symbol stream, based on an input picture to be encoded and a reference picture) and a (local) decoder (533) embedded in the video encoder (503). The decoder (533) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data (because in the video compression technology considered in this application, any compression between the symbols and the encoded video code stream is lossless). The reconstructed sample stream (sample data) is input to a reference picture memory (534). Since the decoding of the symbol stream produces bit-accurate results that are independent of the decoder location (local or remote), the contents of the reference picture memory (534) are also bit-accurate between the local encoder and the remote encoder. In other words, the reference picture samples "seen" by the prediction part of the encoder are exactly the same as the sample values ​​that the decoder will "see" when using the prediction during decoding. This basic principle of reference picture synchronization (and the drift that occurs when synchronization cannot be maintained, eg due to channel errors) is also used in some related techniques.

[0073] The operation of the "local" decoder (533) may be combined with, for example, Figure 4 The "remote" decoder described in detail for the video decoder (410) is identical. However, additional brief reference is made to Figure 4 , when symbols are available and the entropy encoder (545) and parser (420) are capable of losslessly encoding / decoding the symbols into an encoded video sequence, the entropy decoding portion of the video decoder (410), including the buffer memory (415) and the parser (420), may not be fully implemented in the local decoder (533).

[0074] At this point, it can be observed that any decoder technology other than the parsing / entropy decoding present in the decoder must also be present in the corresponding encoder in substantially the same functional form. For this reason, the present application focuses on decoder operation. The description of encoder technology can be simplified because encoder technology is mutually inverse to the decoder technology described comprehensively. A more detailed description is only needed in certain areas and is provided below.

[0075] During operation, in some embodiments, the source encoder (530) may perform motion compensated predictive coding. The motion compensated predictive coding predictively encodes an input picture with reference to one or more previously encoded pictures from a video sequence designated as "reference pictures." In this manner, the encoding engine (532) encodes the differences between pixel blocks of the input picture and pixel blocks of a reference picture that may be selected as a prediction reference for the input picture.

[0076] The local video decoder (533) may decode the encoded video data of the picture that may be designated as the reference picture based on the symbol created by the source encoder (530). The operation of the encoding engine (532) may be a lossy process. When the encoded video data is available at the video decoder ( Figure 5 When the video sequence is decoded at a remote location (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (533) replicates the decoding process that may be performed by the video decoder on the reference picture and may cause the reconstructed reference picture to be stored in the reference picture cache (534). In this way, the video encoder (503) may locally store a copy of the reconstructed reference picture that has common content (absent transmission errors) with the reconstructed reference picture to be obtained by the remote video decoder.

[0077] The predictor (535) may perform a prediction search for the encoding engine (532). That is, for a new picture to be encoded, the predictor (535) may search the reference picture memory (534) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference picture motion vectors, block shapes, etc., that may serve as appropriate prediction references for the new picture. The predictor (535) may operate pixel-by-pixel based on sample blocks to find a suitable prediction reference. In some cases, based on the search results obtained by the predictor (535), it may be determined that the input picture may have prediction references taken from a plurality of reference pictures stored in the reference picture memory (534).

[0078] The controller (550) may manage encoding operations of the source encoder (530), including, for example, setting parameters and subgroup parameters for encoding video data.

[0079] The outputs of all the above functional units may be entropy encoded in an entropy encoder (545). The entropy encoder (545) performs lossless compression on the symbols generated by the various functional units according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc., thereby converting the symbols into a coded video sequence.

[0080] The transmitter (540) may buffer the encoded video sequence created by the entropy encoder (545) in preparation for transmission over a communication channel (560), which may be a hardware / software link to a storage device where the encoded video data will be stored. The transmitter (540) may combine the encoded video data from the video encoder (503) with other data to be transmitted, such as encoded audio data and / or ancillary data streams (source not shown).

[0081] The controller (550) may manage the operation of the video encoder (503). During encoding, the controller (550) may assign a certain coded picture type to each coded picture, but this may affect the coding techniques that can be applied to the corresponding picture. For example, a picture may generally be assigned to any of the following picture types:

[0082] An intra picture (I picture) may be a picture that can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh ("IDR") pictures. Those skilled in the art are aware of the variations of I pictures and their corresponding applications and features.

[0083] A predictive picture (P picture) may be a picture that can be encoded and decoded using intra prediction or inter prediction, which uses at most one motion vector and a reference index to predict sample values ​​for each block.

[0084] Bidirectional predictive pictures (B pictures), which can be pictures that can be encoded and decoded using intra prediction or inter prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for reconstruction of a single block.

[0085] The source picture may typically be spatially subdivided into blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples), and coded block by block. These blocks may be predictively coded with reference to other (already coded) blocks, which are determined according to the coding allocation applied to the block's corresponding picture. For example, blocks of an I picture may be non-predictively coded, or the blocks may be predictively coded (spatial prediction or intra prediction) with reference to already coded blocks of the same picture. Blocks of pixels of a P picture may be predictively coded by spatial prediction with reference to one previously coded reference picture or by temporal prediction. Blocks of a B picture may be predictively coded by spatial prediction with reference to one or two previously coded reference pictures or by temporal prediction.

[0086] The video encoder (503) may perform encoding operations according to a predetermined video encoding technique or standard, such as ITU-T H.265 Recommendation. In operation, the video encoder (503) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in an input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard used.

[0087] In an embodiment, the transmitter (540) may transmit additional data when transmitting the encoded video. The source encoder (530) may include such data as part of the encoded video sequence. The additional data may include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0088] The captured video may be taken as a plurality of source pictures (video pictures) in a temporal sequence. Intra-picture prediction (often simplified to intra-prediction) exploits spatial correlations in a given picture, while inter-picture prediction exploits (temporal or other) correlations between pictures. In an embodiment, a particular picture being encoded / decoded is divided into blocks, and the particular picture being encoded / decoded is referred to as the current picture. When a block in the current picture is similar to a reference block in a reference picture that was previously encoded in the video and is still buffered, the block in the current picture may be encoded by a vector called a motion vector. The motion vector points to a reference block in a reference picture, and in the case where multiple reference pictures are used, the motion vector may have a third dimension that identifies the reference picture.

[0089] In some embodiments, bidirectional prediction techniques may be used in inter-picture prediction. According to the bidirectional prediction technique, two reference pictures are used, for example, a first reference picture and a second reference picture that are both before the current picture in the video in decoding order (but may be in the past and future in display order, respectively). A block in the current picture may be encoded by a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. Specifically, the block may be predicted by a combination of the first reference block and the second reference block.

[0090] In addition, merge mode technology can be used in inter-picture prediction to improve coding efficiency.

[0091] According to some embodiments disclosed in the present application, predictions such as inter-picture prediction and intra-picture prediction are performed in blocks. For example, according to the HEVC standard, pictures in a video picture sequence are divided into coding tree units (CTUs) for compression, and the CTUs in the pictures have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. In general, a CTU includes three coding tree blocks (CTBs), which are a luminance CTB and two chrominance CTBs. Furthermore, each CTU can be split into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be split into a 64×64 pixel CU, or 4 32×32 pixel CUs, or 16 16×16 pixel CUs. In an embodiment, each CU is analyzed to determine the prediction type for the CU, such as an inter-prediction type or an intra-prediction type. In addition, depending on temporal and / or spatial predictability, the CU is split into one or more prediction units (PUs). Typically, each PU includes a luminance prediction block (PB) and two chrominance PBs. In an embodiment, the prediction operation in encoding (encoding / decoding) is performed in units of prediction blocks. Taking the luminance prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values ​​(e.g., luminance values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, and the like.

[0092] Figure 6 1 is a diagram of a video encoder (603) according to another embodiment disclosed in the present application. The video encoder (603) is used to receive a processed block (e.g., a prediction block) of sample values ​​in a current video picture in a video picture sequence, and encode the processed block into an encoded picture that is part of an encoded video sequence. In this embodiment, the video encoder (603) is used to replace Figure 3A video encoder (303) in an embodiment.

[0093] In an HEVC embodiment, a video encoder (603) receives a matrix of sample values ​​for a processing block, such as a prediction block of 8×8 samples. The video encoder (603) uses, for example, rate-distortion (RD) optimization to determine whether to use intra mode, inter mode, or bidirectional prediction mode to encode the processing block. When encoding the processing block in intra mode, the video encoder (603) may use intra prediction techniques to encode the processing block into an encoded picture; and when encoding the processing block in inter mode or bidirectional prediction mode, the video encoder (603) may use inter prediction or bidirectional prediction techniques to encode the processing block into an encoded picture, respectively. In some video encoding techniques, the merge mode may be an inter-picture prediction submode, in which a motion vector is derived from one or more motion vector predictors without the aid of an encoded motion vector component external to the predictor. In some other video encoding techniques, there may be a motion vector component applicable to the subject block. In an embodiment, the video encoder (603) includes other components, such as a mode decision module (not shown) for determining the processing block mode.

[0094] exist Figure 6 In an embodiment of the present invention, the video encoder (603) includes: Figure 6 Shown are an inter-frame encoder (630), an intra-frame encoder (622), a residual calculator (623), a switch (626), a residual encoder (624), a common controller (621), and an entropy encoder (625) coupled together.

[0095] The inter-frame encoder (630) is used to receive samples of a current block (e.g., a processing block), compare the block with one or more reference blocks in a reference picture (e.g., blocks in a previous picture and a subsequent picture), generate inter-frame prediction information (e.g., redundant information description according to an inter-frame coding technique, motion vectors, merge mode information), and calculate an inter-frame prediction result (e.g., a predicted block) based on the inter-frame prediction information using any suitable technique. In some embodiments, the reference picture is a decoded reference picture decoded based on the encoded video information.

[0096] The intra-frame encoder (622) is used to receive samples of a current block (e.g., a processing block), compare the block with an encoded block in the same picture in some cases, generate quantization coefficients after transformation, and in some cases also generate intra-frame prediction information (e.g., intra-frame prediction direction information according to one or more intra-frame coding techniques). In an embodiment, the intra-frame encoder (622) also calculates an intra-frame prediction result (e.g., a predicted block) based on the intra-frame prediction information and a reference block in the same picture.

[0097] The general controller (621) is used to determine the general control data and control other components of the video encoder (603) based on the general control data. In an embodiment, the general controller (621) determines the mode of the block and provides a control signal to the switch (626) based on the mode. For example, when the mode is intra mode, the general controller (621) controls the switch (626) to select the intra mode result for use by the residual calculator (623) and controls the entropy encoder (625) to select the intra prediction information and add the intra prediction information to the bitstream; and when the mode is inter mode, the general controller (621) controls the switch (626) to select the inter prediction result for use by the residual calculator (623) and controls the entropy encoder (625) to select the inter prediction information and add the inter prediction information to the bitstream.

[0098] The residual calculator (623) is used to calculate the difference (residual data) between the received block and the prediction result selected from the intra-frame encoder (622) or the inter-frame encoder (630). The residual encoder (624) is used to operate based on the residual data to encode the residual data to generate a transform coefficient. In an embodiment, the residual encoder (624) is used to convert the residual data from the time domain to the frequency domain and generate a transform coefficient. The transform coefficient is then processed by quantization to obtain a quantized transform coefficient. In various embodiments, the video encoder (603) also includes a residual decoder (628). The residual decoder (628) is used to perform an inverse transform and generate decoded residual data. The decoded residual data can be used appropriately by the intra-frame encoder (622) and the inter-frame encoder (630). For example, the inter-frame encoder (630) can generate a decoded block based on the decoded residual data and the inter-frame prediction information, and the intra-frame encoder (622) can generate a decoded block based on the decoded residual data and the intra-frame prediction information. The decoded blocks are appropriately processed to generate a decoded picture, and in some embodiments, the decoded picture may be buffered in a memory circuit (not shown) and used as a reference picture.

[0099] The entropy encoder (625) is used to format the code stream to produce an encoded block. The entropy encoder (625) generates various information according to a suitable standard such as the HEVC standard. In an embodiment, the entropy encoder (625) is used to obtain general control data, selected prediction information (such as intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information in the code stream. It should be noted that according to the disclosed subject matter, when the block is encoded in the inter-frame mode or the merge sub-mode of the bidirectional prediction mode, there is no residual information.

[0100] Figure 7FIG. 7 is a diagram of a video decoder (710) according to another embodiment disclosed in the present application. The video decoder (710) is used to receive an encoded image as part of an encoded video sequence and decode the encoded image to generate a reconstructed picture. In an embodiment, the video decoder (710) is used to replace Figure 3 A video decoder (310) of an embodiment.

[0101] exist Figure 7 In one embodiment, the video decoder (710) includes: Figure 7 An entropy decoder (771), an inter-frame decoder (780), a residual decoder (773), a reconstruction module (774) and an intra-frame decoder (772) coupled together are shown in FIG.

[0102] The entropy decoder (771) may be used to reconstruct certain symbols from the encoded picture, which represent syntax elements constituting the encoded picture. Such symbols may include, for example, a mode for encoding the block (e.g., intra mode, inter mode, bidirectional prediction mode, a merged sub-mode of the latter two, or another sub-mode), prediction information (e.g., intra prediction information or inter prediction information) that may identify certain samples or metadata for prediction by the intra decoder (772) or the inter decoder (780), respectively, residual information in the form of, for example, quantized transform coefficients, and the like. In an embodiment, when the prediction mode is inter or bidirectional prediction mode, the inter prediction information is provided to the inter decoder (780); and when the prediction type is an intra prediction type, the intra prediction information is provided to the intra decoder (772). The residual information may be inverse quantized and provided to the residual decoder (773).

[0103] The inter-frame decoder (780) is used for receiving the inter-frame prediction information and generating an inter-frame prediction result based on the inter-frame prediction information.

[0104] The intra-frame decoder (772) is used to receive intra-frame prediction information and generate a prediction result based on the intra-frame prediction information.

[0105] The residual decoder (773) is used to perform inverse quantization to extract dequantized transform coefficients, and process the dequantized transform coefficients to convert the residual from the frequency domain to the spatial domain. The residual decoder (773) may also require some control information (to obtain the quantizer parameter QP), and this information can be provided by the entropy decoder (771) (the data path is not indicated because this is only low-volume control information).

[0106] The reconstruction module (774) is used to combine the residual output by the residual decoder (773) with the prediction result (which can be output by the inter-frame prediction module or the intra-frame prediction module) in the spatial domain to form a reconstructed block, which can be part of a reconstructed picture, which in turn can be part of a reconstructed video. It should be noted that other suitable operations such as deblocking operations can be performed to improve visual quality.

[0107] It should be noted that the video encoder (303), video encoder (503) and video encoder (603) and video decoder (310), video decoder (410) and video decoder (710) may be implemented using any suitable technology. In an embodiment, the video encoder (303), video encoder (503) and video encoder (603) and video decoder (310), video decoder (410) and video decoder (710) may be implemented using one or more integrated circuits. In another embodiment, the video encoder (303), video encoder (503) and video encoder (603) and video decoder (310), video decoder (410) and video decoder (710) may be implemented using one or more processors that execute software instructions.

[0108] II. Image Format and Image Partition

[0109] II.1 Source, decoded and output picture formats

[0110] In some embodiments, the relationship between source and decoded pictures is given by the codestream as follows. The video source represented by the codestream is a sequence of pictures in decoding order. The source picture and the decoded picture each include one or more sample arrays:

[0111] - Luminance (Y) only (monochrome).

[0112] - Luma and two chrominances (YCbCr or YCgCo).

[0113] - Green, Blue and Red (GBR, also known as RGB).

[0114] - An array representing otherwise unspecified monochrome or tri-stimulus color sampling (e.g., YZX, also called XYZ).

[0115] For ease of notation and terminology in this disclosure, the variables and terminology associated with these arrays are referred to as luminance (or L or Y) and chrominance, where the two chrominance arrays are referred to as Cb and Cr; regardless of the actual color representation method in use. The actual color representation method in use can be indicated by the syntax.

[0116] The variables SubWidthC and SubHeightC indicate the chroma component subsampling ratios and are specified in Table 1 according to the chroma format sampling structure, which is specified by chroma_format_idc and separate_color_plane_flag.

[0117] Table 1

[0118] chroma_format_idc separate_color_plane_flag Chroma Format SubWidthC SubHeightC 0 0 monochrome 1 1 1 0 4:2:0 2 2 2 0 4:2:2 2 1 3 0 4:4:4 1 1 3 1 4:4:4 1 1

[0119] In monochrome sampling, there is only one sample array, which is nominally considered to be the luma array. In 4:2:0 sampling, each of the two chroma arrays has half the height and half the width of the luma array. In 4:2:2 sampling, each of the two chroma arrays has the same height and half the width of the luma array. In 4:4:4 sampling, depending on the value of separate_color_plane_flag, the following applies:

[0120] - If separate_color_plane_flag is equal to 0, each of the two chroma arrays has the same height and width as the luma array.

[0121] Otherwise (separate_color_plane_flag is equal to 1), the three color planes are processed separately as monochrome sampled pictures.

[0122] The number of bits required to represent each of the samples in the luma and chroma arrays in the video sequence may be, for example, in the range of 8 to 16 (inclusive), and the number of bits used in the luma array may be different from the number of bits used in the chroma array. Figure 8 The nominal vertical and horizontal relative positions of luma and chroma samples in a picture are shown in . Optional chroma sample relative positions may be indicated in the video usability information.

[0123] When the value of chroma_format_idc is equal to 2, the chroma samples are located together with the corresponding luma samples, and the nominal positions in the picture are as follows: Fig. 9 When the value of chroma_format_idc is equal to 3, for all cases of the picture, all array samples are located together and the nominal positions in the picture are as follows Fig.10 shown.

[0124] II.2 Picture Partitioning

[0125] II.2.1 Partitioning a picture into CTUs

[0126] In some embodiments, a picture is divided into a sequence of Coding Tree Units (CTUs). The CTU concept is similar to HEVC. For a picture with three sample arrays, a CTU consists of an N×N block of luma samples and two corresponding blocks of chroma samples. Fig.11 An example of a picture divided into CTUs is shown. In the example, the maximum allowed size of a luma block in a CTU is specified as 128×128. In the example, the maximum size of a luma transform block is 64×64)

[0127] II.2.2 Partitioning images into strips, tiles, and bricks

[0128] In some embodiments, a picture may be divided into one or more tile rows and one or more tile columns. A tile is a sequence of CTUs covering a rectangular area of ​​a picture. A tile is divided into one or more bricks, each brick consisting of multiple CTU rows within a tile. Tiles that are not partitioned into multiple bricks are also referred to as bricks. However, bricks that are true subsets of tiles are not referred to as tiles. A slice either contains multiple tiles of a picture or multiple bricks of a tile. Two strip modes are supported, namely a raster scan strip mode and a rectangular strip mode. In the raster scan strip mode, a strip contains a sequence of strips in a raster scan of tiles of a picture. In the rectangular strip mode, a strip contains several bricks of a picture, and the several bricks together form a rectangular area of ​​a picture. The bricks within a rectangular strip are in the order of a raster scan of the bricks of the strip.

[0129] Fig.12 An example of raster scan strip partitioning of a picture is shown, where the picture is divided into 12 tiles and 3 raster scan strips. Fig.13 An example of rectangular strip partitioning is shown, where a picture is divided into 24 tiles (6 tile columns and 4 tile rows) and 9 rectangular strips. Fig.14 An example of a picture partitioned into tiles, bricks, and rectangular strips is shown, where the picture is divided into 4 tiles (2 tile columns and 2 tile rows), 11 bricks (the upper left tile contains 1 brick, the upper right tile contains 5 bricks, the lower left tile contains 2 bricks, and the lower right tile contains 3 bricks), and 4 rectangular strips.

[0130] II.2.3 Partitioning CTU using tree structure

[0131] In some embodiments, the CTU is partitioned into CUs using a quadtree structure represented as a coding tree to accommodate various local characteristics. The decision to encode a picture area using inter-picture (temporal) or intra-picture (spatial) prediction is made at the leaf-CU level. Each leaf-CU can be further partitioned into one, two, or four PUs according to the PU partition type. Within a PU, the same prediction process is applied, and relevant information is sent to the decoder on a PU basis. After obtaining the residual block by applying a prediction process based on the PU partition type, the leaf CU can be partitioned into transform units (TUs) according to another quadtree structure similar to the coding tree of the CU. One of the key features of the HEVC structure is that it has multiple partition concepts including CU, PU, ​​and TU.

[0132] In some embodiments, a quadtree with nested multi-type trees replaces the concept of multiple partition unit types, and the nested multi-type trees use binary and ternary segmentation structures, that is, it eliminates the separation of CU, PU, ​​and TU concepts unless the size of the CU is too large for the maximum transform length, and supports more flexibility in the CU partition shape. In the coding tree structure, the CU can have a square shape or a rectangular shape. The coding tree unit (CTU) is first partitioned by a quaternary tree (also known as a quadtree) structure. Then, the quadtree leaf nodes can be further divided by a multi-type tree structure.

[0133] like Fig.15 As shown, there are four types of splits in the multi-type tree structure: vertical binary split (SPLIT_BT_VER), horizontal binary split (SPLIT_BT_HOR), vertical ternary split (SPLIT_TT_VER), and horizontal ternary split (SPLIT_TT_HOR). The multi-type tree leaf nodes are called coding units (CU), and unless the CU is too large for the maximum transform length, the segmentation is used for prediction and transform processing without any further partitioning. This means that in most cases, CU, PU, ​​and TU have the same block size in a quadtree with a nested multi-type tree coding block structure. An exception occurs when the maximum supported transform length is less than the width or height of the color component of the CU.

[0134] Fig.16The signaling mechanism of partition split information in a quadtree with a nested multi-type tree coding tree structure is shown. The coding tree unit (CTU) is regarded as the root of the quadtree and is first partitioned by the quadtree structure. Each quadtree leaf node (when large enough to allow partitioning) is then further partitioned by the multi-type tree structure. In the multi-type tree structure, a first flag (mtt_split_cu_flag) is signaled to indicate whether the node is further partitioned; when the node is further partitioned, a second flag (mtt_split_cu_vertical_flag) is signaled to indicate the split direction, and then a third flag (mtt_split_cu_binary_flag) is signaled to indicate whether the split is a binary split or a ternary split. Based on the values ​​of mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-type tree split mode (MttSplitMode, multi-type tree slitting mode) of the CU is derived, as shown in Table 2.

[0135] Table 2

[0136] MttSplitMode mtt_split_cu_vertical_flag mtt_split_cu_binary_flag SPLIT_TT_HOR 0 0 SPLIT_BT_HOR 0 1 SPLIT_TT_VER 1 0 SPLIT_BT_VER 1 1

[0137] Fig.17 A CTU divided into multiple CUs is shown, the CU having a quadtree and nested multi-type tree coding block structure, where the bold block edges represent quadtree partitions and the remaining edges represent multi-type tree partitions. The quadtree with nested multi-type tree partitions provides a content-adaptive coding tree structure including the CU. The size of the CU can be as large as the CTU, or as small as 4×4 in units of luma samples. For the case of 4:2:0 chroma format, the maximum chroma CB size is 64×64 and the minimum chroma CB size is 2×2.

[0138] In some embodiments, the maximum supported luma transform size is 64x64, and the maximum supported chroma transform size is 32x32. When the width or height of a CB is larger than the maximum transform width or height, the CB is automatically split in the horizontal and / or vertical direction to satisfy the transform size limit in that direction.

[0139] In some embodiments, the following parameters are defined and specified by a syntax parameter set (SPS) syntax element for a quadtree with nested multi-type tree coding tree scheme.

[0140] -CTU size: the root node size of the quadtree

[0141] -MinQTSize: Minimum allowed quadtree leaf node size

[0142] -MaxBtSize: Maximum allowed binary tree root node size

[0143] -MaxTtSize: Maximum allowed ternary tree root node size

[0144] -MaxMttDepth: Maximum allowed hierarchical depth of a multi-type tree split from the leaves of a quadtree

[0145] -MinBtSize: minimum allowed binary tree leaf node size

[0146] -MinTtSize: minimum allowed ternary tree leaf node size

[0147] In one example of a quadtree with a nested multi-type tree coding tree structure, the CTU size is set to 128×128 luma samples with two corresponding 64×64 blocks of 4:2:0 chroma samples, MinQTSize is set to 16×16, MaxBtSize is set to 128×128, MaxTtSize is set to 64×64, MinBtSize and MinTtSize (for both width and height) are set to 4×4, and MaxMttDepth is set to 4. Quadtree partitioning is first applied to the CTU to generate quadtree leaf nodes. The quadtree leaf nodes can have sizes from 16×16 (i.e., MinQTSize) to 128×128 (i.e., CTU size). If the leaf QT node is 128×128, it will not be further split by the binary tree because its size exceeds MaxBtSize and MaxTtSize (i.e., 64×64). Otherwise, the leaf quadtree nodes can be further divided by multi-type trees.

[0148] Therefore, the quadtree leaf node is also the root node of the multi-type tree, and it has a multi-type tree depth (mttDepth) of 0. When the multi-type tree depth reaches MaxMttDepth (i.e., 4), no further splitting is considered. When the multi-type tree node has a width equal to MinBtSize and less than or equal to 2*MinTtSize, no further horizontal splitting is considered. Similarly, when the multi-type tree node has a height equal to MinBtSize and less than or equal to 2*MinTtSize, no further vertical splitting is considered.

[0149] In some embodiments, to allow 64×64 luma blocks and 32×32 chroma pipeline designs in some hardware decoders, TT splitting is disabled when the width or height of the luma coding block is greater than 64, e.g. Fig.18 When the width or height of the chroma coding block is greater than 32, TT splitting is also prohibited.

[0150] In some embodiments, the coding tree scheme supports the ability for luma and chroma to have separate block tree structures. For example, for P and B slices, the luma and chroma CTBs in one CTU can share the same coding tree structure. However, for I slices, luma and chroma can have separate block tree structures. When the separated block tree mode is applied, the luma CTB is partitioned into CUs through one coding tree structure, and the chroma CTB is partitioned into chroma CUs through another coding tree structure. This means that a CU in an I slice can consist of coding blocks of a luma component or coding blocks of two chroma components, and a CU in a P slice or a B slice always consists of coding blocks of all three color components unless the video is monochrome.

[0151] II.2.4 CU Split on Picture Boundaries

[0152] In some embodiments, when a portion of a tree node block exceeds the bottom or right picture boundary, the tree node block is forced to be split until all samples of each coded CU are inside the picture boundary. In some embodiments, the following splitting rules are applied.

[0153] If part of a tree node block exceeds the bottom and right picture boundaries: (i) if the block is a QT node and the size of the block is greater than the minimum QT size, the block is forced to be split in QT split mode; otherwise, the block is forced to be split in SPLIT_BT_HOR mode.

[0154] (ii) Otherwise, if part of the tree node block exceeds the bottom picture boundary,

[0155] - If the block is a QT node, and the size of the block is greater than the minimum QT size, and the size of the block is greater than the maximum BT size, then the block is forced to be split in QT split mode.

[0156] - Otherwise, if the block is a QT node, and the size of the block is greater than the minimum QT size, and the size of the block is less than or equal to the maximum BT size, then the block is forced to be split in QT split mode or SPLIT_BT_HOR mode.

[0157] -Otherwise (the block is a BTT node or the block size is less than or equal to the minimum QT size), the block is forced to be split in SPLIT_BT_HOR mode.

[0158] (iii) Otherwise, if part of the tree node block exceeds the right picture boundary,

[0159] - If the block is a QT node, and the size of the block is greater than the minimum QT size, and the size of the block is greater than the maximum BT size, then the block is forced to be split in QT split mode.

[0160] - Otherwise, if the block is a QT node, and the size of the block is greater than the minimum QT size, and the size of the block is less than or equal to the maximum BT size, then the block is forced to be split in QT split mode or SPLIT_BT_VER mode.

[0161] -Otherwise (the block is a BTT node or the block size is less than or equal to the minimum QT size), the block is forced to be split in SPLIT_BT_VER mode.

[0162] II.2.5 Restrictions on redundant CU partitioning

[0163] The quadtree with nested multi-type tree coding block structure provides a highly flexible block partition structure. Due to the support of multiple types of tree segmentation types, different segmentation modes may result in the same coding block structure. In some embodiments, some of these redundant segmentation modes are disabled.

[0164] Fig.19 Redundant partitioning patterns for binary and ternary tree partitioning are illustrated. As shown, two levels of consecutive binary partitioning in one direction can have the same coding block structure as ternary tree partitioning, which is followed by binary tree partitioning of the central partition. In this case, binary tree partitioning (in a given direction) for the central partition of the ternary tree partition is blocked by syntax. This restriction applies to CUs in all pictures.

[0165] When segmentation is prohibited as described above, the signaling of the corresponding syntax elements is modified to take into account the prohibited situation. Fig.19 When any of the cases is identified (ie, binary partitioning is prohibited for the central partitioned CU), the syntax element mtt_split_cu_binary_flag specifying whether the partition is binary or ternary is not signaled and is inferred by the decoder to be equal to 0.

[0166] II.2.6 Virtual pipeline data units (VPDU)

[0167] In some embodiments, a virtual pipeline data unit (VPDU) is used. A VPDU is defined as a non-overlapping unit in a picture. In a hardware decoder, multiple pipeline stages process consecutive VPDUs simultaneously. The VPDU size is roughly proportional to the buffer size in most pipeline stages, so it is important to keep the VPDU size small. In some hardware decoders, the VPDU size can be set to the maximum transform block (TB) size. However, in some cases, ternary tree (TT) and binary tree (BT) partitioning can lead to an increase in the VPDU size.

[0168] To keep the VPDU size to 64×64 luma samples, the following normative partitioning restrictions (with syntax signaling modifications) apply in some examples, e.g. Fig. 20 As shown: TT partition is not allowed for CUs with width or height or both width and height equal to 128. For 128xN CUs with N≤64 (i.e., width equal to 128 and height less than 128), horizontal BT is not allowed. For Nx128 CUs with N≤64 (i.e., height equal to 128 and width less than 128), vertical BT is not allowed.

[0169] II.2.7 Intra-frame chroma partitioning and prediction restrictions

[0170] Compared with the luma coding tree, since the dual tree in the intra picture allows different partitions to be applied in the chroma coding tree, the dual tree introduces a longer coding pipeline, and the QTBT MinQTSizeC value range and MinBtSizeY and MinTTSizeY in the chroma tree allow small chroma blocks such as 2×2, 4×2 and 2×4. This leads to difficulties in decoder design. In addition, several prediction modes such as CCLM, plane and angular modes require multiplication operations. In order to alleviate the above-mentioned problems, in some embodiments, the small chroma block size (2×2 / 2×4 / 4×2) is limited in the dual tree as a partition limit.

[0171] In typical hardware video encoders and decoders, the processing throughput drops when a picture has smaller intra blocks due to the sample processing data dependency between adjacent intra blocks. The predictor generation for an intra block requires upper and left boundary reconstruction samples from adjacent blocks. Therefore, intra prediction must be processed sequentially, block by block.

[0172] In some codecs, the smallest intra CU is 8×8 luma samples. The luma component of the smallest intra CU can be further split into four 4×4 luma intra prediction units (PUs), but the chroma component of the smallest intra CU cannot be further split. Therefore, the worst-case hardware processing throughput occurs when processing 4×4 chroma intra blocks or 4×4 luma intra blocks.

[0173] In some embodiments, in order to improve the worst-case throughput, chroma intra CBs smaller than 16 chroma samples are disabled by limiting the partitioning of chroma intra CBs. In a single coding tree, the smallest chroma intra prediction unit (SCIPU) is defined as a coding tree node whose chroma block size is greater than or equal to 16 chroma samples and has at least one sub-luminance block, wherein the at least one sub-luminance block is smaller than 64 luminance samples. It is required that in each SCIPU, all CBs are inter-frame, or all CBs are non-inter-frame, i.e., intra-frame or intra-frame block copy (IBC). In the case of non-inter SCIPU, it is further required that the chroma of the non-inter SCIPU should not be further split, and the luminance of the SCIPU is allowed to be further split. In this way, the minimum chroma intra CB size is 16 chroma samples, and 2×2, 2×4 and 4×2 chroma CBs are removed.

[0174] In addition, chroma scaling is not applied in the case of non-inter SCIPU. Here, no additional syntax is signaled, and whether the SCIPU is non-inter can be derived from the prediction mode of the first luma CB in the SCIPU. If the current slice is an I slice or has a 4×4 luma partition within the current SCIPU after further splitting once (because inter 4×4 is not allowed in VVC), the type of the SCIPU is inferred to be non-inter; otherwise, the type of the SCIPU (intra or non-intra) is indicated by a flag before parsing the CU in the SCIPU. In addition, restrictions on picture size are considered to avoid 2×2 / 2×4 / 4×2 intra chroma blocks at the corners of the picture by considering the picture width and height to be multiples of max(8, MinCbSizeY).

[0175] II.3 Partition and block size related syntax in SPS

[0176] Table 3 is an example SPS syntax table.

[0177] Table 3

[0178]

[0179] II.4 Semantics related to partitioning and block size

[0180] As an example, the semantics of some syntax elements related to partitioning and block size are described below. For example, the syntax elements may be included in the SPS and applied to pictures that reference the SPS.

[0181] When qtbtt_dual_tree_intra_flag is equal to 1, it may indicate that for an I slice, each CTU is partitioned into coding units with 64×64 luma samples using implicit quadtree partitioning, and these coding units are the roots of two separate coding_tree syntax structures for luma and chroma. When qtbtt_dual_tree_intra_flag is equal to 0, it specifies that the separate coding_tree syntax structures are not used for I slices. When qtbtt_dual_tree_intra_flag is absent, it is inferred to be equal to 0.

[0182] log2_min_luma_coding_block_size_minus2 plus 2 indicates the minimum luma coding block size. The value range of log2_min_luma_coding_block_size_minus2 should be in the range of 0 to log2_ctu_size_minus5 + 3 (including the end values).

[0183] The variables MinCbLog2SizeY, MinCbSizeY, IbcBufWidthY, IbcBufWidthC, and Vsize are derived as follows:

[0184] MinCbLog2SizeY = log2_min_luma_coding_block_size_minus2 + 2 (Equation 2-1)

[0185] MinCbSizeY = 1 << MinCbLog2SizeY (Equation 2-2)

[0186] IbcBufWidthY = 256 * 128 / CtbSizeY (Equation 2-3)

[0187] IbcBufWidthC = IbcBufWidthY / SubWidthC (Equation 2-4)

[0188] VSize = Min(64, CtbSizeY) (Equation 2-5)

[0189] The value of MinCbSizeY should be less than or equal to VSize.

[0190] Variables CtbWidthC and CtbHeightC, which specify the width and height of the array of each chroma CTB respectively, are derived as follows:

[0191] - If chroma_format_idc is equal to 0 (monochrome) or separate_color_plane_flag is equal to 1, then both CtbWidthC and CtbHeightC are equal to 0.

[0192] - Otherwise, CtbWidthC and CtbHeightC are derived as follows:

[0193] CtbWidthC = CtbSizeY / SubWidthC (Equation 2-6)

[0194] CtbHeightC = CtbSizeY / SubHeightC (Equation 2-7)

[0195] For log2BlockWidth in the range from 0 to 4 and log2BlockHeight in the range from 0 to 4 (including the end values), 1 << log2BlockWidth and 1 << log2BlockHeight and 1 << log2BlockHeight can be used as inputs to call the upper-right diagonal and raster scan order array initialization processes, and the outputs are assigned to DiagScanOrder[log2BlockWidth][log2BlockHeight] and Raster2DiagScanPos[log2BlockWidth][log2BlockHeight].

[0196] For log2BlockWidth in the range from 0 to 6 and log2BlockHeight in the range from 0 to 6 (including the end values), 1 << log2BlockWidth and 1 << log2BlockHeight can be used as inputs to call the horizontal and vertical traversal scan order array initialization processes, and the outputs are assigned to HorTravScanOrder[log2BlockWidth][log2BlockHeight] and VerTravScanOrder[log2BlockWidth][log2BlockHeight].

[0197] partition_constraints_override_enabled_flag equal to 1 may indicate that partition_constraints_override_flag exists in the PH referencing the SPS. partition_constraints_override_enabled_flag equal to 0 may indicate that partition_constraints_override_flag does not exist in the PH referencing the SPS.

[0198] sps_log2_diff_min_qt_min_cb_intra_slice_luma indicates the default difference between the logarithm to base 2 of the minimum size in units of luma samples of the luma leaf blocks resulting from quadtree partitioning of the CTU and the logarithm to base 2 of the minimum coding block size in units of luma samples of the luma CU in the slice with slice_type equal to 2(I) of the reference SPS. When partition_constraints_override_enabled_flag is equal to 1, the default difference can be overridden by pic_log2_diff_min_qt_min_cb_luma present in the PH of the reference SPS. The value of sps_log2_diff_min_qt_min_cb_intra_slice_luma should be in the range of 0 to CtbLog2SizeY-MinCbLog2SizeY (inclusive). The base-2 logarithm of the minimum size in units of luma samples of a luma leaf block resulting from a quadtree partition of a CTU is derived as follows:

[0199] MinQtLog2SizeIntraY=sps_log2_diff_min_qt_min_cb_intra_slice_luma+

[0200] MinCbLog2SizeY (Equation 2-8)

[0201] sps_log2_diff_min_qt_min_cb_inter_slice may indicate the default difference between the logarithm to base 2 of the minimum size in units of luma samples of the luma leaf blocks resulting from quadtree partitioning of a CTU and the logarithm to base 2 of the minimum luma coding block size in units of luma samples of the luma CU in slices with slice_type equal to 0 (B) or 1 (P) of the reference SPS. When partition_constraints_override_enabled_flag is equal to 1, the default difference may be overridden by pic_log2_diff_min_qt_min_cb_luma present in the PH of the reference SPS. The value of sps_log2_diff_min_qt_min_cb_inter_slice shall be in the range of 0 to CtbLog2SizeY-MinCbLog2SizeY, inclusive. The base-2 logarithm of the minimum size in units of luma samples of a luma leaf block resulting from a quadtree partition of a CTU is derived as follows:

[0202] MinQtLog2SizeInterY=sps_log2_diff_min_qt_min_cb_inter_slice+

[0203] MinCbLog2SizeY (Equation 2-9)

[0204] sps_max_mtt_hierarchy_depth_inter_slice may indicate the default maximum hierarchical depth of coding units resulting from multi-type tree partitioning of quadtree leaves in slices with slice_type equal to 0 (B) or 1 (P) of the reference SPS. When partition_constraints_override_enabled_flag is equal to 1, the default maximum hierarchical depth may be overridden by pic_max_mtt_hierarchy_depth_inter_slice present in the picture header (PH) of the reference SPS. The value of sps_max_mtt_hierarchy_depth_inter_slice shall be in the range of 0 to 2*(CtbLog2SizeY-MinCbLog2SizeY), inclusive.

[0205] sps_max_mtt_hierarchy_depth_intra_slice_luma may indicate the default maximum hierarchical depth of coding units resulting from multi-type tree partitioning of quadtree leaves in slices with slice_type equal to 2(I) of the reference SPS. When partition_constraints_override_enabled_flag is equal to 1, the default maximum hierarchical depth may be overridden by pic_max_mtt_hierarchy_depth_intra_slice_luma present in the PH of the reference SPS. The value of sps_max_mtt_hierarchy_depth_intra_slice_luma shall be in the range of 0 to 2*(CtbLog2SizeY-MinCbLog2SizeY), inclusive.

[0206] sps_log2_diff_max_bt_min_qt_intra_slice_luma may indicate the default difference between the base-2 logarithm of the maximum size (width or height) in units of luma samples of a luma coding block that may be partitioned using binary partitioning and the minimum size (width or height) in units of luma samples of a luma leaf block resulting from quadtree partitioning of a CTU in a slice with slice_type equal to 2(I) of the reference SPS. When partition_constraints_override_enabled_flag is equal to 1, the default difference may be overridden by pic_log2_diff_max_bt_min_qt_luma present in the PH of the reference SPS. The value of sps_log2_diff_max_bt_min_qt_intra_slice_luma shall be in the range of 0 to CtbLog2SizeY - MinQtLog2SizeIntraY, inclusive. When sps_log2_diff_max_bt_min_qt_intra_slice_luma is not present, the value of sps_log2_diff_max_bt_min_qt_intra_slice_luma is inferred to be equal to 0.

[0207] sps_log2_diff_max_tt_min_qt_intra_slice_luma may indicate the default difference between the base-2 logarithm of the maximum size (width or height) in units of luma samples of a luma coding block that may be partitioned using ternary partitioning and the minimum size (width or height) in units of luma samples of a luma leaf block resulting from a quadtree partitioning of a CTU in a slice with slice_type equal to 2(I) of the reference SPS. When partition_constraints_override_enabled_flag is equal to 1, the default difference may be overridden by pic_log2_diff_max_tt_min_qt_luma present in the PH of the reference SPS. The value of sps_log2_diff_max_tt_min_qt_intra_slice_luma shall be in the range of 0 to CtbLog2SizeY - MinQtLog2SizeIntraY, inclusive. When sps_log2_diff_max_tt_min_qt_intra_slice_luma is not present, the value of sps_log2_diff_max_tt_min_qt_intra_slice_luma is inferred to be equal to 0.

[0208] sps_log2_diff_max_bt_min_qt_inter_slice may indicate the default difference between the base-2 logarithm of the maximum size (width or height) in units of luma samples of a luma coding block that may be partitioned using binary partitioning and the minimum size (width or height) in units of luma samples of a luma leaf block resulting from a quadtree partitioning of a CTU in a slice with slice_type equal to 0 (B) or 1 (P) of the reference SPS. When partition_constraints_override_enabled_flag is equal to 1, the default difference may be overridden by pic_log2_diff_max_bt_min_qt_luma present in the PH of the reference SPS. The value of sps_log2_diff_max_bt_min_qt_inter_slice shall be in the range of 0 to CtbLog2SizeY - MinQtLog2SizeInterY, inclusive. When sps_log2_diff_max_bt_min_qt_inter_slice is not present, the value of sps_log2_diff_max_bt_min_qt_inter_slice is inferred to be equal to 0.

[0209] sps_log2_diff_max_tt_min_qt_inter_slice may indicate the default difference between the base-2 logarithm of the maximum size (width or height) in units of luma samples of a luma coding block that may be partitioned using ternary partitioning and the minimum size (width or height) in units of luma samples of a luma leaf block resulting from a quadtree partitioning of a CTU in a slice with slice_type equal to 0 (B) or 1 (P) of the reference SPS. When partition_constraints_override_enabled_flag is equal to 1, the default difference may be overridden by pic_log2_diff_max_tt_min_qt_luma present in the PH of the reference SPS. The value of sps_log2_diff_max_tt_min_qt_inter_slice shall be in the range of 0 to CtbLog2SizeY - MinQtLog2SizeInterY, inclusive. When sps_log2_diff_max_tt_min_qt_inter_slice is not present, the value of sps_log2_diff_max_tt_min_qt_inter_slice is inferred to be equal to 0.

[0210] sps_log2_diff_min_qt_min_cb_intra_slice_chroma may indicate the default difference between the base-2 logarithm of the minimum size in units of luma samples of a chroma leaf block and the base-2 logarithm of the minimum coding block size in units of luma samples for a chroma CU resulting from quadtree partitioning of a chroma CTU with treeType equal to DUAL_TREE_CHROMA, the chroma CU having treeType equal to DUAL_TREE_CHROMA in a slice with slice_type equal to 2(I) of the reference SPS. When partition_constraints_override_enabled_flag is equal to 1, the default difference may be overridden by pic_log2_diff_min_qt_min_cb_chroma present in the PH of the reference SPS. The value of sps_log2_diff_min_qt_min_cb_intra_slice_chroma shall be in the range of 0 to CtbLog2SizeY-MinCbLog2SizeY, inclusive. When not present, the value of sps_log2_diff_min_qt_min_cb_intra_slice_chroma is inferred to be equal to 0. The base-2 logarithm of the minimum size in units of luma samples of a chroma leaf block resulting from a quadtree partitioning of a CTU with treeType equal to DUAL_TREE_CHROMA is derived as follows:

[0211] MinQtLog2SizeIntraC=sps_log2_diff_min_qt_min_cb_intra_slice_chroma+

[0212] MinCbLog2SizeY (Equation 2-10)

[0213] sps_max_mtt_hierarchy_depth_intra_slice_chroma may indicate the default maximum hierarchical depth of chroma coding units resulting from multi-type tree partitioning of chroma quad leaves with treeType equal to DUAL_TREE_CHROMA in slices with slice_type equal to 2(I) of the reference SPS. When partition_constraints_override_enabled_flag is equal to 1, the default maximum hierarchical depth may be overridden by pic_max_mtt_hierarchy_depth_chroma present in the PH of the reference SPS. The value of sps_max_mtt_hierarchy_depth_intra_slice_chroma shall be in the range of 0 to 2*(CtbLog2SizeY-MinCbLog2SizeY), inclusive. When not present, the value of sps_max_mtt_hierarchy_depth_intra_slice_chroma is inferred to be equal to 0.

[0214] sps_log2_diff_max_bt_min_qt_intra_slice_chroma may indicate the default difference between the base-2 logarithm of the maximum size (width or height) in units of luma samples of a chroma coding block that may be partitioned using binary partitioning and the minimum size (width or height) in units of luma samples of a chroma leaf block resulting from a quadtree partitioning of a chroma CTU with treeType equal to DUAL_TREE_CHROMA in a slice with slice_type equal to 2(I) of the reference SPS. When partition_constraints_override_enabled_flag is equal to 1, the default difference may be overridden by pic_log2_diff_max_bt_min_qt_chroma present in the PH of the reference SPS. The value of sps_log2_diff_max_bt_min_qt_intra_slice_chroma shall be in the range of 0 to CtbLog2SizeY - MinQtLog2SizeIntraC, inclusive. When sps_log2_diff_max_bt_min_qt_intra_slice_chroma is not present, the value of sps_log2_diff_max_bt_min_qt_intra_slice_chroma is inferred to be equal to 0.

[0215] sps_log2_diff_max_tt_min_qt_intra_slice_chroma may specify the default difference between the base-2 logarithm of the maximum size (width or height) in units of luma samples of a chroma coding block that may be partitioned using ternary partitioning and the minimum size (width or height) in units of luma samples of a chroma leaf block resulting from a quadtree partitioning of a chroma CTU with treeType equal to DUAL_TREE_CHROMA in a slice with slice_type equal to 2(I) of the reference SPS. When partition_constraints_override_enabled_flag is equal to 1, the default difference may be overridden by pic_log2_diff_max_tt_min_qt_chroma present in the PH of the reference SPS. The value of sps_log2_diff_max_tt_min_qt_intra_slice_chroma shall be in the range of 0 to CtbLog2SizeY-MinQtLog2SizeIntraC, inclusive. When sps_log2_diff_max_tt_min_qt_intra_slice_chroma is not present, the value of sps_log2_diff_max_tt_min_qt_intra_slice_chroma is inferred to be equal to 0.

[0216] sps_max_luma_transform_size_64_flag equal to 1 may indicate that the maximum transform size in units of luma samples is equal to 64. sps_max_luma_transform_size_64_flag equal to 0 may indicate that the maximum transform size in units of luma samples is equal to 32.

[0217] When CtbSizeY is less than 64, the value of sps_max_luma_transform_size_64_flag shall be equal to 0.

[0218] The variables MinTbLog2SizeY, MaxTbLog2SizeY, MinTbSizeY, and MaxTbSizeY are derived as follows:

[0219] MinTbLog2SizeY=2(Equation 2-11)

[0220] MaxTbLog2SizeY=sps_max_luma_transform_size_64_flag? 6:5 (Equation 2-12)

[0221] MinTbSizeY = 1 << MinTbLog2SizeY (Equation 2-13)

[0222] MaxTbSizeY = 1 << MaxTbLog2SizeY (Equation 2-14)

[0223] pic_log2_diff_min_qt_min_cb_intra_slice_chroma can indicate the difference between the base-2 logarithm of the minimum size in luma samples of a chroma leaf block resulting from the quadtree partitioning of a chroma CTU with treeType equal to DUAL_TREE_CHROMA and the base-2 logarithm of the minimum coded block size in luma samples of a chroma CU with treeType equal to DUAL_TREE_CHROMA in a slice, where the slice has a slice_type associated with PH equal to 2(I). The value of pic_log2_diff_min_qt_min_cb_intra_slice_chroma shall be in the range of 0 to CtbLog2SizeY - MinCbLog2SizeY (inclusive of the end values). When not present, the value of pic_log2_diff_min_qt_min_cb_intra_slice_chroma is inferred to be equal to sps_log2_diff_min_qt_min_cb_intra_slice_chroma.

[0224] slice_type can indicate the coding / decoding type of a slice according to Table 4.

[0225] Table 4

[0226] slice_type The name of slice_type 0 B (B band) 1 P(P stripe) 2 I(I strip)

[0227] When the nal_unit_type is a value of nal_unit_type in the range of IDR_W_RADL to CRA_NUT (inclusive of the end values), and the current picture is the first picture in an access unit, slice_type shall be equal to 2.

[0228] The variables MinQtLog2SizeY, MinQtLog2SizeC, MinQtSizeY, MinQtSizeC, MaxBtSizeY, MaxBtSizeC, MinBtSizeY, MaxTtSizeY, MaxTtSizeC, MinTtSizeY, MaxMttDepthY, and MaxMttDepthC are derived as follows:

[0229] MinQtSizeY = 1 << MinQtLog2SizeY (Equation 2-15)

[0230] MinQtSizeC = 1 << MinQtLog2SizeC (Equation 2-16)

[0231] MinBtSizeY = 1 << MinCbLog2SizeY (Equation 2-17)

[0232] MinTtSizeY = 1 << MinCbLog2SizeY (Equation 2-18)

[0233] (i) If slice_type is equal to 2 (I),

[0234] MinQtLog2SizeY = MinCbLog2SizeY +

[0235] pic_log2_diff_min_qt_min_cb_intra_slice_luma (Equation 2-19)

[0236] MinQtLog2SizeC = MinCbLog2SizeC +

[0237] pic_log2_diff_min_qt_min_cb_intra_slice_chroma (Equation 2-20)

[0238] MaxBtSizeY = 1 << (MinQtLog2SizeY +

[0239] pic_log2_diff_max_bt_min_qt_intra_slice_luma) (Equation 2-21)

[0240] MaxBtSizeC = 1 << (MinQtLog2SizeC +

[0241] pic_log2_diff_max_bt_min_qt_intra_slice_chroma) (Equation 2-22)

[0242] MaxTtSizeY = 1 << (MinQtLog2SizeY +

[0243] pic_log2_diff_max_tt_min_qt_intra_slice_luma) (Equation 2-23)

[0244] MaxTtSizeC=1<<(MinQtLog2SizeC+

[0245] pic_log2_diff_max_tt_min_qt_intra_slice_chroma) (Equation 2-24)

[0246] MaxMttDepthY = pic_max_mtt_hierarchy_depth_intra_slice_luma (Equation 2-25)

[0247] MaxMttDepthC = pic_max_mtt_hierarchy_depth_intra_slice_chroma (Equation 2-26)

[0248] CuQpDeltaSubdiv=pic_cu_qp_delta_subdiv_intra_slice (Equation 2-93)

[0249] CuChromaQpOffsetSubdiv=pic_cu_chroma_qp_offset_subdiv_intra_slice (Equation 2-27)

[0250] (ii) Otherwise (slice_type is equal to 0 (B) or 1 (P)),

[0251] MinQtLog2SizeY=MinCbLog2SizeY+pic_log2_diff_min_qt_min_cb_inter_slice (Equation 2-29)

[0253] MinQtLog2SizeC=MinCbLog2SizeC+pic_log2_diff_min_qt_min_cb_inter_slice (Equation 2-29)

[0255] MaxBtSizeY=1<<(MinQtLog2SizeY+

[0256] pic_log2_diff_max_bt_min_qt_inter_slice)(Equation 2-30)

[0257] MaxBtSizeC=1<<(MinQtLog2SizeC+

[0258] pic_log2_diff_max_bt_min_qt_inter_slice)(Equation 2-31)

[0259] MaxTtSizeY=1<<(MinQtLog2SizeY+

[0260] pic_log2_diff_max_tt_min_qt_inter_slice)(Equation 2-32)

[0261] MaxTtSizeC=1<<(MinQtLog2SizeC+

[0262] pic_log2_diff_max_tt_min_qt_inter_slice)(Equation 2-33)

[0263] MaxMttDepthY = pic_max_mtt_hierarchy_depth_inter_slice (Equation 2-34)

[0264] MaxMttDepthC = pic_max_mtt_hierarchy_depth_inter_slice (Equation 2-35)

[0265] CuQpDeltaSubdiv=pic_cu_qp_delta_subdiv_inter_slice (Equation 2-36)

[0266] CuChromaQpOffsetSubdiv=pic_cu_chroma_qp_offset_subdiv_inter_slice (Equation 2-37)

[0267] II.5 Disable 2×N size chroma intra prediction

[0268] In some embodiments, 2×N chroma intra blocks are removed from both the dual tree and the single tree. The p scheme is given as follows:

[0269] II.5.1 2 × N Limitation in Dual Trees

[0270] In the dual tree, as proposed previously, the 2×N intra chroma is restricted by disabling some partitions. In particular, binary and ternary tree partitions are disabled for blocks of width 4 and 8, respectively.

[0271] II.5.2 Limitation of 2 × N in a single tree

[0272] In order to remove the 2×N in the single tree, two restrictions are proposed, including the expansion of the local dual tree and the restriction of CIIP of chroma 2×N.

[0273] In the first restriction, if the partitions and splits of width 4 are binary vertical splits, or if the partitions and splits of width 8 are ternary vertical splits, they are considered SCIPUs. Following the principle restriction of SCIPUs, chroma components are not split within intra SCIPUs (all luma blocks are coded using non-inter mode, and non-split chroma blocks are coded using intra mode). For inter SCIPUs (all luma blocks and chroma blocks are coded using inter mode), the splits of the chroma components are inherited from the luma components.

[0274] In the second restriction, for a 4xN CIIP block, combined intra-inter prediction is used only for luma components, while inter prediction is used only for chroma components.

[0275] Asserting the proposed restriction ensures that the width of the intra chroma block is always greater than or equal to 4, and thus the 2×N pixel intra process is removed. This restriction makes the implementation of the video codec friendly to hardware implementation in terms of pipeline management and latency.

[0276] III. Minimum Luma Coding Block Size Range

[0277] In some embodiments, the minimum luma coding block size may be in the range of 4 to 128 luma samples (inclusive). For example, as described in B. Bross, J. Chen, S. Liu, Y.-K. Wang, "Next Generation Video Coding (Draft 7)", ISO / IEC JTC1 / SC29 / WG11 JVET-P2001, October 2019, an SPS-level syntax element represented by log2_min_luma_coding_block_size_minus2 may be signaled to specify the minimum luma coding block size. log2_min_luma_coding_block_size_minus2 plus 2 may indicate the base 2 logarithm (log2) value of the minimum luma coding block size. The value range of log2_min_luma_coding_block_size_minus2 should be in the range of 0 to log2_ctu_size_minus5+3 (inclusive).

[0278] The syntax element log2_ctu_size_minus5 (or in another form as sps_log2_ctu_size_minus5 when included in the SPS) may be another SPS-level syntax element that may indicate the luma coding tree block size (i.e., CTU size) for each CTU. sps_log2_ctu_size_minus5 plus 5 may indicate the log2 value of the luma coding tree block size for each CTU. A codestream conformance requirement is that the value of sps_log2_ctu_size_minus5 is less than or equal to 2. In other words, the log2 value of the luma coding tree block size for each CTU is in the range of 5 to 7. Thus, the luma coding tree block size for each CTU is in the range of 32 (2 5 )) to 128(2 7 )) (inclusive of) brightness samples.

[0279] Because the value range of log2_min_luma_coding_block_size_minus2 should be in the range of 0 to log2_ctu_size_minus5+3 (including the end values), the minimum luma coding block size can be in the range of 4 to the luma coding tree block size, which is from 32 luma samples to 128 luma samples.

[0280] As an example, when the luma coding tree block size is signaled to be 128, the encoder may determine to use 128 as the minimum luma coding block size. When such a large block size is used, the feature of partitioning smaller CUs to capture local areas in the picture may be disabled, which may result in a fast decoding speed, however, the quality of the decoded video sequence at the decoder side is lower.

[0281] The present disclosure provides a solution to the low quality problem caused by an excessively large minimum luma coding block size.

[0282] III.1 Limiting the minimum luma coding block size

[0283] In some embodiments, the upper limit of the range of the minimum luma coding block size is defined as a specific value to limit the minimum luma coding block size. For example, the minimum luma coding block size may be limited to a range of 4 to 64 luma samples (including end values) or 4 to 32 luma samples (including end values).

[0284] Example A

[0285] In an embodiment, as a bitstream conformance requirement specified in a video coding standard (e.g., VVC, HEVC, or the like), the minimum luma coding block size signaled in the bitstream needs to be in the range of 4 to (1<<(N+2)) luma samples (i.e., equal to 2 2 to 2 N+2 of the range). Thus, the log2 value of the minimum luma coding block size is in the range of 2 to N+2, and the log2 value of the minimum luma coding block size minus 2 is in the range of 0 to N. N can be an integer such as 0, 1, 2, 3, 4, etc. By specifying the number N, the upper limit of the range of the minimum luma coding block size can be specified.

[0286] As an example, a syntax element represented by log2_min_luma_coding_block_size_minus2 can be signaled in a parameter set (e.g., SPS) to indicate the minimum luma coding block size. The semantics can be specified as follows:

[0287] - log2_min_luma_coding_block_size_minus2 plus 2 can indicate the minimum luma coding block size. The value range of log2_min_luma_coding_block_size_minus2 should be in the range of 0 to N (including the end values).

[0288] - The variables MinCbLog2SizeY and MinCbSizeY can be derived as follows:

[0289] MinCbLog2SizeY = log2_min_luma_coding_block_size_minus2 + 2 (Equation 3-1)

[0290] MinCbSizeY = 1<<MinCbLog2SizeY (Equation 3-2) where MinCbLog2SizeY represents the log2 value of the minimum luma coding block size, and MinCbSizeY represents the minimum luma coding block size in units of luma samples.

[0291] N can be an integer such as 0, 1, 2, 3, 4, etc. When N is equal to 0, the minimum luma coding block size is fixed to the size of 4 luma samples. When N is equal to 4, the minimum luma coding block size is limited to the range of 4 to 64 (1<<(4+2)) luma samples. When N is equal to 3, the minimum luma coding block size is limited to the range of 4 to 32 (1<<(3+2)) luma samples.

[0292] In an embodiment, log2_min_luma_coding_block_size_minus2 may be signaled using fixed length coding. When N is equal to 1, a 1-bit fixed length coding may be used. When N is equal to 2 or 3, a 2-bit fixed length coding may be used. In other embodiments, other coding schemes may be used.

[0293] Example B

[0294] In the above embodiment A, the integer N is used to define the upper limit of the range of the minimum luminance block size as 2 N+2 When N takes the value of 0, 1, 2, 3, or 4, the maximum allowable minimum luma block size can be 4, 8, 16, 32, and 64, respectively. When a value greater than 2 is signaled in a parameter set (e.g., SPS), N+2 When the CTU size is 128, the upper limit is 2 N+2 Used to limit the range of minimum luma block size. When less than 2 is signaled in the parameter set (e.g., SPS) N+2 When the CTU size is large (such as when N=4, and 2 N +2 = 64, when a CTU size of 32 luma samples is signaled), this CTU size becomes the upper limit of the minimum luma block size. Therefore, the equivalent of embodiment A is to limit the minimum luma block size to 0 to min(2 N+2 , CTU size). 2 N+2 The smaller value of and CTU size is used as the upper limit for the minimum luma block size.

[0295] Therefore, in another embodiment, as a codestream consistency requirement specified in the video coding standard, it is required that the minimum luma coding block size signaled in the codestream is in the range of 4 to min((1<<(N+2)), CTU size) luma samples, which is equivalent to 2 2 to min(2 N+2 , 2 lo2 value of CTU size ). Therefore, the log2 value of the minimum luma coding block size is in the range of 2 to min(N+2, the log2 value of the CTU size). N can be an integer such as 0, 1, 2, 3, 4, etc. By specifying the number N, the upper limit of the range of the minimum luma coding block size can be specified.

[0296] As an example, a syntax element denoted by log2_min_luma_coding_block_size_minus2 may be signaled in a parameter set (e.g., SPS) to indicate the minimum luma coding block size. The semantics may be specified as follows:

[0297] -log2_min_luma_coding_block_size_minus2 plus 2 specifies the minimum luma coding block size. The value range of log2_min_luma_coding_block_size_minus2 should be in the range of 0 to min(N, sps_log2_ctu_size_minus5+3), inclusive.

[0298] The syntax element sps_log2_ctu_size_minus5 may be signaled in the same parameter set and indicates the log2 value of the CTU size minus 5.

[0299] N can be an integer such as 0, 1, 2, 3, 4, etc. In the first case, when N is equal to 0, the minimum luma coding block size is fixed to a size of 4 luma samples. In the second case, when N is equal to 4, the minimum luma coding block size is limited to a range of 4 to min(64, CTU size) luma samples. When the CTU size is signaled as 128 luma samples, the minimum luma coding block size is limited to a range of 4 to 64 luma samples. When the CTU size is signaled as 32 luma samples, the minimum luma coding block size is limited to a range of 4 to 32 luma samples.

[0300] Example C

[0301] In an embodiment, the decoder may be configured to verify whether the minimum luma coding block size signaled in the code stream satisfies the code stream consistency requirement during the decoding process. For example, the decoder may first receive a syntax element specifying the minimum luma coding block size from the code stream. For example, the syntax element provides the log2 value of the minimum luma coding block size minus 2. In the present disclosure, the log2 value may also be referred to as a binary logarithmic value or a logarithmic value with base 2.

[0302] The decoder can then verify whether the binary logarithm value of the minimum brightness coding block size minus 2 is within the range of 0 to N (including the end value). N is an integer such as 0, 1, 2, 4, etc., and N+2 represents the binary logarithm value of the predefined maximum allowable minimum brightness coding block size. In the example, when the binary logarithm value of the minimum brightness coding block size minus 2 is greater than N, the decoder can determine that the code stream consistency requirement is not met, and can terminate the decoding process and / or output an error message. When it is determined that the binary logarithm value of the minimum brightness coding block size minus 2 is within the range of 0 to N, the decoder can continue the decoding process.

[0303] Example D

[0304] In an embodiment, the decoder may be configured to verify whether the minimum luma coding block size signaled in the code stream satisfies the code stream consistency requirement during the decoding process. For example, the decoder may receive a first syntax element indicating the minimum luma coding block size from the code stream. The decoder may also receive a second syntax element indicating the CTU size from the code stream before or after the first syntax element.

[0305] The decoder can then verify whether the minimum luma coding block size is within the range of allowable minimum luma coding block sizes. When the CTU size is larger than the predefined maximum allowable minimum luma coding block size, the predefined maximum allowable minimum luma coding block size is used as the upper limit of the range of allowable minimum luma coding block sizes. When the CTU size is smaller than the predefined maximum allowable minimum luma coding block size, the CTU size is used as the upper limit of the range of allowable minimum luma coding block sizes.

[0306] When the minimum luma coding block size is within the range of the allowable minimum luma coding block size, the decoder may continue the decoding process. Otherwise, the decoder may terminate the decoding process.

[0307] Example E

[0308] In an embodiment, the decoder may be configured to verify whether the minimum luma coding block size signaled in the code stream satisfies the code stream consistency requirement during the decoding process. For example, the decoder may receive a first syntax element indicating the minimum luma coding block size from the code stream. For example, the syntax element provides the log2 value of the minimum luma coding block size minus 2. The decoder may receive a second syntax element indicating the CTU size from the code stream before or after the first syntax element. The second syntax element may provide the log2 value of the CTU size minus 5 and is represented by log2_ctu_size_minus5.

[0309] The decoder can then verify whether the binary logarithm value of the minimum luma coding block size minus 2 is in the range of 0 to Min(N, log2_ctu_size_minus5+3) (including the end value). N is an integer such as 0, 1, 2, 3, 4, etc. N+2 represents the binary logarithm value of the predefined maximum allowable minimum luma coding block size. log2_ctu_size_minus5+5 represents the binary logarithm value of the CTU size.

[0310] When the minimum luma coding block size is within the range of the allowable minimum luma coding block size, the decoder may continue the decoding process. Otherwise, the decoder may terminate the decoding process.

[0311] Example F

[0312] In an embodiment, the decoder may be configured to verify whether the minimum luma coding block size signaled in the codestream satisfies the codestream consistency requirements during the decoding process. Similar to embodiment E, the decoder may receive a first syntax element indicating the minimum luma coding block size from the codestream. The syntax element provides the log2 value of the minimum luma coding block size minus 2 and is represented as log2_min_luma_coding_block_size_minus2. The decoder may receive a second syntax element indicating the CTU size from the codestream before or after the first syntax element. The second syntax element may provide the log2 value of the CTU size minus 5 and is represented by log2_ctu_size_minus5.

[0313] Different from embodiment E, the decoder may first determine the binary logarithm value of the minimum luma coding block size as log2_min_luma_coding_block_size_minus2+2, and determine the binary logarithm value of the CTU size as log2_ctu_size_minus5+5. Then, the decoder may verify whether the binary logarithm value of the minimum luma coding block size is greater than Min(N+2, log2_ctu_size). N is an integer such as 0, 1, 2, 3, 4, etc. N+2 represents the binary logarithm value of the predefined maximum allowable minimum luma coding block size. log2_ctu_size represents the binary logarithm value of the CTU size.

[0314] When the minimum luma coding block size is greater than Min(N+2, log2_ctu_size), the decoder may terminate the decoding process and / or output an error message. Otherwise, the decoder may continue the decoding process.

[0315] III.2 Minimum luma coding block size constraints based on CTU size

[0316] In some embodiments, as a code stream consistency requirement specified in a video coding standard, the difference between the maximum allowable minimum luma block size and the signaled CTU size is specified as a range for limiting the minimum luma block size. The difference can be specified based on the log2 value of the maximum allowable minimum luma block size and the log2 value of the signaled CTU size. For example, the minimum luma coding block size may be required to be in the range of 4 to (1<<(log2_ctu_size_minus5+M+2)) luma samples. M may be an integer such as 0, 1, 2, 3, etc. In terms of log2 values, the upper limit of the range, log2_ctu_size_minus5+M+2, may be smaller than the CTU size by (3-M), and the CTU size may be represented by log2_ctu_size_minus5+5. For example, when M is equal to 3, there is no difference between the upper limit and the CTU size. When M is equal to 2, in terms of luma samples, the upper limit is half the CTU size. For example, for a CTU size of 128 luma samples, M equal to 2 defines an upper limit of 64 luma samples.

[0317] As an example, a syntax element denoted by log2_min_luma_coding_block_size_minus2 may be signaled in a parameter set (e.g., SPS) to indicate the minimum luma coding block size. The semantics may be specified as follows:

[0318] -log2_min_luma_coding_block_size_minus2 plus 2 may indicate the minimum luma coding block size. The value range of log2_min_luma_coding_block_size_minus2 shall be in the range of 0 to log2_ctu_size_minus5+M (inclusive).

[0319] In an embodiment, the decoder may be configured to verify whether the minimum luma coding block size signaled in the codestream satisfies the above codestream consistency requirements during the decoding process. The decoder may receive a first syntax element indicating the minimum luma coding block size from the codestream. The syntax element provides the log2 value of the minimum luma coding block size minus 2. The decoder may receive a second syntax element specifying the CTU size from the codestream before or after the first syntax element. The second syntax element may provide the log2 value of the CTU size minus 5 and is represented by log2_ctu_size_minus5.

[0320] The decoder can then verify whether the binary logarithm value of the minimum luma coding block size minus 2 is within the range of 0 to log2_ctu_size_minus5+M (including the end value). Similarly, when the minimum luma coding block size is within the range of the allowable minimum luma coding block size, the decoder can continue the decoding process. Otherwise, the decoder can terminate the decoding process.

[0321] III.3 Fixed minimum luma coding block size

[0322] In order to avoid the coding performance being reduced due to an excessively large minimum luma coding block size, in some embodiments, the minimum luma coding block size may be set to a predefined value (e.g., 4 luma samples). In this way, according to the statistical characteristics of the local area in the picture, the tree-structured block partitioning may be appropriately performed to use various block sizes.

[0323] For example, a predefined value may be specified as a bitstream conformance requirement in a video coding standard. To verify whether a bitstream conforms to the minimum luma coding block size requirement, a decoder may receive a syntax element indicating the minimum luma coding block size in the bitstream and check the value of the syntax element against the predefined value.

[0324] IV. Decoding Process with Limited Range of Minimum Luma Coding Block Size

[0325] Fig.21 A flow chart outlining a process (2100) according to an embodiment of the present disclosure is shown. The process (2100) may be used to decode a sequence of pictures that have been encoded in a code stream. In various embodiments, the process (2100) is performed by a processing circuit such as a processing circuit in a terminal device (210), (220), (230), and (240), a processing circuit that performs the functions of a video decoder (310), a processing circuit that performs the functions of a video decoder (410), and the like. In some embodiments, the process (2100) is implemented in software instructions, so when the processing circuit executes the software instructions, the processing circuit performs the process (2100). The process starts at (S2101) and proceeds to (S2110).

[0326] At (S2110), a first syntax element may be received from a bitstream of an encoded video. The first syntax element may indicate a minimum luma coding block size and be included in a parameter set such as an SPS. Alternatively, in some examples, the first syntax element indicating the minimum luma coding block size may be included in a picture header or other syntax structure of one or more picture slices in the bitstream. A picture header (PH) may be a syntax structure that contains syntax elements that apply to all slices of an encoded picture.

[0327] At (S2120), verify whether the minimum luma coding block size is within the range of the allowable minimum luma coding block size. The range may be specified as a code stream consistency requirement in a video coding standard (e.g., HEVC, VVC, etc.). The range may have a lower limit such as 4, 8, or 16 luma samples, and an upper limit that is less than the maximum allowable CTU size. For example, as specified in the video coding standard, the maximum allowable CTU size may be 128 luma samples. The upper limit of the range of the allowable minimum luma coding block size may be a value less than 128 luma samples.

[0328] For example, the upper limit may be the smaller of a predefined maximum allowable minimum luma coding block size and the CTU size specified by the second syntax element. The predefined maximum allowable minimum luma coding block size may be 1<<(N+2) (or in another form, 2 N+2 ), where N can be 0, 1, 2, 3, 4, etc. The second syntax element can be signaled in the codestream.

[0329] In the example, N is equal to 4, and the predefined maximum allowable minimum luma coding block size is 64 luma samples. When the signaled CTU size is 128 luma samples, the verification operation can use the size, 64 luma samples, as an upper limit. When the signaled CTU size is 32 luma samples, the verification operation can use the size, 32 luma samples, as an upper limit. When the signaled CTU size is 64 luma samples, the same as the predefined maximum allowable minimum luma coding block size, the verification operation can use the size, 64 luma samples, as an upper limit.

[0330] In an example, when it is determined that the signaled minimum luma coding block size is within the range of allowable minimum luma coding block sizes, the process (2100) can continue. Otherwise, the process (2100) can be terminated. Additionally, the decoder can output an error message indicating that the signaled minimum luma coding block size is not within the range of allowable minimum luma coding block sizes.

[0331] At (S2130), a coded picture in the coded video may be decoded based on the minimum luma coding block size. The coded picture references a parameter set, the parameter set including a first syntax element. Alternatively, each coded picture includes a picture header including the first syntax element. Therefore, a CU in the coded picture may have a luma coding block size greater than or equal to the minimum luma coding block size indicated by the first syntax element.

[0332] Various decoding operations may be performed based on the minimum luma coding block size. In an example, the picture size of the encoded video may conform to the following rules. pps_pic_width_in_luma_samples may indicate the width of each decoded picture that references the PPS in units of luma samples. pps_pic_width_in_luma_samples should not be equal to 0, should be an integer multiple of Max(8, MinCbSizeY), and should be less than or equal to sps_pic_width_max_in_luma_samples. MinCbSizeY represents the minimum luma coding block size. Therefore, the decoder may verify whether the width of each decoded picture that references the PPS conforms to the rules based on the minimum luma coding block size.

[0333] As another example, the minimum luma coding block size, MinCbSizeY, may be used to determine the maximum number of bins as specified in Generic Video Coding (Draft 7) resulting from decoding the context of a decoded slice NAL unit. Specifically, let NumBytesInVclNalUnit be the sum of the values ​​of NumBytesInNalUnit for all VCL NAL units of a coded picture. Let BinCountsInNalUnit be the number of times the parsing process function DecodeBin() is called to decode the contents of all VCL NAL units of a coded picture. The variable RawMinCuBits may be derived as follows:

[0334] RawMinCuBits=MinCbSizeY*MinCbSizeY*

[0335] (BitDepth+2*BitDepth / (SubWidthC*SubHeightC)) (Equation 3-3)

[0336] The value of BinCountsInNalUnit should be less than or equal to (32 ÷ 3) * NumBytesInVclNalUnits + (RawMinCuBits * PicSizeInMinCbsY) ÷ 32.

[0337] The process (2100) may proceed to (S2199) after (S2130) and terminate at (S2199).

[0338] V.Computer System

[0339] The above techniques may be implemented as computer software via computer-readable instructions and physically stored in one or more computer-readable media. Fig. 22A computer device (2200) is shown that is suitable for implementing certain embodiments of the disclosed subject matter.

[0340] The computer software may be encoded in any suitable machine code or computer language, and may be assembled, compiled, linked, or the like to create a code comprising instructions, which may be directly executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or executed by decoding, microcode, or the like.

[0341] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablets, servers, smartphones, gaming devices, IoT devices, etc.

[0342] Fig. 22 The components shown for the computer device (2200) are exemplary in nature and are not intended to limit the scope of use or functionality of the computer software implementing the embodiments of the present application. The configuration of the components should not be interpreted as having any dependency or requirement on any component or combination of components shown in the exemplary embodiment of the computer device (2200).

[0343] The computer device (2200) may include certain human-computer interface input devices. Such human-computer interface input devices may respond to input from one or more human users through tactile input (e.g., keyboard input, sliding, data glove movement), audio input (e.g., sound, applause), visual input (e.g., gestures), and olfactory input (not shown). The human-computer interface device may also be used to capture certain media that are not necessarily directly related to human conscious input, such as audio (e.g., speech, music, ambient sound), images (e.g., scanned images, photographic images obtained from a still image camera), and videos (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0344] The human-computer interface input device may include one or more of the following (only one of which is drawn): keyboard (2201), mouse (2202), touchpad (2203), touch screen (2210), data gloves (not shown), joystick (2205), microphone (2206), scanner (2207), camera (2208).

[0345] The computer device (2200) may also include certain human-computer interface output devices. Such human-computer interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-computer interface output devices may include tactile output devices (e.g., tactile feedback through a touch screen (2210), a data glove (not shown), or a joystick (2205), but there may also be tactile feedback devices that are not used as input devices), audio output devices (e.g., speakers (2209), headphones (not shown)), visual output devices (e.g., screens (2210) including cathode ray tube screens, liquid crystal screens, plasma screens, organic light emitting diode screens, each of which has or does not have a touch screen input function, each of which has or does not have a tactile feedback function - some of which can output two-dimensional visual output or output of more than three dimensions by means such as stereoscopic picture output; virtual reality glasses (not shown), holographic displays, and smoke boxes (not shown)) and printers (not shown).

[0346] The computer device (2200) may also include human-accessible storage devices and their associated media, such as optical media including high-density read-only / rewritable optical disks (CD / DVD ROM / RW) (2220) with CD / DVD or similar media (2221), thumb drives (2222), removable hard disk drives or solid state drives (2223), traditional magnetic media such as tapes and floppy disks (not shown), special-purpose ROM / ASIC / PLD-based devices such as security software protectors (not shown), and the like.

[0347] Those skilled in the art should also understand that the term "computer-readable media" used in connection with the disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0348] The computer device (2200) may also include an interface to one or more communication networks. The network may be wireless, wired, or optical. The network may also be a local area network, a wide area network, a metropolitan area network, an in-vehicle network, an industrial network, a real-time network, a delay-tolerant network, and the like. Examples of networks may include local area networks such as Ethernet, wireless local area networks, cellular networks (GSM, 3G, 4G, 5G, LTE, etc.), television wired or wireless wide area digital networks (including cable television, satellite television, and terrestrial broadcast television), in-vehicle and industrial networks (including CANBus), and the like. Some networks typically require an external network interface adapter for connecting to some universal data ports or peripheral buses (2249) (e.g., a USB port of the computer device (2200)); other systems are typically integrated into the core of the computer device (2200) by connecting to a system bus as described below (e.g., an Ethernet interface integrated into a PC computer system or a cellular network interface integrated into a smart phone computer system). By using any of these networks, the computer device (2200) can communicate with other entities. The communication can be one-way, for receiving only (e.g., wireless television), one-way for sending only (e.g., CAN bus to certain CAN bus devices), or two-way, such as to other computer systems via a local or wide area digital network. Each of the above networks and network interfaces can use certain protocols and protocol stacks.

[0349] The above-mentioned human-machine interface devices, human-accessible storage devices, and network interfaces may be connected to the core (2240) of the computer device (2200).

[0350] The core (2240) may include one or more central processing units (CPUs) (2241), graphics processing units (GPUs) (2242), dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) (2243), hardware accelerators for specific tasks (2244), etc. These devices, as well as read-only memory (ROM) (2245), random access memory (2246), internal mass storage (e.g., internal non-user accessible hard disk drives, solid-state drives, etc.) (2247), etc., may be connected via a system bus (2248). In some computer systems, the system bus (2248) may be accessed in the form of one or more physical plugs so that it may be expanded by additional central processing units, graphics processing units, etc. Peripheral devices may be directly attached to the core's system bus (2248) or connected via a peripheral bus (2249). The architecture of the peripheral bus includes PCI, USB, etc. In one example, the screen (2210) may be connected to a graphics adapter (2250). The architecture of the peripheral bus includes PCI, USB, etc.

[0351] The CPU (2241), GPU (2242), FPGA (2243) and accelerator (2244) can execute certain instructions, which can be combined to form the above-mentioned computer code. The computer code can be stored in ROM (2245) or RAM (2246). Transition data can also be stored in RAM (2246), while permanent data can be stored in, for example, internal mass storage (2247). Fast storage and retrieval of any memory device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (2241), GPUs (2242), mass storage (2247), ROM (2245), RAM (2246), etc.

[0352] The computer readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be specially designed and constructed for the purpose of this application, or may be medium and code well known and available to those skilled in the art of computer software.

[0353] As an example and not a limitation, a computer system having an architecture (2200), in particular a core (2240), can provide the function of executing software contained in one or more tangible computer-readable media as a processor (including a CPU, a GPU, an FPGA, an accelerator, etc.). Such a computer-readable medium can be a medium associated with the above-mentioned user-accessible mass storage, as well as a specific memory of the core (2240) having non-volatility, such as a core internal mass storage (2247) or a ROM (2245). Software implementing various embodiments of the present application can be stored in such a device and executed by the core (2240). Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software can enable the core (2240), in particular the processor therein (including a CPU, a GPU, an FPGA, etc.) to perform a specific process or a specific part of a specific process described herein, including defining a data structure stored in the RAM (2246) and modifying such a data structure according to a software-defined process. Additionally or alternatively, the computer system may provide functionality hardwired in logic or otherwise contained in circuitry (e.g., accelerator (2244)) that may operate in place of or in conjunction with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic and vice versa. Where appropriate, references to computer-readable media may include circuitry (e.g., an integrated circuit (IC)) storing the executing software, circuitry containing the executing logic, or both. The present application includes any suitable combination of hardware and software.

[0354] Appendix: Acronyms

[0355] AMVP: Advanced MVP

[0356] ASIC: Application-Specific Integrated Circuit

[0357] BDOF: Bidirectional Optical Flow

[0358] BMS: Benchmark Collection

[0359] CANBus: Controller Area Network Bus

[0360] CBF: Coded Block Flag

[0361] CCLM: Cross-Component Linear Pattern / Model

[0362] CD: compact disc

[0363] CIIP: Combined Inter / Intra Prediction

[0364] CPUs: Central Processing Units

[0365] CRT: cathode ray tube

[0366] CTBs: Coding Tree Blocks

[0367] CTUs: Coding Tree Units

[0368] CU: Coding Unit

[0369] DVD: Digital Video Disc

[0370] FPGA: Field Programmable Gate Array

[0371] GOPs: Group of Pictures

[0372] GPUs: Graphics Processing Units

[0373] GSM: Global System for Mobile Communications

[0374] HEVC: High Efficiency Video Coding

[0375] HMVP: History-based MVP

[0376] HRD: Hypothesized Reference Decoder

[0377] IC: Integrated Circuit

[0378] JEM: Joint Development Model

[0379] LAN: Local Area Network

[0380] LCD: Liquid crystal display

[0381] LFNST: Low Frequency Non-separable Transform

[0382] LIC: Brightness Compensation

[0383] LTE: Long Term Evolution

[0384] MMVD: Merge Mode MVD

[0385] MV: Motion Vector

[0386] MVD: Motion Vector Difference

[0387] MVP: Motion vector predictor motion vector prediction value

[0388] OLED: Organic Light Emitting Diode

[0389] PBs: prediction blocks

[0390] PCI: Peripheral Component Interconnect

[0391] PDPC: Position Dependent Prediction Combination

[0392] PLD: Programmable Logic Device

[0393] PUs: prediction units

[0394] RAM: Random Access Memory

[0395] ROM: Read Only Memory

[0396] RST: Reduced Quadratic Transform

[0397] SbTMVP: Sub-block based TMVP

[0398] SCIPU: Smallest Chroma Intra Prediction Unit

[0399] SEI: Supplemental Enhancement Information

[0400] SNR: Signal to Noise Ratio

[0401] SSD: Solid State Drive

[0402] TMVP: Temporary MVP

[0403] TPM: Triangle Partition Mode

[0404] TUs: Transformation Units

[0405] USB: Universal Serial Bus

[0406] VPDU:Vision Processing Data Unit

[0407] VTM: Versatile Test Model

[0408] VUI: Video Availability Information

[0409] VVC: Next Generation Video Coding

[0410] Although the present application has described a number of exemplary embodiments, various changes, arrangements and various equivalent substitutions of the embodiments are within the scope of the present application. Therefore, it should be understood that those skilled in the art can design a variety of systems and methods, which, although not explicitly shown or described herein, embody the principles of the present application and are therefore within the spirit and scope of the present application.

Claims

1. A method for video decoding, characterized in that: include: Receiving a first syntax element from a bitstream of an encoded video, the first syntax element indicating a minimum luma coding block size; verifying whether the minimum luma coding block size is within a range of allowable minimum luma coding block sizes, the range of allowable minimum luma coding block sizes having an upper limit that is less than a maximum allowable coding tree unit (CTU) size; as well as Decoding an encoded picture in the encoded video based on the minimum luma coding block size; The first syntax element indicates the binary logarithm of the minimum luma coding block size minus 2, and The verifying whether the minimum luma coding block size is within the range of the allowable minimum luma coding block size comprises: Verify whether the binary logarithm of the minimum luma coding block size minus 2 is in the range of 0 to log2_ctu_size_minus5+M (including the end values), where log2_ctu_size_minus5+5 represents the binary logarithm of the CTU size of the encoded video, and log2_ctu_size_minus5+M+2 represents the binary logarithm of the predefined maximum allowable minimum luma coding block size, where the value of M includes 0.

2. The method according to claim 1, characterized in that The upper limit of the range of allowable minimum luma coding block sizes is a predefined maximum allowable minimum luma coding block size.

3. The method according to claim 1, characterized in that Further including: receiving a second syntax element from the bitstream of the encoded video, the second syntax element indicating a CTU size, Wherein, when the CTU size is larger than a predefined maximum allowable minimum luma coding block size, the predefined maximum allowable minimum luma coding block size is used as the upper limit of the range of the allowable minimum luma coding block size.

4. The method according to claim 3, characterized in that When the CTU size is smaller than the predefined maximum allowable minimum luma coding block size, the CTU size is used as the upper limit of the range of the allowable minimum luma coding block sizes.

5. A device for video decoding, characterized in that: comprising a circuit configured to: Receiving a first syntax element from a bitstream of an encoded video, the first syntax element indicating a minimum luma coding block size; verifying whether the minimum luma coding block size is within a range of allowable minimum luma coding block sizes, the range of allowable minimum luma coding block sizes having an upper limit that is less than a maximum allowable coding tree unit (CTU) size; as well as Decoding an encoded picture in the encoded video based on the minimum luma coding block size; The first syntax element indicates the binary logarithm of the minimum luma coding block size minus 2, and The verifying whether the minimum luma coding block size is within the range of the allowable minimum luma coding block size comprises: Verify whether the binary logarithm value of the minimum luma coding block size minus 2 is in the range of 0 to log2_ctu_size_minus5+M (including the end values), where log2_ctu_size_minus5+5 represents the binary logarithm value of the CTU size of the encoded video, and log2_ctu_size_minus5+M+2 represents the binary logarithm value of the predefined maximum allowable minimum luma coding block size, where M=0.

6. The device according to claim 5, characterized in that The upper limit of the range of allowable minimum luma coding block sizes is a predefined maximum allowable minimum luma coding block size.

7. The device according to claim 5, characterized in that The circuit is further configured to: receiving a second syntax element from the bitstream of the encoded video, the second syntax element indicating a CTU size, Wherein, when the CTU size is larger than a predefined maximum allowable minimum luma coding block size, the predefined maximum allowable minimum luma coding block size is used as the upper limit of the range of the allowable minimum luma coding block size.

8. The device according to claim 7, characterized in that When the CTU size is smaller than the predefined maximum allowable minimum luma coding block size, the CTU size is used as the upper limit of the range of the allowable minimum luma coding block sizes.

9. A video encoding method for generating a video code stream, characterized in that: include: encoding a first syntax element indicating a minimum luma coding block size; confirming whether the minimum luma coding block size of a coded picture is within a range of allowable minimum luma coding block sizes, the range of allowable minimum luma coding block sizes having an upper limit that is less than a maximum allowable coding tree unit CTU size; as well as encoding the coded picture based on the minimum luma coding block size; wherein the first syntax element indicates the binary logarithm of the minimum luma coding block size minus 2, and The determining whether the minimum luminance coding block size is within the range of an allowable minimum luminance coding block size comprises: Confirm whether the binary logarithm of the minimum luminance coding block size minus 2 is in the range of 0 to log2_ctu_size_minus5+M (including the end values), where log2_ctu_size_minus5+5 represents the binary logarithm of the CTU size of the encoded video, and log2_ctu_size_minus5+M+2 represents the binary logarithm of the predefined maximum allowable minimum luminance coding block size, where the value of M includes 0.

10. A video encoding device, characterized in that: comprising a circuit configured to: encoding a first syntax element indicating a minimum luma coding block size; confirming whether the minimum luma coding block size of a coded picture is within a range of allowable minimum luma coding block sizes, the range of allowable minimum luma coding block sizes having an upper limit that is less than a maximum allowable coding tree unit CTU size; as well as encoding the coded picture based on the minimum luma coding block size; wherein the first syntax element indicates the binary logarithm of the minimum luma coding block size minus 2, and The determining whether the minimum luminance coding block size is within the range of an allowable minimum luminance coding block size comprises: Confirm whether the binary logarithm of the minimum luminance coding block size minus 2 is in the range of 0 to log2_ctu_size_minus5+M (including the end values), where log2_ctu_size_minus5+5 represents the binary logarithm of the CTU size of the encoded video, and log2_ctu_size_minus5+M+2 represents the binary logarithm of the predefined maximum allowable minimum luminance coding block size, where the value of M includes 0.

11. A computer device, characterized in that: include: The method comprises a processor and a memory; the memory stores computer codes, and when the computer codes are executed by the processor, the processor executes the method according to any one of claims 1 to 4 or 9.

12. A non-volatile computer-readable medium storing instructions, characterized in that: When the instruction is executed by a processor, the processor executes the method according to any one of claims 1 to 4 to decode the video code stream, or executes the video encoding method according to claim 9 to generate an encoded video code stream.

13. A method for processing a video code stream, characterized in that: The video code stream is generated according to the encoding method according to claim 9, or is decoded based on the decoding method according to any one of claims 1 to 8.

Images