Boundary block partitioning in video coding

Abstract

To provide a method for performing partitioning for encoding and decoding, a decoding device, an encoding device, and a storage medium.SOLUTION: A method includes the steps of: determining whether a current block of an image is a boundary block and whether size of the current block is greater than minimum allowed quadtree leaf node size; when the current block is the boundary block and the size of the current block is not greater than the minimum allowed quadtree leaf node size (MinQTSize), applying forced binary tree (BT) partitioning to the current block; determining that the current block of the image is the boundary block and the size of the current block is less than or equal to the minimum allowed quadtree (QT) leaf node size (MinQTSize); and applying, in response to the determination, forced binary tree (BT) partitioning to the current block.SELECTED DRAWING: Figure 7A

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 62 / 697,274, entitled "A Video Encoder, A Video Decoder and Corresponding Methods," filed July 12, 2018 by Futurewei Technologies, Inc., and U.S. Provisional Patent Application No. 62 / 818,996, entitled "Boundary Block Partitioning in Video Coding," filed March 15, 2019 by Futurewei Technologies, Inc., both of which are incorporated by reference.

[0002] TECHNICAL FIELD Embodiments of the present application relate generally to the field of video coding, and more particularly to splitting and partitioning coding units. [Background technology]

[0003] The amount of video data required to depict even a relatively short video can be substantial, which can create problems when streaming or communicating the data over communication networks with limited bandwidth capacity. Thus, video data is typically compressed before being communicated over modern telecommunications networks. When video is stored on a storage device, the size of the video can also be an issue because memory resources can be limited. Video compression devices often use software and / or hardware at the source to encode video data before transmission or storage, thereby reducing the amount of data needed to represent a digital video image. The compressed data is then received at the destination by a video decompressor, which decodes the video data. Due to limited network resources and the ever-increasing demand for higher video quality, improved compression and decompression techniques are desired that increase compression ratios with little or no sacrifice in image quality. Summary of the Invention

[0004] SUMMARY OF THE INVENTION Embodiments of the present application (or disclosure) provide apparatus and methods for encoding and decoding.

[0005] A first aspect relates to a partitioning method, the partitioning method comprising the steps of: determining whether a current block of an image is a border block and whether the size of the current block is greater than a minimum allowed quad-tree leaf node size; and applying forced binary tree (BT) partitioning to the current block if the current block is a border block and the size of the current block is not greater than the minimum allowed quad-tree leaf node size (MinQTSize).

[0006] In a first implementation of the method according to the first aspect itself, the forced binary tree partitioning is a recursive horizontal forced binary partitioning if the current block is located at the bottom boundary of the image, or a recursive vertical forced boundary partitioning if the current block is located at the right boundary of the image.

[0007] In a second embodiment of the method according to the first aspect itself or any preceding embodiment of the first aspect, the forced binary tree partitioning continues until a leaf node block is within the image.

[0008] In a third embodiment of the method according to the first aspect itself or any preceding embodiment of the first aspect, the forced binary partitioning comprises: recursively partitioning the current block by horizontal forced boundary partitioning until a subpartition of the current block is located at the bottom boundary of the image; and recursively partitioning the subpartition by vertical forced boundary partitioning until a leaf node is located entirely at the right boundary of the image.

[0009] In a fourth embodiment of the method according to the first aspect itself or any preceding embodiment of the first aspect, the forced BT partitioning includes: recursively partitioning the current block by vertical forced boundary partitioning until a subpartition of the current block is located on the bottom boundary; and recursively partitioning the subpartition by horizontal forced boundary partitioning until a leaf node is located entirely on the right boundary.

[0010] In a fifth embodiment of the method according to the first aspect itself or any preceding embodiment of the first aspect, the method further comprises applying a minimum allowable quadtree leaf node size to control partitioning of non-border blocks.

[0011] In a sixth embodiment of the method according to the first aspect itself or any preceding embodiment of the first aspect, a border block is a block that is neither entirely inside the image nor entirely outside the image.

[0012] A second aspect relates to a decoding method for decoding a block by partitioning the block according to the first aspect itself or any preceding implementation of the first aspect.

[0013] In a first implementation of the method according to the second aspect itself, the method further comprises receiving a minimum allowed quadtree leaf node size via a sequence parameter set (SPS).

[0014] A third aspect relates to an encoding method for encoding a block by partitioning the block according to the first aspect itself or any preceding implementation of the first aspect.

[0015] In a first implementation of the method according to the third aspect itself, the method further comprises the step of transmitting the minimum allowed quadtree leaf node size via a sequence parameter set (SPS).

[0016] A fourth aspect relates to a decoding device comprising logic circuitry configured to decode blocks by partitioning the blocks according to the partitioning method of the first aspect itself or any preceding implementation of the first aspect.

[0017] In a first implementation of the decoding device according to the fourth aspect itself, the logic circuitry is further configured to receive a minimum allowed quadtree leaf node size via a sequence parameter set (SPS).

[0018] A fifth aspect relates to an encoding apparatus comprising logic circuitry configured to encode blocks by partitioning the blocks according to the partitioning method of the first aspect itself or any preceding implementation of the first aspect.

[0019] In a first implementation of the encoding device according to the fifth aspect itself, the logic circuitry is further configured to transmit the minimum allowed quadtree leaf node size via a sequence parameter set (SPS).

[0020] A sixth aspect relates to a non-transitory storage medium for storing instructions that, when executed by a processor, cause the processor to perform any of the first, second, or third aspects themselves, or any preceding implementation of the first, second, or third aspects.

[0021] A seventh aspect relates to a method, the method including: determining that a current block of an image is a border block and that the size of the current block is less than or equal to a minimum allowable quad-tree (QT) leaf node size (MinQTSize); and applying a forced binary tree (BT) partitioning to the current block in response to the determination.

[0022] In a first implementation of the method according to the seventh aspect itself, the current block is located at the bottom boundary of the image and the forced BT partitioning is a recursive horizontal forced BT partitioning.

[0023] In a second embodiment of the method according to the seventh aspect itself or any preceding implementation of the seventh aspect, the current block is located at the right boundary of the image, and the forced BT partitioning is a recursive vertical forced BT partitioning.

[0024] In a third embodiment of the method according to the seventh aspect itself or any preceding embodiment of the seventh aspect, the forced BT partitioning includes: recursively partitioning the current block by horizontal forced boundary partitioning until the subpartitions of the current block are located on the bottom boundary; and recursively partitioning the subpartitions by vertical forced boundary partitioning until the leaf nodes are located entirely on the right boundary.

[0025] In a fourth embodiment of the method according to the seventh aspect itself or any preceding embodiment of the seventh aspect, the forced BT partitioning includes: recursively partitioning the current block by vertical forced boundary partitioning until a subpartition of the current block is located on the bottom boundary; and recursively partitioning the subpartition by horizontal forced boundary partitioning until a leaf node is located entirely on the right boundary.

[0026] In a fifth embodiment of the method according to the seventh aspect itself or any preceding embodiment of the seventh aspect, the method further comprises applying MinQTSize to control partitioning of non-border blocks.

[0027] In a sixth embodiment of the method according to the seventh aspect itself or any preceding embodiment of the seventh aspect, the method further comprises receiving MinQTSize via a sequence parameter set (SPS).

[0028] In a seventh embodiment of the method according to the seventh aspect itself or any preceding embodiment of the seventh aspect, the method further comprises transmitting MinQTSize via a sequence parameter set (SPS).

[0029] An eighth aspect relates to an apparatus including: a memory; and a processor coupled to the memory; wherein the processor is configured to: determine whether a current block of an image is a boundary block; if the current block is a boundary block, determine whether a size of the current block is greater than a minimum allowed quad-tree (QT) leaf node size (MinQTSize); and apply forced binary tree (BT) partitioning to the current block if the size of the current block is not greater than MinQTSize.

[0030] In a first embodiment of the device according to the eighth aspect itself, the forced BT partitioning is a recursive horizontal forced BT partitioning when the current block is located at the bottom boundary of the image, or a recursive vertical forced BT partitioning when the current block is located at the right boundary of the image.

[0031] In a second embodiment of the device according to the eighth aspect itself or any preceding embodiment of the eighth aspect, the forced BT partitioning includes: recursively partitioning the current block by horizontal forced boundary partitioning until the subpartitions of the current block are located on the lower boundary; and recursively partitioning the subpartitions by vertical forced boundary partitioning until the leaf node is located entirely on the right boundary.

[0032] In a third embodiment of the device according to the eighth aspect itself or any preceding embodiment of the eighth aspect, the forced BT partitioning includes: recursively partitioning the current block by vertical forced boundary partitioning until the subpartitions of the current block are located on the bottom boundary; and recursively partitioning the subpartitions by horizontal forced boundary partitioning until the leaf node is located entirely on the right boundary.

[0033] In a fourth embodiment of the apparatus according to the eighth aspect itself or any preceding embodiment of the eighth aspect, the processor is further configured to apply MinQTSize to control partitioning of non-border blocks.

[0034] In a fifth embodiment of the device according to the eighth aspect itself or any preceding embodiment of the eighth aspect, the device further includes a receiver coupled to the processor and configured to receive MinQTSize via a sequence parameter set (SPS).

[0035] In a sixth embodiment of the device according to the eighth aspect itself or any preceding embodiment of the eighth aspect, the device further includes a transmitter coupled to the processor and configured to transmit the MinQTSize via a sequence parameter set (SPS).

[0036] A ninth aspect relates to a computer program product including computer-executable instructions stored on a non-transitory medium that, when executed by a processor, cause an apparatus to: determine whether a current block of an image is a boundary block; if the current block is a boundary block, determine whether a size of the current block is greater than a minimum allowable quad-tree (QT) leaf node size (MinQTSize); and apply forced binary tree (BT) partitioning to the current block if the size of the current block is not greater than MinQTSize.

[0037] In a first implementation of the computer program product according to the ninth aspect itself, the forced BT partitioning is a recursive horizontal forced BT partitioning if the current block is located at the bottom boundary of the image, or a recursive vertical forced BT partitioning if the current block is located at the right boundary of the image.

[0038] In a second embodiment of a computer program product according to the ninth aspect itself or any preceding embodiment of the ninth aspect, the forced BT partitioning includes: recursively partitioning the current block by horizontal forced boundary partitioning until a subpartition of the current block is located on the bottom boundary; and recursively partitioning the subpartition by vertical forced boundary partitioning until a leaf node is located entirely on the right boundary.

[0039] In a third embodiment of a computer program product according to the ninth aspect itself or any preceding embodiment of the ninth aspect, the forced BT partitioning includes: recursively partitioning the current block by vertical forced boundary partitioning until the subpartitions of the current block are located on the bottom boundary; and recursively partitioning the subpartitions by horizontal forced boundary partitioning until the leaf nodes are located entirely on the right boundary.

[0040] In a fourth embodiment of the computer program product according to the ninth aspect itself or any preceding implementation of the ninth aspect, the instructions further cause the device to apply MinQTSize to control partitioning of non-border blocks.

[0041] In a fifth embodiment of the computer program product according to the ninth aspect itself or any preceding embodiment of the ninth aspect, the instructions further cause the device to receive MinQTSize via a sequence parameter set (SPS).

[0042] In a sixth embodiment of the computer program product according to the ninth aspect itself or any preceding embodiment of the ninth aspect, the instructions further cause the device to transmit the MinQTSize via a sequence parameter set (SPS).

[0043] For clarity, any one of the embodiments disclosed herein may be combined with any one or more of the other embodiments to form new embodiments within the scope of the present disclosure.

[0044] These and other features will be more clearly understood from the following detailed description considered in conjunction with the accompanying drawings and claims. [Brief explanation of the drawings]

[0045] For a more complete understanding of the present disclosure, reference is now made to the following brief description taken in conjunction with the accompanying drawings and detailed description, wherein like reference numerals represent like parts. [Figure 1A] FIG. 1 is a block diagram illustrating an exemplary encoding system in which embodiments of the present disclosure may be implemented. [Figure 1B] FIG. 1 is a block diagram illustrating another exemplary encoding system in which embodiments of the present disclosure may be implemented. [Figure 2] FIG. 1 is a block diagram illustrating an exemplary video encoder capable of implementing embodiments of the present disclosure. [Figure 3] FIG. 2 is a block diagram illustrating an example of a video decoder capable of implementing embodiments of the present disclosure. [Figure 4] FIG. 1 is a schematic diagram of a network device according to an embodiment of the present disclosure. [Figure 5] 1B is a simplified block diagram of equipment that can be used for either or both of source device 12 and destination device 14 of FIG. 1A according to an exemplary embodiment. [Figure 6] 6A to 6F are diagrams showing various CU division modes in VVC. [Figure 7A] A diagram showing the bottom boundary CTU (128x128) forced QT partition for HD (1920x1080). [Figure 7B] FIG. 10 illustrates a bottom boundary CTU (128×128) forced BT partition for HD (1920×1080) according to one embodiment of the present disclosure. [Figure 8]FIG. 1 illustrates an exemplary boundary definition. [Figure 9A] FIG. 10 illustrates an example corner case forced QTBT partition, according to one embodiment of the present disclosure. [Figure 9B] FIG. 10 illustrates an example of a forced QTBT partition for a corner block, according to one embodiment of the present disclosure. [Figure 10] FIG. 1 illustrates an embodiment of a boundary definition. DETAILED DESCRIPTION OF THE INVENTION

[0046] Initially, exemplary implementations of one or more embodiments are provided below, but it should be understood that the disclosed systems and / or methods can be implemented using any number of technologies, whether currently known or in existence. The present disclosure should in no way be limited to the exemplary implementations, drawings, and technologies illustrated below, including the exemplary designs and implementations illustrated and described herein, but may be modified within the scope of the appended claims along with their full range of equivalents.

[0047] FIG. 1A is a block diagram illustrating an exemplary encoding system 10, e.g., a video encoding system 10, capable of implementing embodiments of the present disclosure. The encoding system 10 may utilize predictive techniques, e.g., bidirectional predictive techniques. As shown in FIG. 1A, the encoding system 10 includes a source device 12 that provides encoded video data (to be subsequently decoded by a destination device 14). In particular, the source device 12 may provide the video data to the destination device 14 via a computer-readable medium 16. The source device 12 and the destination device 14 may include or be any of a wide range of devices, including desktop computers, notebook (i.e., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones and so-called "smart" pads, televisions, cameras, display devices, digital media players, video game consoles, or video streaming devices. In some cases, the source device 12 and the destination device 14 may be equipped for wireless communication.

[0048] The destination device 14 can receive the encoded video data to be decoded via a computer-readable medium 16. The computer-readable medium 16 can include any type of medium or device capable of moving encoded video data from the source device 12 to the destination device 14. In one example, the computer-readable medium 16 can include a communications medium that enables the source device 12 to transmit the encoded video data directly to the destination device 14 in real time. The encoded video data can be modulated according to a communications standard, such as a wireless communications protocol, and transmitted to the destination device 14. The communications medium can include any wireless or wired communications medium, such as the radio frequency (RF) spectrum or one or more physical transmission lines. The communications medium can form part of a packet-based network, such as a local area network, a wide area network, or a global network, such as the Internet. The communications medium can include routers, switches, base stations, or any other equipment that can be useful in facilitating communication from the source device 12 to the destination device 14.

[0049] In some examples, the encoded data may be output to a storage device from output interface 22. Similarly, the encoded data may be accessed from a storage device by input interface 28. The storage device may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a digital video disc (DVD), a compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In a further example, the storage device may correspond to a file server or another intermediate storage device that may store the encoded video generated by source device 12. Destination device 14 may access the stored video data from the storage device via streaming or download. The file server may be any type of server capable of storing encoded video data and transmitting the encoded video data to destination device 14. Examples of file servers include a web server (e.g., for a website), a file transfer protocol (FTP) server, a network-attached storage (NAS) device, or a local disk drive. Destination device 14 may access the encoded video data via any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both suitable for accessing encoded video data stored on a file server. The transmission of encoded video data from the storage device may be a streaming transmission, a download transmission, or a combination thereof.

[0050] The techniques of this disclosure are not necessarily limited to wireless applications or settings. The techniques may be applied to video encoding in support of any of a variety of multimedia applications, such as terrestrial television broadcast, cable television transmission, satellite television transmission, Internet streaming video transmission such as Dynamic Adaptive Streaming over HTTP (DASH), digital video encoded on a data storage medium, decoding of digital video stored on a data storage medium, or other applications. In some examples, encoding system 10 may be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and / or video telephony.

[0051] In the example of FIG. 1A , source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Destination device 14 includes an input interface 28, a video decoder 30, and a display device 32. According to this disclosure, video encoder 20 of source device 12 and / or video decoder 30 of destination device 14 may be configured to apply techniques for prediction, e.g., bidirectional prediction. In other examples, source device 12 and destination device 14 may include other components or configurations. For example, source device 12 may receive video data from an external video source, such as an external camera. Similarly, destination device 14 may interface with an external display device rather than including an integrated display device.

[0052] The illustrated encoding system 10 of FIG. 1A is merely an example. Techniques for prediction, e.g., bidirectional prediction, may be performed by any digital video encoding and / or decoding device. While the techniques of this disclosure are generally performed by a video coding device, the techniques may also be performed by a video encoder / decoder, typically referred to as a "CODEC." Additionally, the techniques of this disclosure may also be performed by a video preprocessor. The video encoder and / or decoder may be a graphics processing unit (GPU) or similar device.

[0053] Source device 12 and destination device 14 are merely examples of such encoding devices that generate encoded video data for source device 12 to transmit to destination device 14. In some examples, source device 12 and destination device 14 may operate in a substantially symmetric manner, such that source device 12 and destination device 14 each include video encoding and decoding components. Thus, encoding system 10 may support one-way or two-way video transmission between video devices 12, 14, e.g., for video streaming, video playback, video broadcasting, or video telephony.

[0054] Video source 18 of source device 12 may include a video capture device such as a video camera, a video archive containing previously captured video, and / or a video feed interface for receiving video from a video content provider. As a further alternative, video source 18 may generate computer graphics-based data as the source video or a combination of live video, archived video, and computer-generated video.

[0055] In some cases, when video source 18 is a video camera, source device 12 and destination device 14 may form a so-called camera phone or video telephone. However, as noted above, the techniques described in this disclosure may be applicable to video coding generally, and may be applied to wireless and / or wired applications. In either case, captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video information may then be output to computer-readable medium 16 by output interface 22.

[0056] Computer-readable medium 16 may include a transitory medium, such as an over-the-air broadcast or a wired network transmission, or computer-readable medium 16 may include a non-transitory storage medium, such as a hard disk, flash drive, compact disc, digital video disc, Blu-ray disc, or other computer-readable medium. In some examples, a network server (not shown) may receive encoded video data from source device 12 and provide the encoded video data to destination device 14, for example, via a network transmission. Similarly, computing devices of a media production facility, such as a disc stamping facility, may receive encoded video data from source device 12 and manufacture discs including the encoded video data. Accordingly, computer-readable medium 16 may be understood to include one or more computer-readable media of various forms in various examples.

[0057] Input interface 28 of destination device 14 receives information from computer-readable medium 16. The information on computer-readable medium 16 may include syntax information defined by video encoder 20, which is also used by video decoder 30, including syntax elements that describe the characteristics and / or processing of blocks and other coding units, such as groups of pictures (GOPs). Display device 32 displays the decoded video data to a user and may include any of a variety of display devices, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0058] Video encoder 20 and video decoder 30 may operate in accordance with a video coding standard, such as the High Efficiency Video Coding (HEVC) standard currently under development, and may conform to the HEVC Test Model (HM). Alternatively, video encoder 20 and video decoder 30 may operate in accordance with the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) H.264 standard, or other proprietary or industry standards referred to as Motion Picture Experts Group (MPEG)-4, Part 10, Advanced Video Coding (AVC), H.265 / HEVC, the Versatile Video Coding (VVC) draft standard currently under development by ITU-T and MPEG, or extensions to such standards. However, the techniques of this disclosure are not limited to any particular coding standard. Other examples of video coding standards include MPEG-2 and ITU-T H.263. 1A , in some aspects, video encoder 20 and video decoder 30 may be integrated with an audio encoder and decoder, respectively, and may include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software for handling both audio and video encoding in a common data stream or separate data streams. Where applicable, the MUX-DEMUX units may conform to the ITU-T H.223 multiplexer protocol or other protocols, such as the User Datagram Protocol (UDP).

[0059] The video encoder 20 and the video decoder 30 may each be implemented as any of a variety of suitable encoder or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, graphics processing units (GPUs), processors (e.g., programmable, e.g., software programmable), software, hardware, firmware, or any combination thereof, or as general-purpose logic or coder circuits. If the techniques are implemented partially in software, a device may store software instructions on a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (CODEC) within the respective device. Devices including the video encoder 20 and / or the video decoder 30 may include integrated circuits, microprocessors, and / or wireless communication devices, such as a mobile phone.

[0060] FIG. 1B is an illustrative diagram of an exemplary video encoding system 40 including the video encoder 20 of FIG. 2 and / or the video decoder 30 of FIG. 3, according to an exemplary embodiment. The video encoding system 40 may implement the techniques of the present application. In the illustrated implementation, the video encoding system 40 may include an imaging device 41, a video encoder 20, a video decoder 30 (and / or a video coder implemented via logic 47 of a processing device 46), an antenna 42, one or more processors 43, one or more memories or memory stores 44, and / or a display device 45. As shown, the imaging device 41, the antenna 42, the processing device 46, the logic 47, the video encoder 20, the video decoder 30, the processor 43, the memory store 44, and / or the display device 45 may be in communication with one another. As described, while FIG. 1B shows both the video encoder 20 and the video decoder 30, the video encoding system 40 may include only the video encoder 20 or only the video decoder 30 in various examples.

[0061] As shown, in some examples, video encoding system 40 may include antenna 42. Antenna 42 may be configured to transmit or receive, for example, an encoded bitstream of video data. Additionally, in some examples, video encoding system 40 may include display device 45.

[0062] The display device 45 may be configured to present the video data. As shown, in some examples, the logic circuitry 47 may be implemented via a processing unit 46. The processing unit 46 may include application specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, or the like. The video encoding system 40 may also include an optional processor 43, which may also include application specific integrated circuit (ASIC) logic, a graphics processor, a general-purpose processor, or the like. In some examples, the logic circuitry 47 may be implemented via hardware, dedicated video encoding hardware, or the like, and the processor 43 may be implemented by general-purpose software, an operating system, or the like. Furthermore, the memory store 44 may be any type of memory, such as volatile memory (e.g., static random access memory (SRAM), dynamic random access memory (DRAM), etc.) or non-volatile memory (e.g., flash memory, etc.). In a non-limiting example, the memory store 44 may be implemented by a cache memory. In some examples, the logic circuitry 47 may access the memory store 44 (e.g., for image buffering). In other examples, logic circuitry 47 and / or processing unit 46 may include memory stores (eg, caches, etc.) for implementation, such as image buffers.

[0063] The graphics processing unit includes video encoder 20, executed via logic circuitry 47, and may embody various modules such as those described with respect to FIG. 2 and / or any other encoder system or subsystem described herein. Logic circuitry 47 may be configured to perform various operations discussed herein. Video decoder 30 may be implemented in a similar manner, executed via logic circuitry 47, to embody various modules such as those described with respect to decoder 30 of FIG. 3 and / or any other decoder system or subsystem described herein. In some examples, video encoder 20 and video decoder 30 may be executed via logic circuitry and may include an image buffer (e.g., via either processing unit 46 or memory store 44) and a graphics processing unit (GPU) (e.g., via processing unit 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit includes video decoder 30, executed via logic circuitry 47, and may embody various modules such as those described with respect to FIG. 3 and / or any other decoder system or subsystem described herein.

[0064] In some examples, antenna 42 of video encoding system 40 may be configured to receive an encoded bitstream of video data. As discussed, the encoded bitstream may include data related to the encoding of video frames, indicators, index values, mode selection data, or the like, as discussed herein, such as data related to coding partitions (e.g., transform coefficients or quantized transform coefficients, optional indicators (as discussed), and / or data defining coding partitions). Video encoding system 40 may also include video decoder 30 coupled to antenna 42 and configured to decode the encoded bitstream. Display device 45 is configured to present the video frames.

[0065] 2 is a block diagram illustrating an example of a video encoder 20 capable of implementing the techniques of the present application. The video encoder 20 may perform intra-coding and inter-coding of video blocks within a video slice. Intra-coding relies on spatial prediction to reduce or remove spatial redundancy in video within a given video frame or image. Inter-coding relies on temporal prediction to reduce or remove temporal redundancy in video within adjacent frames or images of a video sequence. Intra-mode (I-mode) may refer to any of several spatial-based coding modes. Inter-mode, such as unidirectional prediction (P-mode) or bidirectional prediction (B-mode), may refer to any of several temporal-based coding modes.

[0066] As shown in FIG. 2, video encoder 20 receives a current video block in a video frame to be encoded. In the example of FIG. 2, video encoder 20 includes a mode select unit 40, a reference frame memory 64, a summer 50, a transform processor 52, a quantization unit 54, and an entropy coding unit 56. Mode select unit 40 includes a motion compensation unit 44, a motion estimation unit 42, an intra prediction unit 46, and a partition unit 48. For video block reconstruction, video encoder 20 also includes an inverse quantization unit 58, an inverse transform unit 60, and a summer 62. A deblocking filter (not shown in FIG. 2) may also be included to filter block boundaries and remove blockiness artifacts from the reconstructed video. If necessary (e.g., if activated or implemented at all), the deblocking filter filters the output of summer 62. In addition to the deblocking filter, additional filters (in-loop or post-loop) may also be used. For example, the output of summer 50 can be filtered (as an in-loop filter) if desired, although such a filter is not shown for simplicity.

[0067] During the encoding process, video encoder 20 receives a video frame or slice to be coded. The frame or slice may be divided into multiple video blocks. Motion estimation unit 42 and motion compensation unit 44 perform inter-predictive coding of the received video block relative to one or more blocks in one or more reference frames to provide temporal prediction. Alternatively, intra-prediction unit 46 may perform intra-predictive coding of the received video block relative to one or more neighboring blocks in the same frame or slice as the block to be coded to provide spatial prediction. Video encoder 20 may perform multiple coding passes, for example, to select an appropriate coding mode for each block of video data.

[0068] Furthermore, partition unit 48 may partition blocks of video data into sub-blocks or partitions based on an evaluation of a previous partitioning scheme in a previous encoding pass or step. For example, partition unit 48 may first partition a frame or slice into largest coding units (LCUs) and then partition each LCU into sub-coding units (sub-CUs) based on a rate-distortion analysis (e.g., rate-distortion optimization). Mode selection unit 40 may further generate a quadtree data structure that indicates or represents the partitioning of the LCUs into sub-CUs. A leaf-node CU of the quadtree may include one or more prediction units (PUs) and one or more transform units (TUs).

[0069] This disclosure uses the term "block" to refer to, for example, any of a CU, PU, ​​or TU in the context of HEVC or VVC, or similar data structures in the context of other standards (e.g., macroblocks and subblocks in H.264 / AVC). A CU includes, for example, a coding node, a PU, and a TU associated with the coding node. The size of a CU corresponds to the size of the coding node and is, for example, square in shape. The size of a CU can range from 8x8 pixels up to a treeblock size of 64x64 pixels or more, e.g., 128x128 pixels. Each CU can include one or more PUs and one or more TUs. Syntax data associated with a CU may, for example, describe partitioning the CU into one or more PUs. The partitioning mode may differ depending on whether the CU is encoded in skip or direct mode, intra-prediction mode, or inter-prediction mode. A PU may be partitioned to be non-square in shape. Syntax data associated with a CU may also describe partitioning the CU into one or more TUs, for example, according to a quadtree. In one embodiment, a CU, PU, ​​or TU may be square or non-square (eg, rectangular) in shape.

[0070] Mode select unit 40 selects one of intra or inter coding modes based on, for example, an error result, provides the resulting intra-coded or inter-coded blocks to summer 50 to generate residual block data, and provides the resulting intra-coded or inter-coded blocks to summer 62 to reconstruct encoded blocks for use as reference frames. Mode select unit 40 also provides syntax elements, such as motion vectors, intra-mode indicators, partition information, and / or other such syntax information, to entropy coding unit 56.

[0071] Motion estimation unit 42 and motion compensation unit 44 may be highly integrated but are shown separately for conceptual purposes. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate the motion of video blocks. A motion vector may indicate, for example, the displacement of a PU of a video block in a current video frame or image relative to a predictive block in a reference frame (or other coding unit) relative to a current block being coded in the current frame (or other coding unit). A predictive block is a block found to closely match a block being coded in terms of pixel differences, where the pixel differences are determined by sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metric. In some examples, video encoder 20 may calculate values ​​for sub-integer pixel locations of a reference image stored in reference frame memory 64. For example, video encoder 20 may interpolate values ​​for quarter-pixel, eighth-pixel, or other fractional pixel locations of a reference image. Thus, motion estimation unit 42 may perform motion searches for whole and fractional pixel locations and output motion vectors with fractional pixel accuracy.

[0072] Motion estimation unit 42 calculates a motion vector for a PU of a video block in an inter-coded slice by comparing the position of the PU with the position of a predictive block in a reference image. The reference image may be selected from a first reference image list (List 0) or a second reference image list (List 1), which each identify one or more reference images stored in reference frame memory 64. Motion estimation unit 42 sends the calculated motion vector to entropy coding unit 56 and motion compensation unit 44.

[0073] The motion compensation performed by motion compensation unit 44 may include fetching or generating a predictive block based on the motion vector determined by motion estimation unit 42. Again, motion estimation unit 42 and motion compensation unit 44 may, in some examples, be functionally integrated. Upon receiving the motion vector of the PU of the current video block, motion compensation unit 44 may locate the predictive block pointed to by the motion vector in one of the reference image lists. Adder 50 forms a residual video block by subtracting pixel values ​​of the predictive block from pixel values ​​of the current video block being coded to form pixel difference values, as described below. Generally, motion estimation unit 42 performs motion estimation on the luma component, and motion compensation unit 44 uses the motion vector calculated based on both the chroma and luma components. Mode select unit 40 may also generate syntax elements associated with the video blocks and video slices for use by video decoder 30 in decoding the video blocks of the video slices.

[0074] Intra prediction unit 46 may intra predict the current block as an alternative to the inter prediction performed by motion estimation unit 42 and motion compensation unit 44, as described above. In particular, intra prediction unit 46 may determine an intra prediction mode to use to encode the current block. In some examples, intra prediction unit 46 may encode the current block using different intra prediction modes, e.g., during separate encoding passes or steps, and intra prediction unit 46 (or mode selection unit 40 in some examples) may select an appropriate intra prediction mode to use from the tested modes.

[0075] For example, intra prediction unit 46 may calculate a rate-distortion value using a rate-distortion analysis of various tested intra prediction modes and select the intra prediction mode with the best rate-distortion characteristics among the tested modes. The rate-distortion analysis generally determines the amount of distortion (or error) between an encoded block and the original unencoded block that was encoded to generate the encoded block, as well as the bitrate (i.e., number of bits) used to generate the encoded block. Intra prediction unit 46 may calculate a ratio from the distortion and rate of the various encoded blocks to determine which intra prediction mode exhibits the best rate-distortion value for the block.

[0076] Furthermore, intra prediction unit 46 may be configured to encode depth blocks of the depth map using a depth modeling mode (DMM). Mode selection unit 40 may determine whether an available DMM mode produces better encoding results than intra prediction modes and other DMM modes, for example, using rate-distortion optimization (RDO). Data of texture images corresponding to the depth map may be stored in reference frame memory 64. Motion estimation unit 42 and motion compensation unit 44 may also be configured to inter-predict depth blocks of the depth map.

[0077] After selecting an intra-prediction mode for the block (e.g., one of a conventional intra-prediction mode or a DMM mode), intra-prediction unit 46 may provide information indicating the selected intra-prediction mode for the block to entropy coding unit 56. Entropy coding unit 56 may encode the information indicating the selected intra-prediction mode. Video encoder 20 may include specifications for encoding various block contexts, the most likely intra-prediction mode to use for each context, an intra-prediction mode index table, and an index to the modified intra-prediction mode index table in the transmitted bitstream configuration data (which may include multiple intra-prediction mode index tables and multiple modified intra-prediction mode index tables (also referred to as codeword mapping tables)).

[0078] Video encoder 20 forms a residual video block by subtracting the prediction data from mode select unit 40 from the original video block being coded. Summer 50 represents one or more components that perform this subtraction operation.

[0079] Transform processor 52 applies a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform, to the residual block to produce a video block containing residual transform coefficient values. Transform processor 52 may perform other transforms that are conceptually similar to the DCT. Wavelet transforms, integer transforms, subband transforms, or other types of transforms may also be used.

[0080] The transform processor 52 applies a transform to the residual block, generating a block of residual transform coefficients. The transform may convert the residual information from the pixel value domain to a transform domain, such as the frequency domain. The transform processor 52 may send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be varied by adjusting a quantization parameter. In some examples, the quantization unit 54 may then perform a scan of a matrix containing the quantized transform coefficients. Alternatively, the entropy encoding unit 56 may perform the scan.

[0081] Following quantization, entropy coding unit 56 entropy codes the quantized transform coefficients. For example, entropy coding unit 56 may perform context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding technique. In the case of context-based entropy coding, the context may be based on neighboring blocks. Following entropy coding by entropy coding unit 56, the encoded bitstream may be transmitted to another device (e.g., video decoder 30) or archived for later transmission or retrieval.

[0082] Inverse quantization unit 58 and inverse transform unit 60 apply inverse quantization and inverse transformation, respectively, to reconstruct a residual block in the pixel domain, e.g., for later use as a reference block. Motion compensation unit 44 may determine a prediction block (also referred to as a predictive block), e.g., a motion-compensated prediction block based on a motion vector and the corresponding reference block, derived from a reference frame, e.g., one of the frames in reference frame memory 64. Motion compensation unit 44 may also apply one or more interpolation filters to calculate sub-integer pixel values ​​of the reference block for use in motion estimation or compensation. Adder 62 adds the reconstructed residual block to the (motion-compensated) prediction block generated by motion compensation unit 44 to generate a reconstructed video block for storage in reference frame memory 64. The reconstructed video block may be used by motion estimation unit 42 and motion compensation unit 44 as a reference block for inter-coding blocks in subsequent video frames.

[0083] Other structural variations of video encoder 20 can be used to encode the video stream. For example, a non-transform-based video encoder 20 can directly quantize the residual signal for a particular block or frame, without transform processor 52. In another implementation, encoder 20 can combine quantization unit 54 and inverse quantization unit 58 into a single unit.

[0084] Figure 3 is a block diagram illustrating an example of a video decoder 30 capable of implementing the techniques of the present application. In the example of Figure 3, video decoder 30 includes an entropy decoding unit 70, a motion compensation unit 72, an intra prediction unit 74, an inverse quantization unit 76, an inverse transform unit 78, a reference frame memory 82, and an adder 80. Video decoder 30 may, in some examples, perform a decoding path or path that is generally inverse to the encoding path or path described with respect to video encoder 20 of Figure 2. Motion compensation unit 72 may generate prediction data based on motion vectors received from entropy decoding unit 70, while intra prediction unit 74 may generate prediction data based on an intra prediction mode indicator received from entropy decoding unit 70.

[0085] During the decoding process, video decoder 30 receives an encoded video bitstream representing video blocks and associated syntax elements of encoded video slices generated from video encoder 20. Entropy decoding unit 70 of video decoder 30 entropy decodes the bitstream to generate quantized coefficients, motion vectors or intra-prediction mode indicators, and other syntax elements. Entropy decoding unit 70 forwards the motion vectors and other syntax elements to motion compensation unit 72. Video decoder 30 may receive syntax elements at the video slice level and / or the video block level.

[0086] If a video slice is coded as an intra-coded (I) slice, intra prediction unit 74 may generate predictive data for video blocks of the current video slice based on signaled intra prediction modes and data from previously decoded blocks of the current frame or picture. If a video frame is coded as an inter-coded (i.e., B, P, or GPB) slice, motion compensation unit 72 generates predictive blocks for video blocks of the current video slice based on motion vectors and other syntax elements received from entropy decoding unit 70. The predictive blocks may be generated from one of the reference images in one of the reference image lists. Video decoder 30 may construct the reference frame lists, List 0 and List 1, using a default construction technique based on reference images stored in reference frame memory 82.

[0087] Motion compensation unit 72 determines prediction information for video blocks of the current video slice by analyzing the motion vectors and other syntax elements, and uses the prediction information to generate predictive blocks for the current video block being decoded. For example, motion compensation unit 72 uses some of the received syntax elements to determine the prediction mode (e.g., intra-prediction or inter-prediction) used to encode the video blocks of the video slice, the inter-prediction slice type (e.g., B slice, P slice, or GPB slice), configuration information of one or more reference image lists of the slice, the motion vectors of each inter-encoded video block of the slice, the inter-prediction status of each inter-coded video block of the slice, and other information to decode the video blocks in the current video slice.

[0088] Motion compensation unit 72 may also perform interpolation based on an interpolation filter. Motion compensation unit 72 may use an interpolation filter used by video encoder 20 during encoding of the video block to calculate interpolated values ​​of sub-integer pixels of the reference block. In this case, motion compensation unit 72 may determine the interpolation filter used by video encoder 20 from the received syntax element and use the interpolation filter to generate the predictive block.

[0089] Data for texture images corresponding to the depth maps may be stored in reference frame memory 82. Motion compensation unit 72 may also be configured to inter-predict depth blocks of the depth maps.

[0090] 1A, the video encoding system 10 is suitable for implementing various video encoding or compression techniques. Several video compression techniques, such as inter-prediction, intra-prediction, loop filter, etc., have proven effective. Therefore, video compression techniques have been adopted in various video encoding standards, such as H.264 / AVC and H.265 / HEVC.

[0091] Various coding tools such as adaptive motion vector prediction (AMVP) and merge mode (MERGE) can be used to predict motion vectors (MVs) and improve inter-prediction efficiency and therefore overall video compression efficiency.

[0092] The above MVs can be used in bi-prediction. In a bi-prediction operation, two prediction blocks are formed. One prediction block is formed using the MVs from list 0 (referred to herein as MV0). Another prediction block is formed using the MVs from list 1 (referred to herein as MV1). The two prediction blocks are then combined (e.g., averaged) to form a single prediction signal (e.g., a prediction block or predictor block).

[0093] Other variations of video decoder 30 can be used to decode the compressed bitstream. For example, decoder 30 can generate an output video stream without a loop filtering unit. For example, a non-transform-based decoder 30 can directly inverse quantize the residual signal for a particular block or frame, without an inverse transform processor 78. In another implementation, video decoder 30 can combine inverse quantization unit 76 and inverse transform processor 78 into a single unit.

[0094] 4 is a schematic diagram of a network device 400 (e.g., an encoding device) according to one embodiment of the present disclosure. Network device 400 is suitable for implementing the embodiments described and disclosed herein. In one embodiment, network device 400 may be a decoder, such as video decoder 30 of FIGS. 1A and 3, or an encoder, such as video encoder 20 of FIGS. 1A and 2. In one embodiment, network device 400 may be one or more components of video decoder 30 of FIGS. 1A and 3 or video encoder 20 of FIGS. 1A and 2, described above.

[0095] The network device 400 includes an input port 410 and a receiver unit (Rx) 420 for receiving data, a processor, logic unit, or central processing unit (CPU) 430 for processing data, a transmitter unit (Tx) 440 and an output port 450 for transmitting data, and a memory 460 for storing data. The network device 400 may also include optical-to-electrical (OE) and electrical-to-optical (EO) components coupled to the input port 410, the receiver unit 420, the transmitter unit 440, and the output port 450 for inputting or outputting optical or electrical signals.

[0096] The processor 430 may be implemented by hardware and software. The processor 430 may be implemented as one or more CPU chips, cores (e.g., as a multi-core processor), FPGA, ASIC, and DSP. The processor 430 is in communication with the input port 410, the receiver unit 420, the transmitter unit 440, the output port 450, and the memory 460. The processor 430 includes an encoding module 470. The encoding module 470 implements the above-disclosed embodiments. For example, the encoding module 470 performs, processes, prepares, or provides various encoding operations. Thus, the inclusion of the encoding module 470 provides substantial improvements to the functionality of the network device 400 and results in the transformation of the network device 400 into different states. Alternatively, the encoding module 470 is implemented as instructions stored in the memory 460 and executed by the processor 430.

[0097] Memory 460 may include one or more disks, tape drives, and solid state drives, may be used as overflow data storage, may store programs when such programs are selected for execution, and may store instructions and data read during the execution of programs. Memory 460 may be volatile and / or non-volatile, and may be read-only memory (ROM), random access memory (RAM), ternary content addressable memory (TCAM), and / or static random access memory (SRAM).

[0098] 5 is a simplified block diagram of a device 500 that can be used as either or both of source device 12 and destination device 14 of FIG. 1A according to an exemplary embodiment. Device 500 can implement the techniques of the present application. Device 500 can be in the form of a computing system including multiple computing devices, or in the form of a single computing device, such as a mobile phone, tablet computer, laptop computer, notebook computer, desktop computer, etc.

[0099] Processor 502 in device 500 may be a central processing unit. Alternatively, processor 502 may be any other type of device or devices now existing or later developed that are capable of manipulating or processing information. While the disclosed embodiments may be implemented with a single processor, such as processor 502, as shown, advantages of speed and efficiency may be achieved using multiple processors.

[0100] The memory 504 in the device 500 may, in embodiments, be a read-only memory (ROM) device or a random-access memory (RAM) device. Any other suitable type of storage device may be used as the memory 504. The memory 504 may contain code and data 506 accessed by the processor 502 using a bus 512. The memory 504 may further include an operating system 508 and application programs 510, which include at least one program that enables the processor 502 to perform the methods described herein. For example, the application programs 510 may include applications 1-N, which further include a video encoding application that performs the methods described herein. The device 500 may also include additional memory in the form of secondary storage 514, which may be, for example, a memory card used with a mobile computing device. Because video communication sessions may contain a significant amount of information, the sessions may be stored in whole or in part in the secondary storage 514 and loaded into the memory 504 as needed for processing.

[0101] Device 500 may also include one or more output devices, such as a display 518. Display 518, in one example, may be a touch-sensitive display that combines a display with touch-sensing elements operable to sense touch input. Display 518 may be coupled to processor 502 via bus 512. Other output devices that enable a user to program or otherwise use device 500 may be provided in addition to, or as an alternative to, display 518. When an output device is or includes a display, the display may be implemented in a variety of ways, including a liquid crystal display (LCD), a cathode ray tube (CRT) display, a plasma display, or a light emitting diode (LED) display, such as an organic LED (OLED) display.

[0102] Device 500 may also include or be in communication with image sensing device 520, e.g., a camera or any other image sensing device now existing or later developed that is capable of sensing images, such as an image of a user operating device 500. Image sensing device 520 may be positioned to be pointed at a user operating device 500. In one example, the position and optical axis of image sensing device 520 may be configured such that the field of view is immediately adjacent to display 518 and includes an area from which display 518 is viewable.

[0103] Device 500 may also include or communicate with sound sensing device 522, such as a microphone or any other sound sensing device now existing or later developed that can sense sounds near device 500. Sound sensing device 522 may be positioned to face a user operating device 500 and may be configured to receive sounds, such as voice or other utterances, made by the user while the user is operating device 500.

[0104] While FIG. 5 depicts the processor 502 and memory 504 of device 500 as integrated into a single unit, other configurations may be utilized. The operations of processor 502 may be distributed across multiple machines (each having one or more processors) that may be coupled directly or through a local area or other network. Memory 504 may be distributed across multiple machines, such as a network-based memory or the memory of multiple machines that perform the operations of device 500. While shown here as a single bus, bus 512 of device 500 may be comprised of multiple buses. Furthermore, secondary storage 514 may be directly coupled to other components of device 500 or may be accessible over a network, and may include a single integrated unit, such as a memory card, or multiple units, such as multiple memory cards. Thus, device 500 may be implemented in a wide variety of configurations.

[0105] The Versatile Video Coding (VVC) next-generation standard is the latest joint video project between the International Telecommunications Union Telecommunication Standardization Sector (ITU-T) Video Coding Experts Group (VCEG) and the International Organization for Standardization / International Electrotechnical Commission (ISO / IEC) Moving Picture Experts Group (MPEG), working together in a partnership known as the Joint Video Exploration Team (JVET). In VVC, a multi-type tree (quadratic / cubic / quaternary) tree (BT / TT / QT or binary / ternary / quadtree) segmentation structure replaces or potentially replaces the concept of multiple partition unit types. That is, the multi-type tree removes the separation of the concepts of CU, PU, ​​and TU, except when required for CUs whose size is too large for the maximum transform length, and supports flexibility in CU partition shapes [JVET-J1002].

[0106] Figures 6A-6F illustrate, by way of example, partition modes currently used in VTMs. Figure 6A illustrates an undivided block (no partitioning) of a block such as a CU or CTU, Figure 6Bb illustrates its quaternary or quadtree (QT) partitioning, Figure 6C illustrates its horizontal quadratic or binary tree (BT) partitioning, Figure 6D illustrates its vertical quadratic or binary tree (BT) partitioning, Figure 6E illustrates its horizontal tertiary or ternary tree (TT) partitioning, and Figure 6F illustrates its vertical tertiary or ternary tree (TT) partitioning. An embodiment may be configured to implement the partition modes illustrated in Figures 6A-6F.

[0107] In an embodiment, the following parameters may be defined and specified by the Sequence Parameter Set (SPS) syntax element for the BT / TT / QT coding tree scheme: CTU size: quadtree root node size; MinQTSize:Minimum allowed quadtree leaf node size; MaxBTTSize: Maximum allowed binary and ternary tree root node size; MaxBTTDepth: the maximum allowable binary and ternary tree depth; and MinBTTSize: The minimum allowed binary and ternary tree leaf node size.

[0108] In other embodiments, the minimum allowed quadtree leaf node size (MinQTSize) parameter may also be included in other headers or sets, for example, the slice header (SH) or picture parameter set (PPS).

[0109] In the HEVC standard, coding tree units (CTUs) or coding units (CUs) that lie on slice / picture boundaries are forced to split using a quadtree (QT) until the bottom-right sample of the leaf node falls within the slice / picture boundary. Because both the encoder and decoder, e.g., video encoder 20 and video decoder 30, know when to apply a forced QT, there is no need to signal a forced QT partition or partitioning in the bitstream. The purpose of the forced partition is to allow boundary CTUs / CUs by video encoder 20 / video decoder 30.

[0110] International Patent Publication No. WO2016 / 090568 discloses the QTBT (quadtree and binary tree) structure, and VTM1.0 also inherits the boundary CTU / CU forced partitioning process from HEVC. That is, CTUs / CUs at frame boundaries are forced to be partitioned by the quadtree (QT) structure without considering rate-distortion (RD) optimization until the entire current CU is within the picture boundary. These forced partitions are not signaled in the bitstream.

[0111] Figure 7A shows an example of a forced partition of a lower-boundary CTU (128x128) of a high-resolution (HD) (1920x1080 pixel) image partitioned by forced QT. In Figure 7, the HD image has or is 1920x1080 pixels, and the CTU has or is 128x128 pixels.

[0112] In SubCE2 (Image Boundary Processing) of CE1 (Partitioning) [JVET-J1021] at the San Diego Conference (April 2018), 15 tests for image boundary processing using BT, TT, or ABT (Asymmetric BT) were proposed. For example, in JVET-K0280 and JVET-K0376, the boundaries are defined as shown in Figure 8. Figure 8 shows the image boundary with a dashed line and the boundary case regions, namely the bottom boundary case, corner boundary case, and right boundary case regions, with solid lines. The bottom boundary can be partitioned with horizontally enforced BT or QT, the right boundary can be partitioned with vertically enforced BT or QT, and the corner cases can be partitioned only with QT. The decision of whether to use forced BT or QT partitioning is based on a rate-distortion optimization criterion and is signaled in the bitstream. Forced partitioning means that the blocks must be partitioned, for example, forced partitioning applies to boundary blocks that cannot be coded using "no partition" as shown in Figure 6A.

[0113] When forced QT splitting is used with forced boundary partitioning, the partitioning constraint of MinQTSize is ignored. For example, in Figure 9A, if MinQTSize is reported as 32 in the SPS, matching the boundaries with the forced QT method would require a QT partitioned into block sizes of 8x8, which would ignore the constraint of MinQTSize being 32.

[0114] According to embodiments of the present disclosure, when forced QT is used for image boundary partitioning, the forced QT partitioning follows partitioning constraints signaled, for example, in the SPS, and does not ignore them, for example. Furthermore, if forced partitioning is required, only forced BT is used, which may be combined and referred to as forced QTBT. In embodiments of the present disclosure, for example, the partition constraint MinQTSize is considered for forced QT partitioning at image boundaries, and no additional signaling is required for forced BT partitioning. The embodiments also enable harmonization of partitioning of regular (non-border) blocks and boundary blocks. For example, conventional solutions require two "MinQTSize" parameters, one for regular block partitioning and one for boundary block partitioning. The embodiments require only one common "MinQTSize" parameter for both regular block partitioning and boundary block partitioning, which can be flexibly configured between the encoder and decoder, for example, by signaling a single "MinQTSize" parameter. Furthermore, the embodiments require fewer partitions than, for example, forced QT.

[0115] Solutions for the lower and right bounding cases

[0116] For the bottom boundary case and the right boundary case, if the block size is larger than MinQTSize, the partition mode for image boundary partitioning can be selected between forced BT partitioning and forced QT partitioning, for example based on RDO. Otherwise (i.e., if the block size is smaller than or equal to MinQTSize), only forced BT partitioning is used for image boundary partitioning, more specifically, horizontal forced BT is used for the bottom boundary of the border block at the bottom boundary of the image, respectively, and vertical forced BT is used for the right boundary of the border block at the right boundary of the image, respectively.

[0117] The forced BT partitioning may include recursively partitioning the current block by horizontal enforced boundary partitioning until a subpartition of the current block is located at the bottom boundary of the image, and recursively partitioning the subpartitions by vertical enforced boundary partitioning until the leaf node is located entirely at the right boundary of the image. Alternatively, the forced BT partitioning may include recursively partitioning the current block by vertical enforced boundary partitioning until a subpartition of the current block is located at the bottom boundary, and recursively partitioning the subpartitions by horizontal enforced boundary partitioning until the leaf node is located entirely at the right boundary. MinQTSize may also be applied to control the partitioning of non-border blocks.

[0118] For example, in the case shown in Figure 7A, if MinQTSize is 32 or limited to 32, but the size of a rectangular (non-square) block with a height or width of 8 samples needs to match the image boundary, forced BT partitioning is used to partition the 32x32 blocks located on the boundary. BT partitions can be further partitioned using the same type of forced BT partitioning; for example, if vertical forced BT partitioning is applied, only further vertical forced BT partitioning is applied, and if horizontal forced BT partitioning is applied, only further horizontal forced BT partitioning is applied. Forced BT partitioning continues until the leaf node is completely within the image.

[0119] 7B illustrates an exemplary partitioning of a lower boundary CTU having a size of 128x128 samples according to one embodiment of the present invention. The lower boundary CTU, which forms the root block or root node of the partitioning tree, is partitioned into smaller partitions, e.g., small blocks of square or rectangular size. These smaller partitions or blocks may be further partitioned into even smaller partitions or blocks. In FIG. 7B, the CTU is a first quadtree partitioned into four square blocks 710, 720, 730, and 740, each of which is 64x64 samples in size. Of these blocks, blocks 710 and 720 are again lower boundary blocks, while blocks 730 and 740 are outside the image (located outside the image, respectively) and are not processed.

[0120] Block 710 is further partitioned using quadtree partitioning into four square blocks 750, 760, 770, and 780, each 32x32 samples in size. Blocks 750 and 760 are inside the image, while blocks 770 and 780 again form the bottom boundary blocks. Because the size of block 770 is no larger than MinQTSize, e.g., 32, recursive horizontal forced binary partitioning is applied to block 770 until the leaf node is completely within the image or is located completely within the image, e.g., until leaf node block 772, a rectangular non-square block with 32x16 samples, falls within the image (after one horizontal binary partitioning), or until leaf node block 774, a rectangular non-square block located at the bottom boundary of the image and with 32x8 samples, falls within the image (after two horizontal binary partitioning). The same applies to block 780.

[0121] Embodiments of the present disclosure allow for harmonization of partitioning of regular blocks located entirely inside an image and partitioning of boundary blocks. Boundary blocks are blocks that are neither entirely inside nor entirely outside an image. In other words, boundary blocks are blocks that consist of a portion that is inside the image and a portion that is outside the image. Furthermore, embodiments of the present disclosure allow for reduced signaling, since forced BT partitioning below MinQTSize does not need to be signaled.

[0122] Corner case solutions

[0123] In corner cases, some approaches only allow forced QT splitting and also ignore the MinQTSize constraint. The embodiments of the present disclosure provide two solutions for the corner cases.

[0124] Solution 1

[0125] The corner cases are considered to be the bottom boundary case or the right boundary case. Figure 10 shows an embodiment of the boundary definition. Figure 10 shows the image boundary with dashed lines and the region of the straight boundary case. As shown, the corner case is defined as the bottom boundary case. Thus, the solution is the same as that described for the bottom boundary case and the right boundary case above. In other words, first, horizontal partitioning is applied (as described for the bottom boundary case) until the block or partition is completely (vertically) within the image, and then vertical partitioning is applied (as described for the right boundary case) until the leaf node is completely (horizontally) within the image.

[0126] Solution 2

[0127] The boundary case convention remains the same: if the forced QT is constrained by MinQTSize (the current block size is less than or equal to MinQTSize), then the horizontal forced BT is used to match the bottom boundary, and if the bottom boundary is matched, then the vertical forced BT is used to match the right boundary.

[0128] For example, in FIG. 9A, which shows an embodiment of forced QTBT for blocks located at the corners of an image, if MinQTSize is 32 or is bounded as 32 for the corner case forced QT partition, then after the partition of the 32x32 block, a further BT partition is used until the forced partition ends.

[0129] FIG. 9B shows further details of an exemplary partitioning of a boundary CTU at or within a corner of an image, according to one embodiment of the present invention, where the CTU has a size of 128 x 128 samples. The CTU is initially quadtree partitioned into four square blocks, each 64 x 64 samples in size. Of these blocks, only the top-left block 910 is a boundary block; the other three are located outside (completely outside) the image and are not processed further. Block 910 is further partitioned using quadtree partitioning into four square blocks 920, 930, 940, and 950, each 32 x 32 samples in size. While block 920 is inside the image, blocks 930, 940, and 950 again form boundary blocks. Because the sizes of these blocks 930, 940, and 950 are not larger than MinQTSize (which is 32), forced binary partitioning is applied to blocks 930, 940, and 950.

[0130] Block 930 is located at the right boundary and is partitioned using recursive vertical forced binary partitioning until a leaf node falls within the image, e.g., block 932 at the right boundary of the image (here after two vertical binary partitionings).

[0131] Block 940 is located at the bottom boundary and is partitioned using recursive horizontal forced binary partitioning until a leaf node falls within the image, e.g., block 942 at the right boundary of the image (here after two horizontal binary partitionings).

[0132] Block 950 is located at a corner boundary and is partitioned using first recursive horizontal forced binary partitioning until a subpartition or block (here block 952) is located at the bottom boundary of the image (here after two horizontal binary partitions), and then recursively partition the subpartition by vertical forced boundary partitioning until a leaf node or block (e.g., block 954) is located at the right boundary of the image (here after two vertical binary partitions), or until the respective leaf node is located within the image.

[0133] The above approach is applicable to both decoding and encoding. In the case of decoding, MinQTSize can be received via SPS. In the case of encoding, MinQTSize can be transmitted via SPS. Embodiments can use boundary definitions such as those shown in Figure 8 or Figure 10 or other boundary definitions.

[0134] Further embodiments of the present disclosure are provided below: Note that the numbering used in the following paragraphs does not necessarily have to comply with the numbering used in the previous paragraphs. Embodiment 1: 1. A partitioning method, the partitioning method comprising: determining whether the current block of the image is a border block; If the current block is a boundary block, determining whether the size of the current block is greater than the minimum allowed quadtree leaf node size; If the size of the current block is not greater than the minimum allowed quad-tree leaf node size, applying a forced binary tree partitioning to the current block. Embodiment 2: A partitioning method as described in embodiment 1, wherein the forced binary tree partitioning is a recursive horizontal forced binary partitioning if the current block is located at the bottom boundary of the image, or a recursive vertical forced boundary binary partitioning if the current block is located at the right boundary of the image. Embodiment 3: Forced binary partitioning recursively partitioning the current block by horizontal forced boundary partitioning until a subpartition of the current block is located directly at the bottom boundary of the image; 3. A partitioning method as described in embodiment 1 or 2, comprising recursively partitioning the subpartitions by vertically forcing boundary partitioning until the leaf node is located entirely directly on the right boundary of the image, or vice versa. Embodiment 4: 4. A partitioning method as claimed in any one of embodiments 1 to 3, wherein the minimum allowed quadtree leaf node size is the minimum allowed quadtree leaf node size that also applies to control partitioning of non-border blocks. Embodiment 5: 5. A decoding method for decoding blocks by partitioning the blocks according to the partitioning method of any one of embodiments 1 to 4. Embodiment 6: 6. A decoding method as described in embodiment 5, wherein the minimum allowable quadtree leaf node size is received via an SPS. Embodiment 7: 5. An encoding method for encoding a block by partitioning the block according to the partitioning method of any one of embodiments 1 to 4. Embodiment 8: 8. The encoding method of embodiment 7, wherein the minimum allowable quadtree leaf node size is transmitted via SPS. Embodiment 9: 7. A decoding device comprising a logic circuit configured to perform any one of the methods of embodiment 5 or 6. Embodiment 10: 9. An encoding device comprising logic circuitry configured to perform any one of the methods of embodiment 7 or 8. Embodiment 11: A non-transitory storage medium for storing instructions, which when executed by a processor, cause the processor to perform any of the methods according to embodiments 1 to 8.

[0135] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing device. Computer-readable media may include computer-readable storage media, which correspond to tangible media such as data storage media, or communication media, including any medium that facilitates transfer of a computer program from one place to another, for example according to a communications protocol. In this manner, computer-readable media may generally correspond to (1) tangible computer-readable storage media that is non-transitory, or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, or data structures for implementing the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0136] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, Electrically Erasable Programmable Read-Only Memory (EEPROM), CD-ROM or other optical disk storage, magnetic disk storage, other magnetic storage, flash memory, or other media that can be used to store target program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, microwave, or the like, definition of medium includes the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, microwave, or the like. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but instead cover non-transitory, tangible storage media. As used herein, disk and disc include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks and Blu-ray discs, where disks typically reproduce data magnetically and discs reproduce data optically using lasers, and combinations of the above should also be included within the scope of computer-readable media.

[0137] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term "processor," as used herein, may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding or incorporated into a composite codec. Alternatively, these techniques may be implemented entirely in one or more circuits or logic elements.

[0138] The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units have been described in this disclosure to highlight functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, as noted above, the various units may be combined into a codec hardware unit or may be provided by a collection of interoperating hardware units including one or more processors as described above in combination with appropriate software and / or firmware.

[0139] The apparatus includes a memory element; and a processor element coupled to the memory element; wherein the processor element is configured to determine whether a current block of the image is a boundary block, and when the current block is a boundary block, determine whether a size of the current block is greater than a minimum allowable quad-tree (QT) leaf node size (MinQTSize), and apply forced binary tree (BT) partitioning to the current block if the size of the current block is not greater than MinQTSize.

[0140] While the present disclosure provides several embodiments, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. The present examples should be considered illustrative and not limiting, and the intention should not be limited to the details provided herein. For example, various elements or components may be combined or integrated into another system, or certain features may be omitted or not implemented.

[0141] Furthermore, the techniques, systems, subsystems, and methods described and illustrated in various embodiments as discrete or separate may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present disclosure. Other items illustrated or described as coupled or directly coupled or in communication with each other may be indirectly coupled or in communication, electrically, mechanically, or otherwise, through some interface, device, or intermediate component. Other examples of modifications, substitutions, and alterations will be ascertainable by those skilled in the art and may be made without departing from the spirit and scope disclosed herein.

[0142] Acronyms and Glossary HEVC High-Efficiency Video Coding VVC Versatile Video Coding VTM VVC Test Model JEM Joint Exploration Model CTU Coding Tree Unit CU Coding Unit BT Binary Tree TT Ternary Tree QT Quad Tree or Quaternary Tree ABT Asymmetric BT MMT Multi-type tree AMP Asymmetric Partition SH Slice Header SPS Sequence Parameter Set PPS Picture Parameter Set CE Core Experiment SubCE SubCore Experiment (part of the core experiment)

Claims

1. 1. A partitioning method, the partitioning method comprising: first partitioning the slice into coding tree units (CTUs); determining whether a current block of an image is a border block and whether the size of the current block is greater than a minimum allowed quadtree leaf node size (MinQTSize); applying a forced binary tree (BT) partitioning to the current block if the current block is the border block and the size of the current block is not greater than MinQTSize; Including, The forced binary tree partitioning is recursively partitioning the current block by horizontal forced boundary partitioning until a sub-partition of the current block is located at the bottom boundary of the image; recursively partitioning the sub-partitions by vertical forced boundary partitioning until a leaf node is located entirely on the right boundary of the image; the MinQTSize is used for both the partitioning of border blocks and the partitioning of non-border blocks; Partitioning method.

2. The partitioning method of claim 1 , wherein the forced binary tree partitioning continues until a leaf node block is within the image.

3. 3. A partitioning method according to claim 1 or 2, wherein the border blocks are blocks that are neither entirely inside the image nor entirely outside the image.

4. A decoding method for decoding a block by partitioning said block according to the partitioning method of any one of claims 1 to 3.

5. The decoding method of claim 4 , further comprising receiving the MinQTSize via a Sequence Parameter Set (SPS).

6. A method for encoding a block by partitioning said block according to the partitioning method of any one of claims 1 to 3.

7. The encoding method of claim 6 , further comprising transmitting the MinQTSize via a Sequence Parameter Set (SPS).

8. A decoding device comprising logic circuitry configured to decode blocks by partitioning said blocks according to the partitioning method of any one of claims 1 to 3.

9. 9. The decoding device of claim 8, wherein the logic circuitry is further configured to receive the MinQTSize via a sequence parameter set (SPS).

10. 4. An encoding device comprising logic circuitry configured to encode a block by partitioning said block according to the partitioning method of any one of claims 1 to 3.

11. A non-transitory storage medium for storing instructions that, when executed by a processor, cause the processor to perform any of the methods of any one of claims 1 to 7. Non-transitory storage media.

Images