Image and video coding and decoding

Abstract

This section describes improvements to the process of splitting image data for coding and / or decoding images and / or videos. The bitstream contains data indicating that the image data should be split into multiple blocks according to a coding tree, and the blocks in the coding tree may be split according to multiple partitions, including quadtree partitions. A value is determined that indicates the depth of the quadtree partition for the current block. If the value indicating the depth of the quadtree partition does not exceed a threshold, only the quadtree partition for the current block is allowed among the multiple partitions.

Description

Technical Field

[0001] The present invention relates to the encoding and decoding of segmented data of images and videos.

Background Art

[0002] A cooperation team formed by MPEG and VCEG of ITU-T Study Group 16, the Joint Video Experts Team (JVET), announced a new video coding standard called VVC (Versatile Video Coding). The goal of VVC is to provide a significant improvement in compression performance that exceeds the existing HEVC standard (i.e., typically twice that of the previous one). The main target applications and services include, but are not limited to, 360-degree and high dynamic range (HDR) videos. It has shown particular effectiveness for ultra-high definition (UHD) video test materials. Therefore, an improvement in compression efficiency far exceeding the target 50% of the final standard can be expected.

[0003] Since the completion of the standardization of VVC v1, JVET has started the exploration phase by establishing exploration software (ECM). This collects additional tools and improvements to existing tools in addition to the VVC standard in order to target better coding efficiency.

Summary of the Invention

[0004] According to a first aspect of the present invention, there is provided a method of encoding image data into a bitstream including data indicating dividing the image data into a plurality of blocks according to a coding tree, or decoding the image data from the bitstream, wherein the blocks in the coding tree can be divided according to a plurality of divisions including quadtree division. The method includes determining a value indicating the depth of the quadtree division of the current block, and restricting possible divisions of the current block based on the depth of the (current) quadtree and at least one other value (e.g., when the other value represents some kind of threshold).

[0005] Broadly speaking, the advantages include reduced encoder complexity due to a reduction in the number of partitions evaluated on the encoder side (limited based on the depth of the quadtree partitioning), and improved coding efficiency in the decoder due to reduced signaling of the required partitions, thus making the decoding process more efficient.

[0006] A second aspect of the present invention provides a method for encoding image data into a bitstream containing data indicating that the image data is to be divided into a plurality of blocks according to a coding tree, or for decoding image data from said bitstream, wherein the blocks in the coding tree may be divided according to a plurality of partitions, including a quadtree partition, and the method comprises determining a value indicating the depth of the quadtree partition of the current block, and if the value indicating the depth of the quadtree partition does not satisfy one or more conditions, allowing only the quadtree partition of the current block from among the plurality of partitions. For example, the value indicating the depth of the quadtree partition does not exceed a threshold (value). The advantage is reduced encoding time, as there is no need to evaluate partitions for QT depths smaller than this value. A second advantage is improved coding efficiency, as there are no relative syntax elements before QT depths smaller than the value need to be transmitted. However, of course, it is a compromise between the preserved bitrate and quality. Nevertheless, surprisingly, the proposed modification yields benefits.

[0007] Multiple partitions may further include a triple tree partition, a double tree partition, and no partition at all.

[0008] A third aspect of the present invention provides a method for encoding image data into a bitstream containing data indicating that the image data is to be divided into a plurality of blocks according to a coding tree, wherein the blocks in the coding tree may be divided according to a plurality of divisions, including double-tree division, triple-tree division, quadrutree division, and no division, and the method comprises determining a value indicating the depth of the quadrutree division of the current block and not allowing a double-tree division based on one or more criteria, the criteria including that the value indicating the depth of the quadrutree division does not satisfy one or more conditions. For example, the value indicating the depth of the quadrutree division does not exceed a threshold (value). If the QT depth is less than this value, only the no-division mode needs to be evaluated and only the nodes of the QT division need to be evaluated, so the advantage is reduced coding time. A second advantage is improved coding efficiency, as only the no-division flag needs to be sent before the QT depth is less than this value.

[0009] This method may further allow quadtree splitting, triple tree splitting, and no splitting if the value indicating the depth of the quadtree splitting does not exceed a threshold.

[0010] Alternatively, this method may further include not allowing triple tree splitting, and allowing quadtree splitting and no splitting if the value indicating the depth of the quadtree splitting does not exceed a threshold.

[0011] Optionally, based on further criteria, only quadtree splitting and no splitting are allowed, while duntree splitting or quadtree splitting is not permitted, and TT splitting is also allowed if the conditions are met.

[0012] A further optional feature of the third aspect may include one or more criteria that triple tree splitting is not performed at a level higher than the current block level, that if triple tree splitting is not performed, double tree splitting is not permitted, and that if triple tree splitting is performed, double tree splitting of the current block is permitted.

[0013] This method may include performing the first method according to any embodiment of the second aspect and performing the second method according to any embodiment of the third aspect, wherein the threshold for allowing only quadtree splitting is the first threshold, and the threshold for not allowing binary tree splitting is the second threshold, and the first threshold and the second threshold are different.

[0014] Optionally, the second method further comprises a further optional feature of the third embodiment, and further comprises performing the third method according to the third embodiment, wherein the threshold of the third method is a third threshold different from the first and second thresholds.

[0015] Optionally, in any of the above embodiments and examples, the value indicating the depth of the quadtree partitioning may be the quadtree partitioning depth value, and if it is less than a predetermined quadtree partitioning depth value, it is said to exceed the threshold.

[0016] The value indicating the depth of the quadtree partitioning may, alternatively or additionally, be related to or be the block size of the current block, and if it is larger than a given block size, it may be said to exceed the threshold.

[0017] A threshold (or a set of values) is obtained based on one or more parameters associated with a (time) area in one or more time frames, where the time frame is a different frame from the frame containing the current block. The time frame can be a reference frame.

[0018] The (time) area may be an area that is collocated with the current block.

[0019] A (time) area may have multiple blocks at different locations.

[0020] The (time) area may be an area that is larger in size than the current block.

[0021] (Time) area may have a size larger than the current block in the coding tree unit CTU.

[0022] The center position of the current block may be used to determine the (time) area within the time frame or within one time frame.

[0023] (Time) area may encompass the entire area of the time frame or one time frame.

[0024] The time frame or one time frame may be a frame having the same time ID as the current frame containing the current block. The frame having the same time ID may be the closest frame having the same time ID.

[0025] The time frame or one time frame may be a frame having the same quantization parameter as the current frame containing the current block.

[0026] The time frame or one time frame may be a frame used for (time) motion vector prediction.

[0027] The time frame or one time frame may be a frame that is the closest reference frame to the current frame containing the current block.

[0028] The first area from the first time frame and the second area from the second time frame may be used to obtain a threshold value.

[0029] The threshold value (or values) may be based on the average quadtree depth determined from the (time) area.

[0030] The threshold value (or values) can be (based on) the minimum quadtree depth determined from the (time) area.

[0031] The threshold (or multiple values) may be determined based on the maximum or average multitree depth from a given time area.

[0032] The threshold can be determined using the maximum or average coding tree depth from a given time area.

[0033] The threshold (or a set of values) may be based on the values ​​sent in the header associated with the coding tree unit of the current block.

[0034] The values ​​sent in the header of the current coding tree unit are used, along with one or more other values ​​sent in the headers associated with other coding tree units, to predict a threshold (or one threshold).

[0035] The values ​​sent in the header may be used to determine multiple or individual thresholds.

[0036] Optionally, if a condition is met, one of the criteria will not apply. For example, if the maximum multitree depth for a time frame is lower than the current maximum multitree depth, the first criterion will not apply.

[0037] The following embodiments of the present invention can be used independently or in combination with any of the embodiments described above.

[0038] This method may involve obtaining the threshold or each threshold by adding an integer offset.

[0039] The aforementioned threshold or one threshold may be adapted based on the slice type corresponding to the time area in the reference frame.

[0040] The aforementioned threshold or one threshold may be adapted based on the maximum multitree depth and maximum multitree depth of the time area frame.

[0041] The aforementioned threshold or one threshold may be adapted based on the maximum multitree depth and the maximum multitree depth time area.

[0042] The aforementioned threshold or one threshold may be based on the minimum quadtree size of the time area frame, or adapted to the minimum, maximum, or average quadtree size of the time area.

[0043] The aforementioned threshold or one threshold may be adapted depending on whether the current frame has a lower QP than the frame containing the time area.

[0044] The aforementioned threshold or one threshold may be applied depending on whether the current frame containing the current block is a reference frame or not.

[0045] Optionally, the threshold or one threshold is based on statistics of block size within a time area.

[0046] The parameters used to determine the aforementioned threshold or a single threshold may be obtained from a coding tree unit-level block.

[0047] Optionally, if the image data associated with a low-latency configuration is used, or if at least one flag is transmitted in at least one header associated with the image, one or more method steps are not performed.

[0048] Optionally, one or more method steps are not performed if the image data is related to screen content.

[0049] Optionally, not performing one or more steps or features does not enable or disable such features. Not performing may apply to one or more blocks in the image, or to any other predetermined region or portion of the image.

[0050] Optionally, one or more method steps are not performed if the image data is related to screen content and one or more reference frames are intraframes.

[0051] Image data may be considered related to screen content if one or more tools are signaled and / or enabled in the bitstream, and one or more tools may include any one of the following tools related to screen content: a conversion skip tool, an intrablock copy (IBC), and a palette.

[0052] One or more tools may be enabled according to the data associated with the hierarchy level in the bitstream.

[0053] The data may consist of one or more of the following: a sequence parameter set (SPS), a picture parameter set (PPS), a picture header, and a slice header.

[0054] One or more tools may signal and / or enable for at least one block of an image or reference frame (therefore indicating screen content).

[0055] One or more tools may be signaled and / or enabled for at least one block in the time area of ​​the reference frame.

[0056] Optionally, one or more method steps are not performed if the number of blocks having one or more enabled and / or signaled tools is greater than a predetermined number.

[0057] Optionally, one or more steps that are not performed are one or more steps related to at least one of the first and second methods described in the embodiments and aspects above.

[0058] Failure to perform one or more steps of the second method includes disabling at least one of the steps, one or more criteria including that a triple tree split is not performed at a level higher than the current block level, and if a triple tree split is not performed, not allowing a double tree split, and if a triple tree split is performed, allowing a double tree split for the current block, and i) determining a value indicating the depth of a quadtree split for the current block, and not allowing a double tree split based on one or more criteria, the criterion or one criterion including that the value indicating the depth of a quadtree split does not exceed a threshold, ii) allowing a quadtree split, a triple tree split, or no split if the value indicating the depth of a quadtree split does not exceed a threshold, and iii) not allowing a triple tree split and allowing a quadtree split, no split if the value indicating the depth of a quadtree split does not exceed a threshold.

[0059] Optionally, the threshold or one threshold is based on statistics of block size within a time area.

[0060] The parameters used to determine the aforementioned threshold or a single threshold may be obtained from a coding tree unit-level block.

[0061] A fourth aspect of the present invention provides a method for encoding image data into a bitstream containing data indicating that the image data is to be divided into a plurality of blocks according to a coding tree, and for decoding image data from said bitstream, wherein the blocks in the coding tree may be divided according to a plurality of divisions, including triple-tree and double-tree divisions, and the method comprises determining whether a triple-tree division has been performed at a level higher than the current block level, and if a triple-tree division has been performed, allowing a double-tree division for the current block. The advantage of this aspect is an improvement in overall coding efficiency with only a very small increase in coding time.

[0062] The determination can be made by obtaining the multi-tree depth value at the level above the current block.

[0063] A fifth aspect of the present invention provides a method for encoding image data into a bitstream that includes data indicating that the image data is to be divided into a plurality of blocks according to a coding tree, wherein the blocks in the coding tree may be divided according to a plurality of divisions, including a double-tree division, a quadtree division, and no division, and the double-tree division may be one of a horizontal and a vertical division, and the method has the effect of performing a control such that a quadtree division is tested for the current block before a double-tree division if it is determined that both a double-tree horizontal and a double-tree vertical division are not permitted to determine the division of the current block in the coding tree.

[0064] This provides an improvement in coding efficiency with or without the proposed method, without an increase in coding time. This is due to the fact that encoder optimizations involve avoiding testing QT when BT is detected better. Therefore, these optimizations are inefficient if TT is the only available MTT split.

[0065] Optionally, quadtree splitting is tested before all splits except for no splitting of the current block.

[0066] According to a sixth aspect of the present invention, a bitstream is provided which includes data indicating that image data is to be divided into a plurality of blocks according to a coding tree, the blocks in the coding tree may be divided according to a plurality of divisions which may be one of horizontal and vertical divisions, and the method is characterized in that if it is determined that no division is available, the method does not include a division in the testing list for the current block of the coding tree. This results in an improvement in coding efficiency without an increase in coding time, with or without the proposed method of the preceding aspects or embodiments.

[0067] In a first high-level syntax aspect of the present invention, a method for decoding or encoding video or image data from or into a bitstream is provided, and a (high-level) syntax element (flag) of the entire sequence is transmitted to enable or disable any of the above-described aspects or embodiments. For example, a flag controlling whether the method is enabled or not may be transmitted within the SPS.

[0068] A second high-level syntax aspect of the present invention provides a method for decoding or encoding video or image data from or into a bitstream, wherein a (high-level) syntax element (flag) of a set of pictures is transmitted or retrieved to enable or disable transmission, in order to enable or disable any of the above-described aspects or embodiments. For example, a flag may be transmitted within a PPS to control activation.

[0069] A third high-level syntax aspect of the present invention provides a method for decoding or encoding video or image data from or into a bitstream, wherein a (high-level) syntax element (flag) for each picture is transmitted or retrieved to enable or disable any of the above-described aspects or embodiments. For example, the flag is transmitted within the picture header (PH).

[0070] A fourth high-level syntax aspect of the present invention provides a method for decoding or encoding video or image data from or to a bitstream, where high-level syntax elements (flags) are transmitted or retrieved at a lower level in a slice or tile to enable or disable any of the above aspects or embodiments. For example, the flags are transmitted within the slice header (SH).

[0071] According to the embodiment, a flag indicating the result of a threshold determination is transmitted and / or retrieved. In other words, a flag is transmitted or retrieved to indicate each criterion that is met.

[0072] The advantage of the embodiments and configurations of high-level syntax signaling is the encoder's flexibility in determining the best compromise.

[0073] Furthermore, it will be understood that the above embodiments can be combined where feasible to form other embodiments according to the present invention.

[0074] For example, a further aspect of the present invention provides a method for encoding image data into a bitstream containing data indicating that the image data is to be divided into a plurality of blocks according to a coding tree, or for decoding image data from said bitstream, wherein the blocks in the coding tree may be divided according to a plurality of divisions, including a quadtree division, and the method comprises: determining a value indicating the depth of the quadtree division of the current block; allowing only the depth of the quadtree division of the current block from a plurality of modes; and if the depth of the quadtree division of the current block is less than a first (threshold) value (first criterion) based on the minimum quadtree depth of the area of ​​the reference frame and less than a second (threshold) value (second criterion) based on a value indicating the average quadtree depth of the area of ​​the reference frame, then i) allowing a quadtree division, ternary division, or no division; and optionally, ii) further allowing a binary division if a ternary division has been used at a higher level in the coding tree of the current coding unit.

[0075] In another aspect of the present invention, for each block, the permissible value of the partition is predicted according to the minimum QT partition and the average QT partition obtained from the time area. Thus, if the current QT depth is less than the time minimum QT depth minus 1, only QT partitions are allowed; if the current QT depth is less than the time average QT depth minus 1, no partition, QT partition, and TT partition are allowed, and if TT is selected in the parent node, BT partitions are allowed.

[0076] In at least some aspects and embodiments, the inventors refer to binary partitioning. Such binary partitioning may include horizontal binary partitioning and / or vertical binary partitioning. In some aspects and embodiments, the inventors refer to ternary partitioning. Such ternary partitioning may include horizontal ternary partitioning and / or vertical ternary partitioning.

[0077] Furthermore, the embodiments described above refer to possible partitions of a coding unit or CTU, such as no partition, quadtree, trebletree (horizontal and vertical), and diplextree (horizontal and vertical), and it should be understood that the present invention is not limited in this way and other modes may be considered. For example, other geometric partitions may be considered in a different number of blocks and may be limited according to one or more criteria mentioned in the above embodiments and aspects.

[0078] Furthermore, the terms binary partitioning, ternary partitioning, and quadtree partitioning are used, and these are generally understood to refer to modes of partitioning in which a block (for example, a coding unit or coding tree unit) is divided or split into two, three, or four subblocks, respectively.

[0079] Other aspects of the present invention relate to corresponding encoding methods, encoding devices, decoding devices, and computer programs that are operable to perform the decoding and / or encoding methods of the present invention.

[0080] Further aspects of the present invention are provided by independent and dependent claims.

[0081] The program may be provided by itself, or it may be carried on, by, or within a carrier medium. The carrier medium may be non-temporary, for example, a storage medium, in particular a computer-readable storage medium. The carrier medium may also be temporary, for example, a signal or other transmission medium. The signal may be transmitted over any suitable network, including the Internet. Further features of the present invention are characterized by the independent and dependent claims.

[0082] Any feature in one aspect of the present invention may be applied to other aspects of the present invention in any suitable combination. In particular, aspects of a method may be applied to aspects of an apparatus, and vice versa.

[0083] Furthermore, features implemented in hardware may also be implemented in software, and vice versa. Any reference to software and hardware features in this specification should be interpreted accordingly.

[0084] Any apparatus feature described herein may also be provided as a method feature, and vice versa. Where used herein, means-plus-function features may be expressed alternatively with respect to their corresponding structures, such as a well-programmed processor and associated memory.

[0085] Furthermore, it should be understood that specific combinations of the various features described and defined in any aspect of the present invention may be independently implemented and / or supplied and / or used.

[0086] Further aspects of the present invention are provided by independent and dependent claims. [Brief explanation of the drawing]

[0087] For example, please refer to the attached drawing: [Figure 1] Figure 1 is a diagram illustrating the coding structure used in HEVC. [Figure 2] Figure 2 is a schematic block diagram showing a data communication system in which one or more embodiments of the present invention may be implemented. [Figure 3] Figure 3 is a block diagram showing the components of a processing apparatus in which one or more embodiments of the present invention may be implemented. [Figure 4] Figure 4 is a schematic diagram showing the functional elements of an encoder according to an embodiment of the present invention. [Figure 5] Figure 5 is a schematic diagram showing the functional elements of a decoder according to an embodiment of the present invention. [Figure 6] Figure 6 shows the blocks placed relative to the current block, including the juxtaposed blocks. [Figure 7] Figure 7 shows the time-random-access GOP structure for 33 frames with associated time IDs and POCs. [Figure 8] Figure 8 shows the six possible splitting modes of VVC. [Figure 9] Figure 9 shows maxBtSize and maxMttDepht. [Figure 10] Figure 10 shows an example of the minQtSize variable. [Figure 11] Figure 11 shows some of the partitioning constraints. [Figure 12] Figure 12 shows an incomplete CTU at the frame boundary. [Figure 13] Figure 13 shows the encoding based on the setting of maxMttDepth based on the time ID. [Figure 14] Figure 14 shows an embodiment of the present invention. [Figure 15] Figure 15 shows an embodiment of the present invention. [Figure 16] Figure 16 shows one or several time positions. [Figure 17] Figure 17 shows a system having an encoder or decoder and a communication network according to an embodiment of the present invention. [Figure 18]Figure 18 is a schematic block diagram of a computing device for implementing one or more embodiments of the present invention. [Figure 19] Figure 19 shows a network camera system. [Figure 20] Figure 20 shows a smartphone. [Modes for carrying out the invention]

[0088] Figure 1 illustrates the coding structure used in HEVC (High Efficiency Video Coding) and VVC (Versatile Video Coding) standards. Video sequence 1 consists of a series of digital images i. Each such digital image is represented by one or more matrices. Matrix coefficients represent pixels.

[0089] Image 2 of the sequence can be divided into slice 3. In some examples, a slice may constitute an entire image. These slices are divided into non-overlapping coding tree units (CTUs). A coding tree unit (CTU) is a fundamental processing unit in the HEVC (High Efficiency Video Coding) and VVC (Versatile Video Coding) video standards and conceptually and structurally corresponds to the macroblock units used in some earlier video standards. A CTU is sometimes called an LCU (Largest Coding Unit). A CTU has a luminous component portion and a chroma component portion, and each component portion is called a CTB (Coding Tree Block). These different color components are not shown in Figure 1.

[0090] A CTU is generally 64x64 pixels in the case of HEVC, but in the case of VVC, this size can be 128x128 pixels. Each CTU can be iteratively divided into smaller variable-size coding units (CUs) 5 using quadtree (QT) decomposition.

[0091] A coding unit is a fundamental coding element and consists of two types of subunits called PU (Prediction Unit) and TU (Transform Unit). The maximum size of a PU or TU is equal to the size of the CU. The prediction unit corresponds to a partition of the CU for predicting pixel values. As shown in 6, various different partitions of the CU into PUs can include a partition into four square PUs and two different partitions into two rectangular PUs. The transformation unit is a fundamental unit that undergoes a spatial transformation using DCT. The CU can be partitioned into TUs based on a quadtree representation 7.

[0092] Each slice is embedded in a single Network Abstraction Layer (NAL) unit. Furthermore, the coding parameters of a video sequence are stored in a dedicated NAL unit called a parameter set. HEVC and H.264 / AVC use two types of parameter set NAL units: firstly, the SPS (Sequence Parameter Set) NAL unit, which collects all parameters that do not change throughout the entire video sequence. Typically, it handles the coding profile, video frame size, and other parameters. Secondly, the PPS (Picture Parameter Set) NAL unit contains parameters that may change from one image (or frame) in the sequence to another. HEVC also includes a VPS (Video Parameter Set) NAL unit, which contains parameters describing the overall structure of the bitstream. VPS is a type of parameter set defined in HEVC and applies to all layers of the bitstream. A layer can contain multiple time sublayers, while all version 1 bitstreams are limited to a single layer. HEVC has specific layered extensions for scalability and multi-view, which enable multiple layers with backward-compatible version 1 base layers.

[0093] Another method of dividing images is introduced in VVCs, which include subpictures, which are independently coded groups of one or more slices.

[0094] Figure 2 shows a data communication system in which one or more embodiments of the present invention may be implemented. The data communication system has a transmitting device, in this case a server 201, which is capable of transmitting data packets of a data stream to a receiving device, in this case a client terminal 202, via a data communication network 200. The data communication network 200 may be a WAN (Wide Area network) or a LAN (Local Area Network). Such a network may be, for example, a wireless network (Wifi / 802.11a or b or g), an Ethernet® network, an Internet network, or a mixed network consisting of several different networks. In a particular embodiment of the present invention, the data communication system may be a digital television broadcasting system in which the server 201 transmits the same data content to multiple clients.

[0095] The data stream 204 provided by the server 201 may consist of multimedia data representing video and audio data. In some embodiments of the present invention, the audio and video data streams may be captured by the server 201 using a microphone and a camera, respectively. In some embodiments, the data streams may be stored in the server 201, received by the server 201 from another data provider, or generated in the server 201. The server 201 particularly includes encoders for encoding the video and audio streams to provide compressed bitstreams for transmission in a more compact representation of the data presented as input to the encoder.

[0096] To obtain a better ratio between the quality of transmitted data and the amount of transmitted data, video data compression may follow, for example, the HEVC format, H.264 / AVC format, VVC format, or the format of data generated by ECM.

[0097] Client 202 receives the transmitted bitstream, decodes the reconstructed bitstream, and plays the video image on the display device and the audio data through the loudspeaker.

[0098] While the example in Figure 2 considers a streaming scenario, it will be understood that in some embodiments of the present invention, data communication between the encoder and decoder may be performed using a storage medium such as an optical disc.

[0099] In one or more embodiments of the present invention, a video image is transmitted along with data representing a compensation offset, which is applied to the reconstructed pixels of the image to provide filtered pixels in the final image.

[0100] Figure 3 schematically shows a processing unit 300 configured to carry out at least one embodiment of the present invention. The processing unit 300 may be a device such as a microcomputer, a workstation, or a light portable device. The device 300 has a communication bus 313 to which the following is connected: - A central processing unit such as a microprocessor labeled CPU 311; - A read-only memory 306, indicated by ROM, for storing a computer program for carrying out the present invention; - Random access memory 312, designated RAM, which stores registers adapted to record variables and parameters necessary to carry out the executable code for the method of an embodiment of the present invention, and the method for encoding a sequence of digital images and / or decoding a bitstream according to an embodiment of the present invention; - A communication interface 302 connected to a communication network 303 through which the digital data to be processed is transmitted and received. Optionally, the device 300 may also include the following components: - A computer program for carrying out a method of one or more embodiments of the present invention, and data storage means such as a hard disk for storing data used or generated during the implementation of one or more embodiments of the present invention; - A disk drive 305 for disk 306, the disk drive being adapted to read data from or write data to disk 306; - A screen 309 that serves as a graphical interface with the user and / or displays data, via a keyboard 310 or any other pointing means.

[0101] The device 300 can be connected to various peripheral devices, such as a digital camera 320 or a microphone 308, each of which is connected to an input / output card (not shown) to supply multimedia data to the device 300.

[0102] The communication bus provides communication and interoperability between various elements included in or connected to the device 300. The representation of the bus is not limited, and in particular, the central processing unit can operate to communicate instructions directly to any element of the device 300 or through another element of the device 300.

[0103] The disk 306 can be replaced with any information medium, such as a compact disc (CD-ROM), rewritable or non-rewritable, ZIP disc, or memory card, and can generally be replaced with information storage means that can be read by a microcomputer or microprocessor, and which may be incorporated into or not incorporated into the device, or which may be removable and adapted to store one or more programs, and which, by execution, enables the implementation of the method for decoding a bitstream and / or encoding a series of digital images according to the present invention.

[0104] The executable code may be stored in read-only memory 306, hard disk 304, or a removable digital medium such as disk 306 as described above. In a modified example, the executable code of a program may be received by the communication network 303 via interface 302 to be stored in one of the storage means of device 300, such as hard disk 304, before being executed.

[0105] The central processing unit 311 is adapted to control and direct the execution of a set of programs or a portion of the program instructions or software code according to the present invention, based on instructions stored in one of the above-described storage means. When power is turned on, the set of programs or programs stored in non-volatile memory, such as the hard disk 304 or read-only memory 306, is transferred to random access memory 312, which includes the executable code of the program or set of programs, as well as registers for storing variables and parameters necessary to carry out the present invention.

[0106] In this embodiment, the device is a programmable device that uses software to carry out the present invention. However, the present invention may also be implemented in hardware (for example, in the form of an application-specific integrated circuit or ASIC).

[0107] Figure 4 shows a block diagram of an encoder according to at least one embodiment of the present invention. The encoder is represented by connected modules, each module adapted to perform at least one corresponding step of a method for performing at least one embodiment of encoding an image of a sequence of images according to one or more embodiments of the present invention, for example, in the form of program instructions executed by the CPU 311 of the device 300.

[0108] The original sequence of digital images i0 to in401 is received as input by encoder 400. Each digital image is represented by a set of samples, sometimes also called pixels (hereinafter referred to as pixels).

[0109] The bitstream 410 is output by the encoder 400 after the encoding process is performed. The bitstream 410 has multiple encoding units or slices, each slice having a slice header for transmitting encoded values ​​of encoding parameters used to encode the slice, a slice body, and encoded video data.

[0110] Input digital images i0~i n 401 is divided into blocks of pixels by module 402. The blocks correspond to parts of the image and can be of variable size (for example, 4x4, 8x8, 16x16, 32x32, 64x64, 128x128 pixels, and several rectangular block sizes may also be considered). A coding mode is selected for each input block. Two families of coding modes are provided: coding modes based on spatial prediction coding (intra prediction) and coding modes based on temporal prediction (intercoding, merge, SKIP). Possible coding modes are tested.

[0111] Module 403 performs an intra-prediction process in which a given block to be coded is predicted by predictors calculated from neighboring pixels of the block to be coded. If intra-coding is selected, the selected intra-predictors and the indication of the difference between a given block and its predictors are coded to provide a residual.

[0112] Time prediction is performed by motion estimation module 404 and motion compensation module 405. First, from the set of reference images 416, the motion estimation module 404 selects the portion of the reference image, also called the reference area or image portion, that is closest (closest in pixel value similarity) to a given block to be encoded. Next, the motion compensation module 405 uses the selected area to predict the block to be encoded. The difference between the selected reference area and the given block, also called the residual block, is calculated by the motion compensation module 405. The selected reference area is represented using a motion vector.

[0113] Therefore, in both cases (spatial and temporal prediction), the residual is calculated by subtracting the predictor from the original block.

[0114] In the INTRA prediction performed by module 403, the prediction direction is encoded. In the inter prediction performed by modules 404, 405, 416, 418, and 417, at least one motion vector or data is encoded for time prediction to identify such motion vectors.

[0115] If interpretation is selected, information related to the motion vector and residual block is encoded. To further reduce the bitrate, assuming uniform motion, the motion vector is encoded by the difference with respect to the motion vector predictor. The motion vector predictor from the set of motion information predictor candidates is obtained from the motion vector field 418 by the motion vector prediction coding module 417.

[0116] The encoder 400 further includes a selection module 406 for selecting a coding mode by applying coding cost criteria such as rate distortion criteria. To further reduce redundancy, a transformation (such as DCT) is applied to the residual block by a transformation module 407, and the acquired transformed data is then quantized by a quantization module 408 and entropy coded by an entropy coding module 409. Finally, the coded residual block of the currently coded block is inserted into the bitstream 410.

[0117] The encoder 400 also performs decoding of the encoded image to generate a reference image (e.g., a reference image in reference image / picture 416) for motion estimation of subsequent images. This allows the encoder and the decoder receiving the bitstream to have the same reference frame (a reconstructed image or a portion of the image is used). The inverse quantization ("inverse quantization") module 411 performs inverse quantization ("inverse quantization") of the quantized data, followed by the inverse transform module 412. The intra-prediction module 413 uses the prediction information to determine which predictor should be used for a given block, and the motion compensation module 414 actually adds the residuals obtained by module 412 to the reference area obtained from the set of reference images 416.

[0118] Next, post-filtering is applied by module 415 to filter the reconstructed frame (image or portion of an image) of pixels. In embodiments of the present invention, an SAO loop filter is used, and a compensation offset is added to the pixel values ​​of the reconstructed pixels in the reconstructed image. It is understood that post-filtering is not necessarily required. In addition to or instead of SAO loop filtering, any other type of post-filtering may be performed.

[0119] Figure 5 shows a block diagram of a decoder 60 that may be used to receive data from an encoder, according to one embodiment of the present invention. The decoder is represented by connected modules, each module adapted to perform the corresponding steps of the method performed by the decoder 60, for example, in the formation of program instructions executed by the CPU 311 of the device 300.

[0120] Decoder 60 receives a bitstream 61 having encoded units (e.g., data corresponding to blocks or coding units), each consisting of a header containing information about encoding parameters and a body containing encoded video data. As described with reference to Figure 4, the encoded video data is entropy encoded, and the indices of the motion vector predictors are encoded for a given block with a predetermined number of bits. The received encoded video data is entropy decoded by module 62. The residual data is then inversely quantized by module 63, and then the inverse transform is applied by module 64 to obtain the pixel values.

[0121] Mode data indicating the coding mode is also entropy-decoded, and based on that mode, INTRA-type decoding or INTER-type decoding is performed on the coded blocks (units / sets / groups) of the image data.

[0122] In INTRA mode, the INTRA predictor is determined by the INTRA prediction module 65 based on the INTRA prediction mode specified in the bitstream.

[0123] When the mode is INTER, motion prediction information is extracted from the bitstream to find (identify) the reference area used by the encoder. The motion prediction information includes the reference frame index and the motion vector residual. The motion vector predictor is added to the motion vector residual by the motion vector decoding module 70 to obtain the motion vector. The various motion prediction tools used in VVC are described in more detail below with reference to Figures 6 to 10.

[0124] The motion vector decoding module 70 applies motion vector decoding to each current block encoded by motion prediction. Once the index of the motion vector predictor for the current block is obtained, the actual value of the motion vector associated with the current block is decoded and can be used by module 66 to apply motion compensation. The reference image portion indicated by the decoded motion vector is extracted from the reference image 68 and motion compensation 66 is applied. The motion vector field data 71 is updated with the decoded motion vector for use in predicting subsequent decoded motion vectors.

[0125] Finally, the decoded block is obtained. If appropriate, post-filtering is applied by the post-filtering module 67. The decoded video signal 69 is finally obtained and provided by the decoder 60.

[0126] Random access configuration Figure 7 shows the time random access GOP structure for 33 consecutive frames 0-32. The length of the vertical line representing each frame corresponds to its time ID (for example, the longest length corresponds to time ID 0, and the shortest length corresponds to time ID 5). A frame with time ID 0 is the highest in the temporal hierarchy because it can be decoded independently of all other frames with higher time ID values. Similarly, a frame with time ID 1 is second in the temporal hierarchy, and they can be decoded independently of all other frames with higher time IDs, and so on. In other words, a frame with a particular time ID can be decoded independently of frames with higher time IDs, but may depend on frames with lower time IDs. This is known as temporal scalability.

[0127] This parameter is similar to the hierarchical depth, but the hierarchical depth does not imply independent decoding for all other frames with higher depths.

[0128] VVC split VVC partitioning has a specific block partitioning. For a single tree node, six possible partitions are possible, as shown in Figure 8: - Quad division QT,801 divides a block into four equal-sized square blocks. -Binary partition BT with two possible subdivisions 802, 803: - Vertical binary split, 802, SPLIT_BT_VER - Horizontal binary split, 803, SPLIT_BT_HOR - The block is divided into three blocks with a larger bandwidth in the center, and the two possible subdivisions TT have 804 and 805: - Vertical ternary split, 804, SPLIT_TT_VER - Horizontal ternary split, 805, SPLIT_TT_HOR - Terminate tree nodes without splitting, No Split, 806.

[0129] In this explanation, a block can be a CTU and / or CU in the coding tree, or more generally, any unit.

[0130] VVC partitioning control variables For the current block, not all possible partitions are always allowed. Which partitions are available depends on several conditions. These conditions depend on several defined partition control variables. The first set of variables defines the maximum and minimum block / node sizes: • CTU size: Corresponds to the root node size of a quadtree (e.g., 256x256, 128x128, 64x64, 32x32, 16x16 sample); • maxBtSize: This is the maximum allowable size of a bilingual root node, i.e., the maximum size of a leaf quadtree node that can be partitioned by binary partitioning. If both the height and width of the current block are less than or equal to maxBtSize, the current block can be partitioned by BT partitioning. Figure 9 illustrates the concept of maxBtSize when maxBtSize is the size of quadtree leaf node 902 of CTU901. • minBtSize: This is the minimum allowable size of a bilingual leaf node, i.e., the minimum width or height of a binary leaf node. Therefore, the current block can be partitioned by a horizontal BT partition if its height is greater than minBtSize. Also, the current block can be partitioned by a vertical BT partition if its width is greater than minBtSize. • maxTtSize: The maximum allowable trinity root node size, i.e., the maximum size of a leaf quadtree node that can be partitioned by a ternary partition. If both the height and width of the current block are less than or equal to maxTtSize, the current block can be partitioned by a TT partition. • minTtSize: Represents the minimum allowable trinity (TT) leaf node size, i.e., the minimum width or height of a binary leaf node. However, in contrast to BT partitioning, a minimum TT partition size is considered. Therefore, the current block can be partitioned by a horizontal TT partition if its height is greater than twice minTtSize. Also, the current block can be partitioned by a vertical TT partition if its width is strictly greater than twice minTtSize. • minQtSize: This is the minimum allowed quadtree (QT) leaf node size, and therefore, if the current block width is not greater than minQtSize, the QT partitioning mode is not allowed for the current block. Figure 10 shows an example of minQtSize. Considering CTU128, in the example shown, minQtsize is equal to 16.

[0131] Since maxQtSize is not defined, it corresponds to the CTU size.

[0132] The minimum allowable block size for width and height is 4.

[0133] A set of depths is also defined. • Depth: This is the depth within the tree. In the VVC specification, a leaf is the end node of a tree, which is the root node of a tree with a depth of 0. This means that this value is incremented (by 1) with each split. • mttDepth: This is the depth of the multitree. The multitree includes BT partitioning and TT partitioning. • maxMttDepth is the maximum allowable multitree depth, as defined in the VVC specification. Therefore, mttDepth must be greater than or equal to maxMttDepth. Figure 14 illustrates the concept of maxMttDepth.

[0134] In VVC, these variables are defined independently for Luma and Chroma.

[0135] VTM and ECM software have several other variables that correspond to depth.

[0136] The variable `currBtDepth` is the current number of BT partitions used to reach the current tree node (or current block). The variable `currMttDepth` is the current number of BT and TT partitions used to reach the current tree node (or current block). The variable `maxBtDepth` corresponds to the variable `maxMttDepth` in the VVC specification. `currQtDepth` is the current number of QT partitions used to reach the current tree node (or current block). MaxBtDepth: The maximum allowable div tree depth, i.e., the minimum level at which a binary partition can occur, where the quadtree leaf node is the root (e.g., 3).

[0137] VVC partition control syntax elements To set the values ​​of these different variables, several high-level syntax elements are sent in SPS, as shown in the following table of SPS syntax elements.

[0138] [Table 1]

[0139] When sps_partition_constraints_override_enabled_flag is enabled in SPS, several picture header syntax elements are sent to update partition variables, as shown in the following table of PH syntax elements.

[0140] [Table 2]

[0141] VVC coding split mode In VVC, the coding split mode is sent within the coding_tree, with conditionally parsed flags, split_cu_flag, split_qt_flag, mtt_split_cu_vertical_flag, and mtt_split_cu_binary flags defining the CU split as shown in the following syntax table.

[0142] [Table 3]

[0143] VVC partitioning limits VVC partitioning has several limitations. These limitations are primarily to avoid the same partitioning after several consecutive partitions. Figure 11 illustrates some of these constraints. The idea is to avoid the same partitioning as BT and TT. As shown in Figure 11(a), two consecutive vertical BT partitions are permitted, but as shown in Figure 11(b), a vertical TT partition following a vertical BT partition in the center block is not permitted.

[0144] Similarly, as shown in Figure 11(c), two consecutive horizontal BT divisions are permitted, but as shown in Figure 11(d), horizontal TT divisions following horizontal BT divisions in the center block are not permitted.

[0145] In VVC, there are additional constraints on the interblock size, including the maximum TT and BT block sizes and the minimum chroma block size. These constraints are removed in ECM software.

[0146] Chroma splitting In VVC, chroma partitioning can be inferred based on Luma partitioning, but this can be disabled. For example, in Dual Tree mode, the chroma partitioning tree does not depend on the Luma tree. However, some limitations exist.

[0147] The tree can partially rely on Luma partitioning for CCLM mode, or it can be independent otherwise.

[0148] Picture Boundary Frame resolution is not necessarily equal to an integer multiple of the CTU size. As a result, as shown in Figure 12, incomplete CTUs may exist at frame boundaries, with CTUs 1201-1206 being incomplete due to the lower boundary 1207 and right boundary 1208 of the frame. In VVC, in contrast to previous standards, splitting signaling is permitted at picture boundaries. Splitting at boundaries is applied until a coding tree node represents a CU that is fully located within the picture. However, some splits are inferred (not transmitted). Therefore, different variables such as maxMttDepth, minQtDepth, and minQtSize may be incremented or decremented and may differ from the variables used for splits that are not within boundaries.

[0149] QT BT TT coding selection VTM and ECM software employ several encoder-side optimizations for QT BT TT encoding selection.

[0150] One such optimization involves determining whether the QT partition is tested before the BT partition.

[0151] The conditions are that at least one CU to the left or above the current coding tree node has a QT depth greater than the current coding tree node's QT depth, and the CU width represented by the current coding tree node is greater than minQtSize*2.

[0152] If this condition is true, QT precedes BT, and the partitions are treated in the following order: - No splitting -QT -BT horizontal -BT Vertical -TT horizontal -TT vertical Otherwise, the order is as follows: - No splitting -BT horizontal -BT Vertical -TT horizontal -TT vertical -QT According to some optimizations, this order is important because some partitions are not tested depending on the results of the first test mode. Therefore, if QT is tested last, there are many opportunities for it to not be evaluated.

[0153] maxMttDepth The maximum MTT depth significantly impacts the complexity of the encoder. The common test conditions for the ECM have been updated to reduce encoding by using different maxMttDepth settings, as shown in Figure 13. This setting results in a lower maxMttDepth for some time IDs in high-resolution or low-QP settings.

[0154] Adaptable maxBtSize VTM and ECM have a frame-level coding select that sets maxBtSize according to the average block size of previous coded frames with the same depth (=> same time ID within a CTC RA case). The average block size is compared to a threshold as the following pseudocode:

[0155] [Table 4]

[0156] If AMAXBT_TH32 is equal to 15, then AMAXBT_TH64 is equal to 30, and AMAXBT_TH128 is equal to 60. This method decreases the maximum BT size if the average block size is small, and increases it if it is large.

[0157] Embodiment The available split modes are limited according to the current QT depth and other values.

[0158] In one embodiment, if the current QT partition does not reach a certain value, the possible partitions of the block are limited to a limited number of partition modes. In other words, if the current QT partition is less than or greater than a certain value, i.e., the value indicating a QT partition does not exceed a threshold (value). More precisely, whether BT, TT, or no-partition modes are allowed to partition the block depends on the value.

[0159] In this embodiment, a QT partition may be indicated by the block size of the current QT partition or by the QT partition depth.

[0160] The advantages of this embodiment are the reduction in encoder complexity due to the reduction in the amount of divisions evaluated on the encoder side, and the improvement in coding efficiency due to the reduction in division signaling.

[0161] Solution 1: QT splitting only According to the solution, block splitting (partitioning) is restricted to allowing only QT splitting if the current QT split does not reach a value. In one embodiment, if the block's QT depth is less than a value, only QT splitting of the current block is allowed. Therefore, BT, TT, and no splitting are not allowed.

[0162] Alternatively, if the block size of a block is greater than the value, only QT partitioning for the current block is permitted. This is valid when there is a relationship between block size and QT depth. In some embodiments, there is a direct relationship, but in other implementations, the block size of the QT depth value may depend on another parameter, for example, in VVC, it also depends on the CTU size.

[0163] Figure 14(a) illustrates this embodiment. In this figure, the block size is assumed to be the maximum between height and width. For example, the minimum between width and height should also be considered so that it can be used for blocks within picture boundaries where a square CTU is not possible. Or, only width or height. In this figure, the value compared to the QT depth is set to equal to 3 (corresponding to size 32 for block size). As shown, for current QT depths 0, 1, and 2, only QT division mode is permitted.

[0164] The advantage is reduced encoding time, as there is no need to evaluate partitions for QT depths smaller than this value. The second advantage is improved coding efficiency, as there is no need to send preceding relative syntax elements for QT depths smaller than this value. However, of course, this is a compromise between preserved bitrate and quality. Nevertheless, surprisingly, the proposed change yields benefits.

[0165] For example, the syntax element split_cu_flag does not need to be sent, thus reducing signaling overhead.

[0166] The CTU size is applied per block. In one embodiment, instead of the previously described embodiment, the CTU size is determined per block based on a value representing, for example, the QT depth or block size. For example, for the current block, another value is obtained, which represents the QT depth. Furthermore, the CTU size is obtained in relation to another value. Based on both of these values, the CTU for the current block is obtained. And the current QT depth starts based on this value.

[0167] Solution 2: QT and no splitting only In another embodiment, a solution is provided in which, if the QT depth of a block is less than a value, only QT partitioning and no partitioning of the current block are permitted. Thus, BT and TT partitioning are not permitted. Alternatively, if the block size of a block is greater than a value, only QT partitioning and no partitioning of the current block are permitted.

[0168] Figure 14(b) shows this embodiment. With respect to Figure 14(a), the value compared to the QT depth is set to equal to 3. As shown, for the current QT depths of 0, 1, and 2, only QT split mode and no-split mode are permitted.

[0169] If the QT depth is less than this value, only the no-split mode needs to be evaluated, and there are no QT splitting nodes, so the advantage is reduced coding time. The second advantage is improved coding efficiency, as only the no-split flag before the QT depth is less than this value needs to be sent.

[0170] QT, no splitting, TT only In one embodiment, if the block's QT depth is less than the value, or if the block size is greater than the value, only QT partitioning, TT partitioning, and no partitioning are allowed for the current block. Therefore, BT partitioning alone is not allowed.

[0171] Figure 15 illustrates this embodiment. With respect to Figures 14(a) and 14(b), the value compared to the QT depth is set to equal to 3. As shown, for the current QT depths of 0, 1, and 2, only BT splitting is not permitted.

[0172] This advantage also includes improved coding efficiency and, as in the previous embodiment, reduced coding time.

[0173] Further features regarding allowing BT splitting by relying on TT splitting at a higher level In one embodiment, only QT partitioning, TT partitioning, and no partitioning are permitted for the current block, and BT partitioning is also permitted if TT partitioning is selected at a higher level. To determine if TT is permitted, it is checked whether the value of mttDepth is greater than 0.

[0174] The advantage compared to the previous embodiment is an improvement in coding efficiency with little impact on the coding execution time.

[0175] Combination of solutions In one embodiment, if the QT depth of a block is less than the value "val1", only QT partitioning of the current block is permitted; if the QT depth of a block is less than another value "val2", only QT partitioning and no partitioning modes are permitted for the current block; and, or alternatively, if the QT depth of a block is less than another value "val3", only QT partitioning, TT, and no partitioning modes are permitted for the current block; and, or alternatively, BT partitioning is also permitted if TT partitioning is selected at a higher level.

[0176] Other combinations may be considered. This embodiment is illustrated by the following pseudocode:

[0177] [Table 5]

[0178] The advantages of this approach are reduced complexity and increased coding efficiency for a more optimal encoder. Of course, the improved coding efficiency is achieved through optimal value selection ("val1", "val2", "val3").

[0179] In the case of multiple solutions, the values ​​are ordered. In one embodiment, the values ​​"val1", "val2", and "val3" are ordered according to the following formula: val1 <= val2 <= val3 val1 is less than or equal to val2, and val3 is less than or equal to val2. Therefore, in this case, the smaller the value, the more possible partitioning modes are restricted.

[0180] The advantages of this approach are reduced complexity and increased coding efficiency for the optimal encoder.

[0181] This embodiment is dedicated to comparing QT depth and value. If block size is considered instead, the order may be indicated according to the following conditional expression: val1>=val2>=val3 Implementations relating to time areas Obtained from time area In one embodiment, for one or more of the aforementioned criteria, the QT depth of the current block is compared to a value determined from the time area; in other words, from the area of ​​a reference frame that is not the current frame. In some embodiments, the time area may be at the same location and / or the same size as the current block (e.g., juxtaposed), or at the same location and / or size as the current CTU (maximum block) that has a coding tree containing the current block. More generally, the time area may in some way correspond to or be associated with the current block of the current frame.

[0182] For example, the QT depth for a time area (e.g., juxtaposed blocks) is considered for the first criterion, which is that only QT is available, according to the following pseudocode:

[0183] [Table 6]

[0184] Here, QTDepthCol is the QT depth in the time area (e.g., juxtaposed blocks).

[0185] The advantages of this approach are reduced optimal coding execution time and improved coding efficiency.

[0186] Further details regarding the different possible time periods for obtaining the value are provided below.

[0187] juxtaposed areas In one embodiment, the block used to determine the time value of the QT depth or block size is a temporally juxtaposed area or block, i.e., an area or block in the reference frame that is juxtaposed with the current block. An example of one possible definition of a juxtaposed area or block is shown, for example, in Figure 6. As described above, juxtaposed in this context can mean an area or block that has the same position (e.g., the same origin) as the block in the current frame, or an area or block that has both the same size and position (as in the example in Figure 6). Alternatively, it can mean an area or block in the reference frame that, when projected onto the reference frame, is encompassed by the area of ​​the current block. A time area can also be said to correspond to (i.e., juxtaposed with) the current block even if, when projected directly onto the reference frame, that area completely encompasses the area of ​​the current block. Such a situation may arise, for example, when the division of the reference frame is different from that of the current frame, and rounding operations are required to determine which blocks in the reference frame should be said to correspond to the current block, or should be juxtaposed with the current block.

[0188] Multiple locations In one embodiment, multiple block positions within a reference frame are used to determine the time value of the block size or QT depth. For example, positions C, TL, TR, BL, and BR in Figure 16 may be considered. In this figure, position C is the center of the temporally juxtaposed block. Positions TL, TR, BL, and BR are the upper left, upper right, lower left, and lower right positions around the temporally juxtaposed block, respectively. Other adjacent block positions are also possible. Compared to previous embodiments, the current embodiments often provide greater coding time reduction and increased coding efficiency because the determined temporal QT depth or block size is more reliable.

[0189] Area larger than the current block area In one embodiment, the block size or QT depth time value is determined based on a time area that is larger in size (area) than the current block. For example, the time area is a juxtaposed CTU.

[0190] Compared to the previous two embodiments, more blocks can be considered, and therefore a better compromise is reached between reduced coding time and coding efficiency.

[0191] The center of the current block is used to determine its temporal position. In one embodiment, the center of the current block is considered in order to determine juxtaposed blocks or several time blocks or time areas. This is the “center” block shown within the juxtaposed area in Figure 6, or further indicated by position c in Figure 16.

[0192] The advantage of this configuration is that the center is the best position to represent the current block, resulting in a better overall structure.

[0193] Alternatively, if the center of the block is outside the current frame, the top-left position may be considered.

[0194] Entire frame In one embodiment, the time value of the block size or QT depth is determined based on all blocks in the time frame.

[0195] The advantage of this embodiment is the simplification of the process for determining the time QT depth or block size value, but it is less efficient compared to the previous embodiment because it is not adapted to the content.

[0196] Frames with the same time ID In one embodiment, juxtaposed blocks or several time blocks or time areas come from frames having the same time ID.

[0197] In the random access example, if the current frame has a time ID equal to 4, another encoded / decoded frame with the same time ID is used to determine the value of the proposed method, in a configuration as shown in Figure 7.

[0198] Frames with the same time ID often have the same coding parameters, and in particular, they have the same or similar QP and the same spatial distance to their reference frame. Therefore, these are very interesting for predicting QT depth, as this data correlates with QP and spatial distance between frames.

[0199] The nearest frame with the same time ID In one embodiment, juxtaposed blocks or several time blocks or time areas come from the nearest frame having the same time ID.

[0200] In the example of a random access configuration, as shown in Figure 7, the nearest frame with the same time ID is (generally) more correlated than other frames. Therefore, the results are better.

[0201] Frames or reference frames that have the same QP In one embodiment, juxtaposed blocks or several time blocks or time areas come from a frame or reference frame that has the same QP. Ideally, it is a reference frame that has the same QP.

[0202] As mentioned above, QP has a significant impact on block partitioning. Therefore, for frames with the same QP, time-based QT depth or block size are better predictors.

[0203] The reference frame is the same as the one used for time motion vector prediction. In one embodiment, juxtaposed blocks or several time blocks or time areas come from the reference frame used for time motion vector prediction. This can be the first reference in reference list 0 or the first reference frame in list 1, according to flags sent in the picture header or slice header.

[0204] Surprisingly, this embodiment provides the best compromise between encoder time reduction and coding efficiency, even when this reference frame has a lower QP. However, it is closer to the current frame compared to all frames with the same time ID.

[0205] Nearest reference frame In one embodiment, juxtaposed blocks or several time blocks or time areas come from the nearest reference frame.

[0206] As described in the previous embodiment, the distance to the current frame is of greater interest in the compromise between encoder time reduction and coding efficiency, even if frames with the same QP have statistically more correlation between their QP depths.

[0207] Multiple reference frames In one embodiment, two reference frames are considered, and two time areas, or two sets of some blocks, or two juxtaposed blocks are used to determine two time QT depths or two block sizes. These are then used to determine one QT depth or one block size. For example, the minimum QT depth from the two time areas may be considered.

[0208] Three or more reference frames may also be considered.

[0209] The advantage of this approach is that the QT depth value is calculated from more data, representing a better compromise between reduced encoder time and coding efficiency. This is particularly efficient when both reference frames have the same time distance, but it increases the amount of memory access.

[0210] Optional features related to (threshold) values The other value is the average QT depth value from the time area. In one embodiment, the current block's QT depth is compared to the average of the QT depth values ​​determined from the time area.

[0211] For example, the QT depth value at the time position in Figure 20 can be calculated using the following pseudocode to obtain the average value AverageQTDepthTempo, which can be used as the first criterion:

[0212] [Table 7]

[0213] Compared to the previous embodiment, this embodiment provides greater coding time reduction and increased coding efficiency because the average of the time QT depth is more reliable in many cases.

[0214] In one alternative embodiment, a larger time area may be considered.

[0215] The other value is the minimum QT depth value from the time area. In one embodiment, the current block's QT depth is compared to the minimum QT depth value determined from the time area.

[0216] For example, the first criterion can be written as the following pseudocode:

[0217] [Table 8]

[0218] The advantage is that the minimum time QT depth gives a value that is very unlikely to be lower than the current block's QT depth. Therefore, it provides a very optimal value and thus offers an excellent compromise between reduced encoding time and coding efficiency.

[0219] Other values ​​are calculated from the average MTT depth or maximum MTT depth from the time area. In one embodiment, a value compared to the current block's QT depth against a defined criterion is calculated according to the maximum MTT depth from the time area "MaxTempoMTTDepth".

[0220] Alternatively, the average MTT depth may be considered.

[0221] Other values ​​are calculated from the average or maximum depth over a given time period. In one embodiment, the value compared to the current block's QT depth against a defined criterion is calculated according to the maximum coding tree depth used in the time area "MaxTempoDepth," or according to the time-averaged depth.

[0222] An example of the two previous embodiments may be for the second criterion given by the following pseudocode:

[0223] [Table 9]

[0224] This alternative embodiment provides a good compromise between reduced encoding time and coding efficiency.

[0225] Embodiments having a particularly advantageous combination of features In one embodiment, QT partitioning is the only partition permitted when the QT depth of the current block value is less than the minimum QT depth value obtained by subtracting 1 from the time area "MinQTDepthTempo". Additionally, only QT partitioning, TT partitioning, and no partitioning modes are permitted for the current block at average depth, and BT partitioning is also permitted if the QT depth of the current block value is less than the average QT depth value obtained by subtracting 1 from the time area "AverageQTDepthTempo", when TT partitioning is selected at a higher level.

[0226] [Table 10]

[0227] Note that "If (QTDepth < MinQTDepthTempo - 1)" can also be written as "If (QTDepth + 1 < MinQTDepthTempo)", and similarly, "If (QTDepth < AverageQTDepthTempo - 1)" can also be written as "If (QTDepth + 1 < AverageQTDepthTempo)".

[0228] This particular combination provides the best compromise between coding efficiency and encoder runtime reduction. In fact, a threshold fixed at MinQTDepthTempo-1 yields a nearly systematic bitrate saving. The difference between MinQTDepthTempo and AverageQTDepthTempo allows for a good choice between QT and TT partitioning, as there is a partitioning mode with the highest number of sub-partitions. Ultimately, if TT is not selected, or if QT is above the time average, there is a high probability that BT or TT partitioning will be selected.

[0229] The value sent in the header corresponding to the current CTU follows. In one embodiment, a list of values ​​is sent directly in the high-level header as the picture header or slice header for each CTU. For example, the picture header table ph_QTDepthLimit[] is sent. The size of this table is equal to the number of CTUs in the picture.

[0230] An advantage of this embodiment is that the encoder can select a compromise between coding efficiency and encoder complexity using adapted encoder criteria. Furthermore, compared to time-area-based embodiments, parsing and decoding do not require access to time information.

[0231] A value can be predicted between them. In one embodiment, the transmitted values ​​are predicted to be between them.

[0232] For example, the value ph_QTDepthLimit[N] associated with CTU number N is predicted by the value ph_QTDepthLimit[N-1] associated with CTU number N-1. Then, only the residual (ph_QTDepthLimit[N]-ph_QTDepthLimit[N-1]) is sent.

[0233] The advantage is the rate reduction of these syntax elements within the picture header.

[0234] A single set of values ​​can be sent for all criteria (e.g., all conditional restrictions on the split mode). When considering several criteria, only one set of values ​​should be submitted for each criterion, not one value per criterion.

[0235] For example, if we use two criteria, the pseudocode would be:

[0236] [Table 11]

[0237] It is possible. The advantage is a rate reduction compared to the solution when several values are transmitted.

[0238] Further variations The value is adapted between MinQTDepthCol or QTDepthCol / +2, 1, 0, -1, -2, etc.: In one embodiment, the value of the current QT depth (or block size) is adapted according to one or more parameters. According to an example of a preferred embodiment, an offset "OFFSET" is added to the value representing the temporal QT depth or the minimum QT depth as in the following pseudocode:

[0239] [Table 12]

[0240] Note that two offsets can also be considered and it should be noted that one can be selected from them. The advantage is that since the method is adapted to several parameters, it is a better compromise between encoder time reduction and coding efficiency. In particular, the following.

[0241] Switch only between No BT, No BT and TT. In one embodiment, for the second criterion, switching between methods is enabled according to several parameters. For example, when the condition "Cond" is false, BT and TT are not permitted, and otherwise only BT is not permitted as in the following pseudocode:

[0242] [Table 13]

[0243] For example, the condition "Cond" can be that the maximum multi-tree depth of the temporal frame is less than the current maximum multi-tree depth.

[0244] Avoid one or more criteria In one embodiment, if a condition is met, one of the criteria is not applied. For example, if the maximum multitree depth for a time frame is less than the current maximum multitree depth, the first criterion is not applied. In fact, if the maximum multitree increases, QTdepth should be reduced. Therefore, there is a risk that interesting coding efficiencies will not be obtained.

[0245] In fact, in some configurations, neither criterion is useful.

[0246] Time frame slice type In one embodiment, one parameter for adapting the method is the slice type of the time area within the reference frame. For example, if the time area comes from an intra-slice, the slice will differ, as intra-prediction works better when the block size is small. In that case, the value of "OFFSET" is set to equal to -1 to take into account that the QT depth of intra-frames is generally higher than that of inter-frames for equivalent quality. However, the threshold may also depend on other parameters used in coding intra-frames, such as the maximum MTT depth and minimum QT size.

[0247] Time area frame and MaxMttDepth of the current frame In one embodiment, the method is adapted according to the maxMttDepth of the current frame and the maxMttDepth of the time-area frame. For example, the offset "OFFSET" is set to equal to +1 if the current maxMttDepth is smaller than the maxMttDepth of the time-area frame (higher, -1). With this setting "OFFSET is set to equal to +1", QT depth inference increases by a first criterion. In fact, if maxMttDepth decreases, it is more likely that the optimal minimum block size cannot be reached. Therefore, it is preferable to increase the QT depth.

[0248] MaxMttDepth of the current frame and maxMttDepth from the time area In one embodiment, the method is adapted according to the maxMttDepth of the current frame and the maxMttDepth of the time area. For example, the maxMttDepth of the time area is the average of the mttDepths from the time area (minimum or maximum may also be considered). The MaxMttDepth of the frame in the time area is the value for the entire frame and may be sent in the picture header PH_MaxMttDepth. The MaxMttDepth of the time area is the maximum mttDepth of a block or group of blocks contained in a time (e.g., juxtaposed) area.

[0249] This embodiment is superior to the previous embodiment because, instead of encoding parameters being set based on target quality content, they are adapted to the content.

[0250] The MaxMttDepth of the current frame and the MaxMttDepth reference frame of the time area are the same. In one embodiment, if the maxMttDepth of the current frame and the maxMttDepth of the time area reference frame are the same, the method is adapted according to the maxMttDepth of the current frame and the maxMttDepth of the time area.

[0251] This embodiment is particularly efficient when the CTU size is equal to 256.

[0252] MinQtSize In one embodiment, the method is adapted based on minQtSize. In fact, since minQtSize affects the QT depth of each block, this parameter affects the baseline efficiency.

[0253] Regarding the two previous embodiments for maxMttDepth, the minQtSize of the current frame can be compared with the minQtSize of the frames in the temporal area, or with the minimum / maximum or average of the minQtSize of the temporal area.

[0254] If the current frame has a higher QP than the frames in the temporal area In one embodiment, the method is adapted when the current frame has a lower QP than the frames in the temporal area. For example, when the QP difference is high, the OFFSET value is set to +1 or more. In fact, blocks within a frame with a higher QP generally have a lower QT depth on average. Thus, the minimum QT depth of this frame with a higher QP is smaller than when this frame has a lower QP. As a result, the reference value should be increased.

[0255] Whether the current frame is a reference frame or not.

[0256] In one embodiment, the method is adapted according to whether the current frame is a reference frame or not.

[0257] For example, when the current frame is not a reference frame, the OFFSET value has a positive value. In fact, with a higher offset, the coding decreases, but if the quality of the frame decreases, this does not affect future frames. This particularly applies to the second criterion where BT and TT are not permitted.

[0258] Disable the coding of screen content In one embodiment, the proposed method is adapted to screen content coding. In particular, this method is disabled for screen content. The inventors have found that it seems more difficult to predict the splitting parameters when the content of the sequence includes screen content.

[0259] Alternatively, the number of IBC blocks is calculated in the time area, and according to a threshold, the method is disabled for the current block.

[0260] Partially disable time forecasting. In one embodiment, the proposed method is partially disabled when screen content is detected. This embodiment and the following embodiments are described independently of the detection or presence of screen content as described later. The advantages of this embodiment are a compromise in complexity and improved coding efficiency. In fact, as mentioned above, predicting the segmentation parameters of screen content sequences is difficult. Partial prediction allows for a reduction in coding time while minimizing the loss of these sequences.

[0261] Disable Solution 1: QT splitting only In one embodiment, if the current block is considered to be adapted for screen content coding, the restriction that only allows QT splitting if the current QT split does not reach a value is disabled (Solution 1), which is not allowed. In this embodiment, at least one other splitting prediction solution as described above remains in effect.

[0262] The advantage of this is that it improves the coding efficiency of blocks containing screen content while maintaining reduced coding time by at least one other split prediction solution / embodiment as described above.

[0263] Disable Solution 2: QT and no splitting only In one embodiment, if the current block is considered to be suitable for screen content coding, and the QT depth of the block is less than a certain value, the restriction on allowing QT splitting and no splitting (Solution 2) is not allowed for the current block only. In this embodiment, at least one other splitting prediction solution as described above remains in effect.

[0264] The advantage, as mentioned above, is that it improves the coding efficiency of blocks containing screen content while maintaining reduced coding time through at least one other split prediction solution / embodiment.

[0265] Disable solution 2.2: TT, no splitting, QT only In one embodiment, if the current block is considered to be suitable for screen content coding, and the block's QT depth is less than a certain value, or the block size is greater than a certain value, the restrictions allowing only QT splitting, TT, and no splitting for the current block are not permitted. In this embodiment, at least one other splitting prediction solution as described above remains valid.

[0266] The advantage, as mentioned above, is that it improves the coding efficiency of blocks containing screen content while maintaining reduced coding time through at least one other split prediction solution / embodiment.

[0267] Solution 2 allows BT splitting via TT splitting by disabling + TT splitting. In one embodiment, if the current block is considered to be suitable for screen content coding, restrictions are placed on the current block to allow only QT splitting, TT, and no splitting, and if TT splitting is selected at a higher level, BT splitting is also allowed, but not allowed. To determine if TT is allowed, it is checked to see if the value of mttDepth is greater than 0. In this embodiment, at least one other splitting prediction solution as described above remains in effect.

[0268] The advantage, as mentioned above, is that it improves the coding efficiency of blocks containing screen content while maintaining reduced coding time through at least one other split prediction solution / embodiment.

[0269] The following example illustrates one implementation of the proposed embodiment, based on one previous example:

[0270] [Table 14]

[0271] In this example, && is the AND operator, ! is the NO operator, and IsScreenContent is a Boolean value equal to true if the current block should be applied to screen content coding, and equal to FALSE otherwise.

[0272] Adaptation when screen content is detected As described above, enabling or disabling one or more or all embodiments of time-division prediction depends on the detection (presence) of screen content. The following embodiments describe several methods for detecting whether the current block potentially contains screen content. Each of these methods is applicable to the embodiments already described that rely on the detection or implicit presence of screen content.

[0273] If palette, IBC, or conversion skip or other SCC tools are enabled In one embodiment, if palette, IBC, or conversion skip is enabled, the current block is considered to potentially contain screen content. Another efficient future coding tool for screen content may also be considered.

[0274] In fact, IBC and palette modes are some of the most useful modes for screen content coding. Also, conversion skipping is an efficient way to code texture residuals in screen content. Therefore, if these tools are enabled or allowed, coding blocks may reside within the screen content area. However, IBC and conversion skipping are known to provide coding efficiency for natural content.

[0275] Therefore, in the alternative embodiment, screen content detection relies solely on palette mode.

[0276] The advantage of these embodiments is increased coding efficiency compared to the enable / disable signaling of the embodiments for coding blocks or slices, pictures or sequences, because enable / disable depends on the data present in the bitstream.

[0277] If signaled In one embodiment, if a palette, IBC, or conversion skip is signaled, the current block is considered to potentially contain screen content. Also, as mentioned above, this can rely on palette mode-only signaling or on future screen content coding tools.

[0278] The advantages are the same as those of the previous embodiment.

[0279] SPS, PPS, Picture Header, Slice Header In one embodiment, enabling / disabling the palette mode IBC, or using a future coding tool or conversion skip for screen content, is used to determine whether the current block is potentially within the screen content area. Enabling / disabling may be signaled in the SPS, PPS, picture header, or slice header.

[0280] For example, as mentioned above, the Boolean IsScreenContent can be the SPS flag sps_palette_enabled_flag. Alternatively, as defined, this Boolean can be modified by the PPS flag pps_palette_enabled_flag, or the picture header flag ph_palette_enabled_flag, or the slice header flag sh_palette_enabled_flag.

[0281] The advantages are the same as those of the previous embodiment.

[0282] When signaling for a reference frame that includes a time area In one embodiment, if a palette mode, IBC, or conversion skip or future coding tool for screen content exists within a reference frame containing a time area, the current block is detected as a potential screen content area. Each coding tool may be considered, or some may be considered. In particular, only the palette mode may be considered. In this embodiment, one or more coding tools may be enabled at the slice level, picture level, or PPS level of the reference frame.

[0283] The advantages are the same as in the previous embodiment, and are more precise.

[0284] If at least one SCC content tool is detected In one embodiment, if at least one block is coded with a screen content coding tool or additional screen content coding tools instead of the previous one, the current block may be considered as potential screen content. As mentioned above, alternatively, only palettes may be considered.

[0285] Compared to the previous embodiment, detection is better, as is the actual selection, as well as enabling or disabling the coding tool. However, compared to the previous embodiment, this embodiment is more complex because it requires checking all blocks.

[0286] If at least one of the SCC content tools is detected within the reference frame In one embodiment, in addition to the features of the previous embodiment, the block considered is a block of the reference frame.

[0287] Compared to the previous embodiment, detection is better because the reference frame is the frame most relevant to the coding of the current one.

[0288] If at least one of the SCC content tools is detected within the time area, in an additional embodiment, the block of reference frame considered for detection is a block within the time area (or said time area). Alternatively, juxtaposed blocks may be considered, or a juxtaposed block and its adjacent blocks may be considered.

[0289] Compared to the previous embodiment, detection is better because the time area or other alternative is most relevant as being closest in time to the current block. Furthermore, this embodiment is particularly more efficient for mixed content sequences (including screen content portions and natural content portions). However, this embodiment is also more complex because it is necessary to check each time area within the internal block. Unlike the reference frame, the block needs to be checked only once.

[0290] Based on the number of occurrences In one embodiment, if a certain number of palette modes or IBC or conversion skip or future coding tools for screen content exist within a reference frame containing a time area, the current block is detected as a potential screen content area. In this embodiment, a reference frame or time area may be considered. The occurrence of these coding tools is calculated for the area under consideration (reference frame or time area). As previously stated, palette modes are efficient modes only for screen content coding, and therefore their occurrence may be considered equal to 1 for detection. In the case of IBC, this mode is more preferred for screen content coding than natural content coding, and screen content is detected, for example, if more than 10% of the block of area under consideration is coded in this mode. Similarly, in the case of conversion skip tools, appropriate statistics may be considered for future coding tools for screen content.

[0291] Alternatively, the number of pixels (or samples) using these coding tools can be considered instead of the occurrence.

[0292] The advantage of this embodiment is improved coding efficiency, as it offers the possibility of using transformation skipping and IBC for more efficient generation and detection. Furthermore, this embodiment is better adapted for detecting natural and screen content for mixed content. However, the computation of generation and their storage is more complex than in the previous embodiment.

[0293] If an SCC is detected and the criterion is an intraframe, disable it completely or partially. In one embodiment, if it is detected that the current block potentially contains screen content and that the reference frame containing the time area is an intraframe, the proposed time prediction for the split is disabled all or partially. Alternatively, this restriction may apply to one solution but not to another solution / embodiment. In other words, the additional restriction that the reference frame is an intraframe may be applied selectively, to one or more of the solutions described above, or whenever screen content detection is used.

[0294] The advantages of this embodiment are improved coding efficiency and reduced complexity compared to the previous embodiment. In fact, intraframes containing screen content are coded in small blocks, for example, to match characters of text in palette mode or IBC mode. Conversely, interframes contain large blocks because the movement is very small or is located only on the mouse pointer. Therefore, predicting interframe divisions by intraframe divisions is not applicable to our proposed division time prediction. However, division prediction works between interframes for screen content.

[0295] Several examples are given to illustrate various embodiments. In the following examples, screen content detection is based on the sequence parameter flag sps_palette_enabled_flag, which can be replaced by each screen content detection embodiment as described above.

[0296] In the following example, if palette mode is enabled for the sequence (sps_palette_enabled_flag), solutions that only allow QT splitting are not applicable, but solutions that allow all splitting except BT splitting are permitted.

[0297] [Table 15]

[0298] In the following example, if palette mode is enabled for the sequence (sps_palette_enabled_flag), the solution that only allows QT splitting is not allowed only if palette mode is enabled and the current reference frame is an intraframe; otherwise, it is allowed. Also, if palette mode is enabled, the second criterion that only BT splitting is not allowed does not apply. In this example, if palette mode is enabled and the reference frame is an intraframe, the proposed time prediction for the split is completely disabled. Also, if the reference frame is not an intraframe, only the first part of the prediction is allowed.

[0299] [Table 16]

[0300] In the following example, if palette mode is enabled for the sequence (sps_palette_enabled_flag), the solution that only allows QT splitting is not applied, only when palette mode is enabled. The second criterion that only BT splitting is not allowed is not applied when palette mode is enabled and the current reference frame is an intraframe, and is allowed otherwise. In this example, if palette mode is enabled and the reference frame is an intraframe, the proposed time prediction for splitting is completely disabled. If the reference frame is not an intraframe, only the second part of the prediction is allowed.

[0301] [Table 17]

[0302] Low-latency configuration is disabled. In one embodiment, the proposed method is adapted to a low-latency configuration. In particular, this method is disabled.

[0303] Disabled using a flag In one embodiment, the proposed method is enabled or disabled using at least one flag transmitted in at least one header.

[0304] It is not based on QTDepthCol, but rather on block size statistics such as those used in VTM "Adaptive Max BT size" encoding selection. In one embodiment, the proposed method is based on statistics of block sizes in the time area. For example, the average size of blocks in the time area may be considered. For example, if the average multiplied by the maximum multitree depth of the current picture is less than the current block size (or a value indicating the current block size), then the first criterion is applied.

[0305] The decision is made at the CTU level, not at the block level.

[0306] In one embodiment, the reference parameters are determined on a CTU basis rather than a block basis, so that the same parameters are used for all blocks of the CTU. For example, MinQTDepthTempo and AverageQTDepthTempo are determined per CTU and are the same for all blocks of the CTU. This can be easily achieved by considering the current center of the CTU for each block of the CTU.

[0307] This reduces the trade-off between reduced encoding execution time and coding efficiency, but it is only applicable when combined with other methods.

[0308] Solution 3: If TT splitting was previously selected in the splitting process, BT is allowed. In any embodiment independent of or combinatorial to any prior embodiment, BT partitioning is also permitted if TT partitioning is selected at a higher level.

[0309] To determine if TT is permitted, it is checked whether the value of mttDepth is greater than 0. Alternatively, if the difference between mttDepth and btDepth is equal to 0 (this may be used for specific settings of minTtSize, maxTtSize, minBtSize, maxBtSize).

[0310] The advantage of this embodiment is the improvement in coding efficiency, which has little impact on the increase in coding time.

[0311] High-level syntax signaling The method is enabled throughout the entire sequence. In one embodiment, a high-level syntax flag for the entire sequence is sent to enable or disable the proposed method. For example, a flag may be sent within the SPS to indicate that the method is enabled.

[0312] The method is enabled for the set of pictures. In one embodiment, a high-level syntax flag for a set of pictures is sent to enable or disable the proposed method. For example, a flag may be sent within the PPS to indicate enablement at the picture level.

[0313] The method is enabled for each picture. In one embodiment, a high-level syntax flag is sent for each picture to enable or disable the proposed method. For example, the flag may be sent within the picture header (PH).

[0314] The method is enabled at a lower level. In one embodiment, a lower-level high-level syntax flag, such as a slice or tile level, is sent to enable or disable the proposed method. For example, the flag may be sent within the SH (slice header).

[0315] Each standard In one embodiment, one flag is sent for each criterion. For example, the flag may signal whether a particular restriction should apply, and this may be transmitted by being encoded and retrieved from the bitstream by a decoder. For example, the picture header flag PH_Restriction_QT is sent. Then the first criterion described above is applied as follows:

[0316] [Table 18]

[0317] The advantage of all the high-level syntax signaling embodiments described above is the encoder's flexibility in determining the best compromise.

[0318] Encoding selection If both BT H and BTV are not allowed, QT precedes BT. In one embodiment, when a different test order for divisions is determined, if both BT horizontal and BT vertical are not permitted on the encoder side, the variable QT before BT is set to true equality.

[0319] This provides an improvement in coding efficiency with or without the proposed method, without an increase in coding time. This is due to the fact that encoder optimizations involve avoiding testing QT when BT is detected better. Therefore, these optimizations are inefficient if TT is the only available MTT split.

[0320] No additional possible divisions are allowed, and their list is not permitted.

[0321] In one embodiment, if an unavailable split is unavailable, it is not added to the test list. For example, if the variable QT before BT is true, the pseudocode is applied to satisfy the list of splits "listSplits":

[0322] [Table 19]

[0323] A similar pseudo is used when the variable QT before BT is false.

[0324] Similar to the previous embodiments, this provides an improvement in coding efficiency with or without the proposed method of the previously described embodiments, without an increase in coding time. This benefit relates to several encoder optimizations that can avoid TT or QT evaluation under certain conditions and when some divisions are previously listed.

[0325] If this condition is true, QT precedes BT, and the split is processed in the following order: - No splitting -QT -BT horizontal -BT Vertical -TT horizontal -TT vertical Otherwise, the order is as follows: - No splitting -BT horizontal -BT Vertical -TT horizontal -TT vertical -QT others In one embodiment, other division modes may be applied, and the method may be adapted accordingly.

[0326] Unless otherwise specified, all described embodiments can be combined. In fact, many combinations are synergistic and can result in greater efficiency improvements than the sum of their parts.

[0327] Implementation of the invention Figure 17 shows systems 191, 195 according to embodiments of the present invention, having at least one of an encoder 150 or decoder 100 and a communication network 199. According to one embodiment, system 195 is for processing and providing content (e.g., video and audio content for display / output, or streaming video / audio content) to a user who accesses the decoder 100, for example, via a user interface of a user terminal having the decoder 100 or a user terminal capable of communicating with the decoder 100. Such a user terminal may be a computer, mobile phone, tablet, or any other type of device capable of providing / displaying (provided / streamed) content to the user. System 195 acquires / receives a bitstream 101 (e.g., in the form of a continuous stream or signal while previous video / audio is being displayed / output) via the communication network 199. According to one embodiment, systems 191, 195 are for processing content and storing the processed content, for example, processed video and audio content for later display / output / streaming. Systems 191 and 195 acquire / receive content having the original sequence 151 of images that have been received and processed by encoder 150 (including filtering by a deblocking filter according to the present invention), and encoder 150 generates a bitstream 101 that is communicated to decoder 100 via communication network 191. The bitstream 101 is then communicated to decoder 100 in several ways, for example, it may be pre-generated by encoder 150 and stored as data in a storage device (e.g., on a server or cloud storage) within communication network 199 until a user requests the content (i.e., bitstream data) from the storage device, at which point the data is communicated / streamed from the storage device to decoder 100.Systems 191 and 195 may also have a content provider for receiving and processing user requests for content, so that the requested content can be delivered / streamed from the storage device to the user terminal (for example, by communicating data for the user) and provide / stream content information (e.g., the title of the content and other metadata / storage location data for identifying, selecting, and requesting the content) of the content stored in the storage device. Alternatively, encoder 150 generates a bitstream 101 and communicates / streams it directly to decoder 100 when a user requests content. Decoder 100 then receives the bitstream 101 (or signal) and performs filtering using a deblocking filter according to the present invention to obtain / generate a video signal 109 and / or an audio signal, which are then used by the user terminal to provide the requested content to the user.

[0328] Any step of the method / process according to the present invention or any function described herein may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the step / function may be stored or transmitted as one or more instructions or code or programs, or as computer-readable media, and may be executed by one or more hardware-based processing units, such as a PC ("Personal Computer"), a DSP ("Digital Signal Processor"), a circuit, a processor and memory, a general-purpose microprocessor or central processing unit, a microcontroller, an ASIC ("Application-Specific Integrated Circuit"), a field-programmable logic array (FPGA), or other equivalent integrated or discrete logic circuits. Accordingly, the term "processor" as used herein may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein.

[0329] Embodiments of the present invention can also be implemented by a wide variety of devices or apparatus, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chipsets). While various components, modules, or units are described herein to illustrate functional aspects of devices / apparatus configured to perform their embodiments, implementation by different hardware units is not necessarily required. Rather, the various modules / units may be combined in a codec hardware unit, or provided by a collection of interoperable hardware units including one or more processors in conjunction with appropriate software / firmware.

[0330] Embodiments of the present invention may be realized by a computer in a system or device that includes one or more processing units or circuits for reading and executing computer-executable instructions (e.g., one or more programs) recorded on a storage medium, executing one or more modules / units / functions of the embodiments described above, and / or for executing one or more functions of the embodiments described above, and by controlling, for example, one or more processing units or circuits for executing one or more functions of the embodiments described above. The computer may include separate processing units or a network of separate computers for reading and executing computer-executable instructions. Computer-executable instructions may be provided to the computer from a computer-readable medium, such as a communication medium via a network or tangible storage medium. The communication medium may be a signal / bitstream / carrier wave. Tangible storage media are “non-temporary computer-readable storage media” which may include one or more of the following: hard disks, random access memory (RAM), read-only memory (ROM), storage devices for distributed computing systems, optical discs (such as Compact Discs (CDs), Digital Multipurpose Discs (DVDs), or Blu-ray Discs (BDs) (trademarks)), flash memory devices, memory cards, etc. At least some of the steps / functions may also be implemented in hardware by devices or dedicated components such as FPGAs (“Field-Programmable Gate Arrays”) or ASICs (“Application-Specific Integrated Circuits”).

[0331] Figure 18 is a schematic block diagram of a computing device 3600 for implementing one or more embodiments of the present invention. The computing device 3600 may be a device such as a microcomputer, workstation, or light portable device. The computing device 3600 has a communication bus connected to: - a central processing unit (CPU) 3601 such as a microprocessor; - random access memory (RAM) 3602 for storing executable code of the methods of embodiments of the present invention, and registers adapted to record variables and parameters necessary to implement the methods for encoding or decoding at least a portion of an image according to embodiments of the present invention, the memory capacity of which may be expanded, for example, by optional RAM connected to an expansion port; - read-only memory (ROM) 3603 for storing computer programs for implementing embodiments of the present invention; - a network interface (NET) 3604, which is typically connected to a communication network through which digital data to be processed is transmitted or received. The network interface (NET) 3604 may be a single network interface or may consist of a set of different network interfaces (e.g., wired and wireless interfaces, or different types of wired or wireless interfaces). Data packets are written to the network interface for transmission or read from the network interface for reception under the control of a software application running on the CPU 3601; a user interface (UI) 3605 may be used to receive input from the user or to display information to the user; a hard disk (HD) 3606 may be provided as mass storage; and an input / output module (IO) 3607 may be used to receive / transmit data to and from external devices such as a video source or display. Executable code may be stored in ROM 3603, HD 3606, or on a removable digital medium such as a disk.In a modified version, the executable code of a program may be received via a communication network through NET3604 to be stored in one of the storage means of a communication device 3600, such as HD3606, before execution. The CPU3601 is adapted to control and direct the execution of instructions or parts of instructions of a set of programs or software code of a program according to embodiments of the present invention, and these instructions are stored in one of the aforementioned storage means. After power-up, and after those instructions have been loaded, for example, from the program ROM3603 or HD3606, the CPU3601 can execute instructions from the main RAM memory 3602 relating to a software application. When such a software application is executed by the CPU3601, it causes the steps of the method according to the present invention to be performed.

[0332] Furthermore, according to another embodiment of the present invention, it is understood that the decoder according to the above embodiment is provided in a user terminal such as a computer, a mobile phone, a table, or any other type of device (e.g., a display device) that can provide / display content to a user. According to yet another embodiment, the encoder according to the above embodiment is provided in an image capture device which also has a camera, a digital video camera, or a network camera (e.g., a closed-circuit television or video surveillance camera) that captures and provides content for the encoder to encode. Two such examples are provided below with reference to Figures 19 and 20.

[0333] Figure 19 shows a network camera system 3700, which includes a network camera 3702 and a client device 202.

[0334] The network camera 3702 includes an imaging unit 3706, an encoding unit 3708, a communication unit 3710, and a control unit 3712.

[0335] The network camera 3702 and the client device 202 are interconnected via the network 200 so that they can communicate with each other.

[0336] The imaging unit 3706 includes a lens and an image sensor (e.g., a CCD (charge-coupled device) or a CMOS (complementary metal oxide semiconductor)), captures an image of the subject, and generates image data based on the image. This image can be a still image or a video image.

[0337] The encoding unit 3708 encodes the image data using the encoding method described above, or a combination of the encoding methods described above.

[0338] The communication unit 3710 of the network camera 3702 transmits the encoded image data encoded by the encoding unit 3708 to the client device 202.

[0339] Furthermore, the communication unit 3710 receives commands from the client device 202. These commands include commands for setting the encoding parameters of the encoding unit 3708.

[0340] The control unit 3712 controls other units within the network camera 3702 according to commands received by the communication unit 3712.

[0341] The client device 202 includes a communication unit 3714, a decoding unit 3716, and a control unit 3718.

[0342] The communication unit 3714 of the client device 202 sends a command to the network camera 3702.

[0343] Furthermore, the communication unit 3714 of the client device 202 receives encoded image data from the network camera 3712.

[0344] The decoding unit 3716 decodes the encoded image data using the decoding method described above, or a combination thereof.

[0345] The control unit 3718 of the client device 202 controls other units within the client device 202 according to user operations and commands received by the communication unit 3714.

[0346] The control unit 3718 of the client device 202 controls the display device 2120 to display the image decoded by the decoding unit 3716.

[0347] Furthermore, the control unit 3718 of the client device 202 controls the display device 2120 to display a GUI (Graphical User Interface) and specifies the parameter values ​​of the network camera 3702, including the encoding parameters of the encoding unit 3708.

[0348] Furthermore, the control unit 3718 of the client device 202 controls other units within the client device 202 according to user input to the GUI displayed by the display device 2120.

[0349] The control unit 3718 of the client device 202 controls the communication unit 3714 of the client device 202 to send a command to the network camera 3702 that specifies the parameter values ​​of the network camera 3702, in accordance with user input to the GUI displayed by the display device 2120.

[0350] Figure 20 shows the smartphone 3800.

[0351] The smartphone 3800 includes a communication unit 3802, a decoding unit 3804, a control unit 3806, and a display unit 3808.

[0352] The communication unit 3802 receives encoded image data via the network 200.

[0353] The decoding unit 3804 decodes the encoded image data received by the communication unit 3802.

[0354] The decoding / encoding unit 3804 decodes and encodes the encoded image data using the decoding method described above.

[0355] The control unit 3806 controls other units within the smartphone 3800 according to user operations or commands received by the communication unit 3806.

[0356] For example, the control unit 3806 controls the display unit 3808 to display the image decoded by the decoding unit 3804. The smartphone 3800 may also have a sensor 3812 and an image recording device 3810. In this way, the smartphone 3800 can record images and encode them (using the method described above).

[0357] The smartphone 3800 can then decode the encoded images (using the method described above) and display them via the display unit 3808, or transmit the encoded images to another device via the communication unit 3802 and the network 200.

[0358] Substitute and change While the present invention has been described with reference to embodiments, it should be understood that the present invention is not limited to the disclosed embodiments. Those skilled in the art will understand that various changes and modifications can be made without departing from the scope of the invention, as defined in the appended claims. All features disclosed in this specification (including any appended claims, abstract, and drawings) and / or all steps of any method or process so thus disclosed can be combined in any combination, except for combinations in which at least some of such features and / or steps are mutually exclusive. Each feature disclosed in this specification (including any appended claims, abstract, and drawings) may be replaced by an alternative feature serving the same, equivalent, or similar purpose, unless otherwise expressly stated. Therefore, unless specifically stated, each disclosed feature is merely an example of a general set of equivalent or similar features.

[0359] Furthermore, any result of the above comparison, judgment, evaluation, selection, execution, performing, or consideration, for example, a selection made during the encoding or filtering process, may be indicated in data within the bitstream, for example, in a flag or data indicating the result, or may be decidable / inferable from there, and as a result, the indicated or determined / inferred result may be used in the process instead of actually performing the comparison, judgment, evaluation, selection, execution, performing, or consideration, for example, during the decoding process.

[0360] In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude the plural. The mere fact that different features are described in different dependent claims does not imply that combinations of these features cannot be used to one's advantage.

[0361] The reference numerals appearing in the claims are for illustrative purposes only and do not have any limiting effect on the scope of the claims.

Claims

1. A method for encoding image data into a bitstream containing data indicating that the image data is divided into a plurality of blocks according to a coding tree, or for decoding the image data from a bitstream, wherein the blocks in the coding tree may be divided according to a plurality of partitions, including a quadtree partition, and the method Determine a value that indicates the depth of the quadtree partition of the current block, If the value indicating the depth of the quadtree partition does not exceed the threshold, then only the quadtree partition of the current block is permitted from among the multiple partitions. A method of having.

2. The method according to claim 1, wherein the plurality of divisions further comprises a triple tree division, a double tree division, and no division.

3. A method for encoding image data into a bitstream containing data indicating that image data is to be divided into a plurality of blocks according to a coding tree, wherein the blocks in the coding tree may be divided according to a plurality of partitions including a double-tree partition, a triple-tree partition, a quad-tree partition, and no partition, and the method Determine a value that indicates the depth of the quadtree partition of the current block, Not allowing the two-tree split based on one or more criteria, and the criterion or one criterion includes ensuring that the value indicating the depth of the quadtree split does not exceed a threshold. A method of having.

4. The method according to claim 3, further comprising allowing the quadtree split, the triple tree split, and no split if the value indicating the depth of the quadtree split does not exceed the threshold.

5. Furthermore, the method according to claim 3, wherein the triple tree split is not permitted, and the quadtree split and no split are permitted if the value indicating the depth of the quadtree split does not exceed the threshold.

6. The one or more of the above criteria include that the triple tree split is not performed at a level higher than the current block level, If the aforementioned three-tree splitting is not performed, the aforementioned two-tree splitting is not permitted. If the three-tree split is performed, the two-tree split for the current block is permitted. The method according to claim 4 or 5.

7. A method for decoding a bitstream, Performing the first method described in any one of claims 1 to 2, The second method described in any one of claims 3 to 6 It has, A method in which the threshold for allowing only quadtree splitting is a first threshold, and the threshold for not allowing binary tree splitting is a second threshold, and the first threshold and the second threshold are different.

8. The method according to claim 7, wherein the second method is the method according to claim 6, further comprising carrying out the third method according to any one of claims 3 to 5, wherein the threshold for the third method is a third threshold different from the first threshold and the second threshold.

9. The method according to any one of claims 1 to 8, wherein the value indicating the depth of the quadtree partition is the depth value of the quadtree partition, and if it is less than a predetermined depth value of the quadtree partition, it exceeds the threshold.

10. The method according to any one of claims 1 to 8, wherein the value indicating the depth of the quadtree partition is the block size of the current block, and if it is greater than a predetermined block size, it exceeds the threshold.

11. The method according to any one of claims 1 to 10, wherein the threshold is based on one or more parameters associated with a time area in one or more reference frames, and the reference frame is a frame different from the frame containing the current block.

12. The method according to claim 11, wherein the area is an area placed alongside the current block.

13. The method according to claim 11 or 12, wherein the area has a plurality of blocks located at different positions.

14. The method according to claim 11 or 12, wherein the area is an area having a larger size than the current block.

15. A coding tree unit, the area having a size larger than the current block in the CTU, the method according to claim 14.

16. The method according to any one of claims 11 to 15, wherein the center position of the current block is used to determine the reference frame or the area within one reference frame.

17. The method according to claim 11, wherein the area includes the entire area of ​​the reference frame or one reference frame.

18. The method according to any one of claims 11 to 17, wherein the reference frame or one reference frame is a frame having the same time ID as the current frame which includes the current block.

19. The method according to claim 18, wherein the frame having the same time ID is the closest frame having the same time ID.

20. The method according to any one of claims 11 to 19, wherein the reference frame or one reference frame is a frame having the same quantization parameters as the current frame which includes the current block.

21. The method according to any one of claims 11 to 19, wherein the reference frame or one reference frame is a frame used for time motion vector prediction.

22. The method according to any one of claims 11 to 19, wherein the reference frame or one reference frame is a frame which is the reference frame closest to the current frame which contains the current block.

23. The method according to any one of claims 1 to 22, wherein a first area from a first reference frame and a second area from a second reference frame are used to obtain the threshold.

24. The method according to any one of claims 11 to 23, wherein the threshold is based on the mean quadtree depth determined from the time area.

25. The method according to any one of claims 11 to 23, wherein the threshold is based on the minimum quadtree depth determined from the time area.

26. The method according to any one of claims 11 to 23, wherein the threshold is determined based on the maximum multitree depth or the average multitree depth from the time area.

27. The method according to any one of claims 11 to 23, wherein the threshold is determined using the maximum or average coding tree depth from the time area.

28. The method according to any one of claims 1 to 10, wherein the threshold is based on a value transmitted in the header associated with the coding tree unit of the current block.

29. The method of claim 28, wherein the value transmitted in the header of the current coding tree unit is used to predict the threshold using one or more other values ​​transmitted in a header associated with another coding tree unit.

30. The method of claim 28, as dependent on claim 8 or 9, wherein the value transmitted in the header can be used to determine a plurality or all of the thresholds.

31. The method according to any one of claims 24 to 30, further comprising adding an integer offset to obtain the threshold or each threshold.

32. The method according to any one of claims 24 to 31, further comprising adapting the threshold based on the slice type corresponding to the time area in the reference frame.

33. The method according to any one of claims 24 to 32, wherein the threshold can be adapted based on the maximum multitree depth of the frame in the time area and the maximum multitree depth.

34. The method according to any one of claims 24 to 33, wherein the threshold can be adapted based on the maximum multitree depth and the time area of ​​the maximum multitree depth.

35. The method according to any one of claims 24 to 34, wherein the threshold can be applied based on the minimum quadtree size of the frame in the time area, or to the minimum, maximum, or average quadtree size of the time area.

36. The method according to any one of claims 24 to 35, wherein the threshold can be adapted according to whether the current frame has a lower QP than the frame containing the time area.

37. The method according to any one of claims 24 to 36, wherein the threshold can be applied depending on whether the current frame containing the current block is a reference frame.

38. The method according to any one of claims 1 to 23, wherein the threshold is based on statistics of block size in the time area.

39. The method according to any one of claims 1 to 38, wherein the parameter used to determine the threshold is obtained from a coding tree unit-level block.

40. The method according to any one of claims 1 to 39, wherein if the image data associated with a low-latency configuration is used, or if at least one flag is transmitted in at least one header associated with the image, one or more method steps are not performed.

41. The method according to any one of claims 1 to 40, wherein if the image data relates to screen content, one or more method steps are not performed.

42. The method according to claim 41, wherein if the image data is related to screen content and one or more reference frames are intraframes, the one or more method steps are not performed.

43. The method according to claim 41 or 42, wherein the image data is associated with screen content if one or more tools are signaled and / or enabled in the bitstream, and the one or more tools may include one of the following: another tool associated with screen content, a conversion skip tool, an intrablock copy (IBC), and a palette.

44. The method according to claim 43, wherein the one or more tools are enabled according to the data associated with the hierarchy levels in the bitstream.

45. The method according to claim 44, wherein the data is one or more of a sequence parameter set (SPS), a picture parameter set (PPS), a picture header, and a slice header.

46. The method according to claim 43, wherein the one or more tools are signaled and / or enabled for a reference frame or for at least one block in the image.

47. The method according to claim 46, wherein the one or more tools are signaled and / or enabled for at least one block in the time area of ​​the reference frame.

48. The method according to claim 46 or 47, wherein the one or more method steps are not performed if the number of blocks having the one or more tools enabled and / or signaled is greater than a predetermined number.

49. The method according to any one of claims 41 to 48, as dependent on claim 7, wherein one or more steps relating to at least one of the first method and the second method are not performed.

50. The method according to claim 49, wherein not performing one or more steps of the second method comprises disabling the step of claim 6 and at least one of the steps of claims 3 to 5.

51. A method for encoding image data into a bitstream containing data indicating that the image data is divided into a plurality of blocks according to a coding tree, or decoding the image data from a bitstream, wherein the blocks in the coding tree may be divided according to a plurality of partitions including a triple-tree partition and a quad-tree partition, and the method To determine whether the triple tree split was performed at a level higher than the current block level, If the aforementioned three-tree split is performed, then the aforementioned two-tree split for the current block shall be permitted. A method of having.

52. The method according to claim 51, wherein the determination is performed by obtaining the value of the multitree depth at a level above the current block.

53. A device for decoding image data from a bitstream, wherein the device is configured to perform the method described in any one of claims 1 to 52.

54. An apparatus for encoding image data into a bitstream, wherein the apparatus is configured to perform the method described in any one of claims 1 to 52.

55. A computer program configured to perform the method described in any one of claims 1 to 52 during execution.

Images