Temporal prediction of partition mode

Abstract

A method and apparatus for encoding or decoding video are provided, wherein segmentation information of at least one first block of a reference image for a current image is stored, and is used to predict the segmentation of at least one current block for the current image when encoding or decoding the current block. In some embodiments, the segmentation information includes at least information representing a segmentation pattern for dividing the at least one first block into smaller blocks and / or information indicating a segmentation direction for segmenting the at least one first block or a segmentation direction from the segmentation of a larger block that produces the first block.

Description

[0001] This application claims the benefit of European application 24305492.1, filed on 29 March 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This embodiment generally relates to video compression. Specifically, it relates to a method and apparatus for encoding or decoding images or videos. More specifically, this embodiment relates to temporal prediction of segmentation patterns for video blocks. Background Technology

[0003] To achieve high compression efficiency, image coding schemes and video coding schemes typically employ prediction and transform to utilize spatial and temporal redundancy in video content. Intra-frame or inter-frame prediction is usually used to leverage intra-frame or inter-frame image correlations, followed by transform, quantization, and entropy coding of the differences between the original and predicted blocks (often referred to as prediction error or prediction residual). In inter-frame prediction, motion vectors used for motion compensation are typically predicted based on a motion vector predictor. To reconstruct the video, the compressed data is decoded through the inverse processes corresponding to entropy coding, quantization, transform, and prediction. Summary of the Invention

[0004] According to one aspect, a method for encoding video is provided. The method includes: storing partitioning information of at least one first block of a reference image for a current image, the partitioning information including at least information indicating that the at least one first block is divided into smaller blocks; and encoding at least one current block of the current image using predictions of partitioning of the at least one current block based on at least a portion of the stored partitioning information.

[0005] According to another aspect, an apparatus for encoding video is provided. The apparatus includes one or more processors operable to: store partitioning information of at least one first block of a reference image for a current image, the partitioning information including at least information indicating that the at least one first block will be divided into smaller blocks; and encode at least one current block of the current image using predictions of partitioning of the at least one current block based on at least a portion of the stored partitioning information.

[0006] According to one aspect, a method for decoding video is provided. The method includes: storing partitioning information of at least one first block of a reference image for a current image, the partitioning information including at least information indicating that the at least one first block is divided into smaller blocks; and decoding at least one current block of the current image using a prediction of partitioning of the at least one current block based on at least a portion of the stored partitioning information.

[0007] According to another aspect, an apparatus for decoding video is provided. The apparatus includes one or more processors operable to: store partitioning information of at least one first block of a reference image for a current image, the partitioning information including at least information indicating that the at least one first block will be divided into smaller blocks; and decode at least one current block of the current image using predictions of partitioning of the at least one current block based on at least a portion of the stored partitioning information.

[0008] According to one aspect, a method for encoding or decoding video is provided. The method includes: storing partitioning information of at least one first block of a reference image for a current image, the partitioning information including at least information indicating the direction of a dividing line that divides the at least one first block into smaller blocks or is used to obtain the at least one first block from a larger block; and encoding or decoding at least one current block of the current image using a prediction of partitioning of the at least one current block based on at least a portion of the stored partitioning information.

[0009] According to another aspect, an apparatus for encoding or decoding an image or video is provided. The apparatus includes one or more processors operable to: store partitioning information of at least one first block of a reference image for a current image, the partitioning information including at least information indicating the direction of a dividing line that divides the at least one first block into smaller blocks or is used to obtain the at least one first block from a larger block; and encode or decode at least one current block of the current image using a prediction of the partitioning of the at least one current block based on at least a portion of the stored partitioning information.

[0010] The above aspects can be used individually or in combination. Further variations and embodiments of the above aspects are presented in this document.

[0011] One or more embodiments also provide a computer program including instructions that, when executed by one or more processors, cause the one or more processors to perform any of the methods for encoding or decoding video according to any of the embodiments described herein. One or more embodiments of this disclosure also provide a non-transitory computer-readable medium and / or a computer-readable storage medium storing instructions thereon for encoding or decoding video according to the methods described herein.

[0012] One or more embodiments also provide a computer-readable storage medium having a bit stream generated according to the methods described herein stored thereon.

[0013] One or more embodiments also provide a method and apparatus for sending or receiving a bit stream generated according to the above method.

[0014] According to one aspect, an apparatus is provided, comprising: means for decoding video according to any of the above embodiments; and at least one of: (i) an antenna configured to receive or transmit a signal including data representing the video, (ii) a band limiter configured to limit the signal to a frequency band including data representing the video, and (iii) a display configured to display the video.

[0015] In one variation, the device includes at least one of a television, a mobile phone, a tablet computer, and a set-top box. Attached Figure Description

[0016] Figure 1A A block diagram of a system in which aspects of this embodiment may be implemented, according to one embodiment, is shown.

[0017] Figure 1B A block diagram of a system according to another embodiment in which aspects of this embodiment may be implemented is shown.

[0018] Figure 1C A block diagram of a system according to another embodiment in which aspects of this embodiment may be implemented is shown.

[0019] Figure 2 A block diagram is shown of an embodiment of a video encoder in which aspects of this embodiment may be implemented.

[0020] Figure 3 A block diagram is shown of an embodiment of a video decoder in which aspects of this embodiment may be implemented.

[0021] Figure 4 Examples of different partitioning types in VVC and ECM are shown.

[0022] Figure 5 An example of recursive signaling for partitioning is shown.

[0023] Figure 6 An example of a geometric dividing line description is shown.

[0024] Figure 7 An example of intra-frame sub-partitioning that depends on block size is shown.

[0025] Figure 8 An exemplary sub-block transformation is shown.

[0026] Figure 9 An example of sub-block temporal motion vector prediction is shown.

[0027] Figure 10 An example of a method for encoding video according to an embodiment is shown.

[0028] Figure 11 An example of a method for decoding video according to an embodiment is shown.

[0029] Figure 12 An example of a method for determining block segmentation information of a reference image is shown.

[0030] Figure 13 An example of a method for encoding or decoding the block division of the current image using partitioning information from a reference image, according to an embodiment, is shown.

[0031] Figure 14 An example of a method for encoding or decoding the block division of the current image using partitioning information from a reference image, according to another embodiment, is shown.

[0032] Figure 15 This shows an example of the selection of a time-domain region in the partition information buffer of the reference frame used by the current block.

[0033] Figure 16 An example of two remote devices communicating over a communication network is shown, based on this principle.

[0034] Figure 17 The syntax of an example signal based on this principle is shown.

[0035] Figure 18 This shows the target in the PI buffer. Figure 4 The example shown is of the QT depth and MTT depth stored in the CTU(440).

[0036] Figure 19 An example is shown. Figure 4 An example of the composite depth determined by CTU(440) is shown.

[0037] Figure 20 An example is shown. Figure 4 Examples of vertical and horizontal composite depths determined by CTU(440) are shown.

[0038] Figure 21 The example illustrates the use of a PI buffer for... Figure 4 The example shown is of the QT depth, MTT depth, and index value stored in the CTU(440).

[0039] Figure 22 The example illustrates the use of a PI buffer for... Figure 4 The example shown is of the QT depth and MTT depth stored in the CTU(440). Detailed Implementation

[0040] This application describes various aspects, including tools, features, embodiments, models, methods, etc. Many of these aspects are specifically described and are generally described in a manner that may sound restrictive, at least for the purpose of illustrating the various features. However, this is for clarity of description and does not limit the application or scope of these aspects. In fact, all the different aspects can be combined and interchanged to provide more aspects. Furthermore, these aspects can be combined and interchanged with aspects described in earlier applications.

[0041] The aspects described and envisioned in this application can be implemented in a variety of different forms. The following... Figure 1A , Figure 1B , Figure 1C , Figure 2 and Figure 3 Some embodiments have been provided, but other embodiments are also contemplated. Figure 1A , Figure 1B , Figure 1C , Figure 2 and Figure 3 The discussion does not limit the breadth of implementations. At least one aspect generally relates to video encoding and decoding, and at least another aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions stored thereon for encoding or decoding video data according to any of the methods, and / or computer-readable storage media having a bitstream generated according to any of the methods stored thereon.

[0042] In this application, the terms “reconstruction” and “decoding” are used interchangeably, the terms “pixel” and “sample” are used interchangeably, and the terms “image”, “picture” and “frame” are used interchangeably.

[0043] This document describes various methods, each including one or more steps or actions for implementing the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined. Furthermore, terms such as "first" and "second" can be used in various embodiments to modify elements, components, steps, operations, etc., e.g., "first decoding" and "second decoding." Unless specifically required, the use of these terms does not imply an ordering of the modified operations. Therefore, in this example, the first decoding need not be performed before the second decoding, and can be performed, for example, before, during, or within a time period overlapping with the second decoding.

[0044] Various aspects of this invention are not limited to VVC or HEVC, and can be applied to, for example, other standards and recommendations (whether pre-existing or developed in the future), as well as any extensions of such standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically excluded, the various aspects described in this application may be used alone or in combination.

[0045] Figures 1A to 1C A block diagram illustrating examples of systems in which various aspects and embodiments can be implemented is shown. Any of systems 100A, 100B, or 100C can be implemented as a device including the various components described below and configured to perform one or more aspects described in this application. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. For example, in at least one embodiment, the processing and encoder / decoder elements of system 100 are distributed across multiple ICs and / or discrete components. In various embodiments, systems 100A, 100B, or 100C are communicatively coupled to other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, systems 100A, 100B, or 100C are configured to implement one or more aspects described in this application.

[0046] Figure 1A A block diagram illustrating an example of a system in which various aspects and embodiments may be implemented. System 100A includes at least one processor 110 configured to execute instructions loaded therein for implementing various aspects, such as those described in this application. Processor 110 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 100A includes at least one memory 120 (e.g., a volatile memory device and / or a non-volatile memory device), including but not limited to EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash memory, disk drives, and / or optical disk drives. As a non-limiting example, memory 120 may include internal storage devices, attached storage devices, and / or network-accessible storage devices. Processor 110 may be interconnected with memory 120 via interconnect bus 115.

[0047] The program code to be loaded onto processor 110 to execute the various aspects described in this application is then loaded into memory 120 for execution by processor 110.

[0048] In some embodiments, the memory within processor 110 is used to store program code instructions and provides working memory for processing required during encoding or decoding. Inputs to the components of system 100A can be provided through various input devices (not shown). Both processor 110 and memory 120 may have one or more additional interconnects to external connections.

[0049] Figure 1B A block diagram illustrating an example of a system 100B in which various aspects and embodiments can be implemented is shown. System 100B includes, as referenced... Figure 1A The processor 110 and memory 120 are described. The inputs to the components of system 100B can be obtained through the following combination... Figure 1C Further, various input devices are provided as shown in block 105. Such input devices include, but are not limited to: (i) a radio frequency (RF) section that receives, for example, RF signals transmitted over the air by a broadcasting company, (ii) a component (COMP) input terminal (or a set of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Figure 1B Other examples not shown include composite video.

[0050] Various components can be interconnected and data can be transmitted therebetween using a suitable connection arrangement 115 (e.g., an internal bus known in the art, including an I2C bus, wires, and printed circuit boards).

[0051] System 100B includes a communication interface 150, which enables communication with other devices via a communication channel 190. The communication interface 150 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 190. The communication interface 150 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 190 may be implemented, for example, within a wired and / or wireless medium.

[0052] System 100B can provide output signals to various output devices, including displays, speakers, and other peripheral devices. The output devices can be communicatively coupled to system 100B via dedicated connections through corresponding interfaces 160, 170, and 180. Alternatively, the output devices can be connected to system 100B via communication interface 150 using communication channel 190.

[0053] Figure 1CA block diagram illustrating an example of a system 100C according to another embodiment, in which various aspects and embodiments may be implemented. The elements of the system 100C may be implemented individually or in combination in a single integrated circuit, multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing elements and encoder / decoder elements of the system 100C are distributed across multiple ICs and / or discrete components.

[0054] System 100C includes, as referenced Figure 1A or Figure 1B The processor 110 and memory 120 are mentioned above.

[0055] System 100C includes storage device 140, which may include non-volatile memory and / or volatile memory, including but not limited to EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash memory, disk drive, and / or optical disk drive. As a non-limiting example, storage device 140 may include internal storage devices, attached storage devices, and / or network-accessible storage devices.

[0056] System 100C includes an encoder / decoder module 130 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 130 may include its own processor and memory. The encoder / decoder module 130 represents a module that may be included in the device for performing encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. Alternatively, the encoder / decoder module 130 may be implemented as a separate element of system 100C or may be incorporated into processor 110 as a combination of hardware and software as known to those skilled in the art.

[0057] Program code to be loaded onto processor 110 or encoder / decoder 130 to execute the various aspects described in this application may be stored in storage device 140 and subsequently loaded onto memory 120 for execution by processor 110. According to various embodiments, one or more of processor 110, memory 120, storage device 140, and encoder / decoder module 130 may store one or more of various items during execution of the processes described in this application. These stored items may include, but are not limited to, input data (images, videos, volumetric content), decoded data (images, videos, volumetric content) or portions of decoded data, bitstreams, matrices, variables, and intermediate or final results from processing equations, formulas, operations, and operational logic.

[0058] In some embodiments, the memory within processor 110 and / or encoder / decoder module 130 is used to store instructions and provides working memory for processing required during encoding or decoding. However, in other embodiments, one or more of these functions are performed using memory external to the processing device (e.g., the processing device may be processor 110 or encoder / decoder module 130). External memory may be memory 120 and / or storage device 140, such as volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store the television's operating system. In at least one embodiment, fast external volatile memory, such as RAM, is used as working memory for data encoding and decoding operations such as MPEG-2, HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Various Video Coding (also known as H.266, a standard developed by JVET (Joint Video Experts Group))).

[0059] It can be accessed through various input devices (as shown in box 105), in Figure 1B The same applies to box 105, which provides input to components of system 100C. Such input devices in system 100B or 100C include, but are not limited to, (i) an RF section that receives radio frequency (RF) signals, for example, transmitted over the air by a broadcaster, (ii) component input terminals (or a set of COMP input terminals), (iii) universal serial bus (USB) input terminals, and / or (iv) high-definition multimedia interface (HDMI) input terminals. (Not mentioned in the original text) Figure 1B or Figure 1C Other examples shown include composite videos.

[0060] In various embodiments, the input device of block 105 in system 100B or 100C has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting the signal band to a band), (ii) down-converting the selected signal, (iii) further band-limiting it to a narrower band to select, for example, a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section in various embodiments includes one or more elements to perform these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners that perform various of these functions, including, for example, down-converting received signals to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band. Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0061] Additionally, the USB and / or HDMI terminals may include corresponding interface processors for connecting system 100B or 100C to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) can be implemented as needed, for example, in a separate input processing IC or processor 110. Similarly, various aspects of USB or HDMI interface processing can be implemented as needed within a separate interface IC or within processor 110. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements (including, for example, processor 110 and encoder / decoder 130, which operates in conjunction with memory and storage elements) to process the data streams as needed for presentation on the output device.

[0062] Various components of systems 100A, 100B, or 100C can be housed within an integrated housing. Within the integrated housing, the various components enable 100A, 100B, or 100C to be interconnected and to transmit data therebetween using suitable connection means 115 (e.g., internal buses (including I2C buses), wiring, and printed circuit boards known in the art).

[0063] Similarly, with Figure 1BSimilar to 100B, system 100C includes a communication interface 150, which enables communication with other devices via a communication channel 190. The communication interface 150 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 190. The communication interface 150 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 190 may be implemented, for example, within a wired and / or wireless medium.

[0064] In various embodiments, a Wi-Fi network (such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers)) is used to stream data to system 100B or 100C. In these embodiments, the Wi-Fi signal is received via a communication channel 190 and a communication interface 150 suitable for Wi-Fi communication. The communication channel 190 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other embodiments use a set-top box to provide streaming data to system 100B or 100C, transmitting data via an HDMI connection in input box 105. Still other embodiments use an RF connection in input box 105 to provide streaming data to system 100B or 100C. As described above, various embodiments provide data in a non-streaming manner. Furthermore, various embodiments use wireless networks other than Wi-Fi (such as cellular networks or Bluetooth networks).

[0065] System 100C can provide output signals to various output devices, including a display 165, a speaker 175, and other peripheral devices 185. The display 165 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 165 can be used in televisions, tablets, laptops, mobile phones, or other devices. The display 165 can also be integrated with other components (e.g., as in a smartphone) or standalone (e.g., an external display for a laptop). Other peripheral devices 185 include standalone digital video discs (or digital versatile discs) (referred to as DVRs for both terms), disc players, stereo systems, and / or lighting systems. Various embodiments use one or more peripheral devices 185 that provide functionality based on the output of system 100C. For example, a disc player performs the function of playing the output of system 100C.

[0066] In various embodiments, control signals are transmitted between system 100C and display 165, speaker 175, or other peripheral devices 185 using signaling such as AV.Link, CEC, or other communication protocols that enable device-to-device control with or without user intervention. Output devices may be communicatively coupled to system 100C via dedicated connections through corresponding interfaces 160, 170, and 180. Alternatively, output devices may be connected to system 100C via communication interface 150 using communication channel 190. Display 165 and speaker 175 may be integrated into a single unit within an electronic device (e.g., a television) along with other components of system 100C. In various embodiments, display interface 160 includes a display driver, such as a timing controller chip.

[0067] For example, if the RF portion of input 105 is part of a separate set-top box, then display 165 and speaker 175 may alternatively be separated from one or more other components. In various embodiments where display 165 and speaker 175 are external components, the output signal may be provided via a dedicated output connection, such as an HDMI port, USB port, or COMP output.

[0068] In any of systems 100A, 100B, and 100C, embodiments may be implemented by a computer program product comprising code instructions that implement any of the embodiments described herein. The computer program product may be computer software implemented by processor 110 or by hardware, or a combination of hardware and software. As a non-limiting example, embodiments may be implemented by one or more integrated circuits. The memory 120 of any of systems 100A, 100B, and 100C may be of any type suitable for the technical environment and may be implemented using any suitable data storage technology, such as optical storage devices, magnetic storage devices, semiconductor-based storage devices, fixed-state memory, and removable memory, as non-limiting examples. The processor 110 of any of systems 100A, 100B, or 100C may be of any type suitable for the technical environment and, as a non-limiting example, may include one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0069] Figure 2 An example of a block-based hybrid video encoder 200 is shown. Variations of this encoder 200 are conceivable, but for clarity, not all anticipated variations are described below when describing the encoder 200.

[0070] In some embodiments, Figure 2 It also shows the HEVC standard or VVC standard ( Universal Video Coding (VVC), standard Compliant with ITU-T H.266, ISO / IEC 23090-3, 2020 An improved encoder or an encoder that employs a technology similar to HEVC or VVC (such as the ECM (Enhanced Compression Model) encoder being developed by JVET (Joint Video Exploration Team)).

[0071] Before being encoded, the video sequence may undergo pre-coding processing (201), such as applying color transformations to the input color picture (e.g., converting from RGB 4:4:4 to YCbCr 4:2:0), performing remapping of the input picture components to obtain a more compression-resistant signal distribution (e.g., using histogram equalization of the color components), or resizing the picture (e.g., downsampling). Metadata may be associated with pre-processing and appended to the bitstream.

[0072] In encoder 200, the image is encoded by encoder elements as described below. The image to be encoded is divided (202) into units such as coding units (CUs) or blocks. In this disclosure, different terms may be used to refer to such units or blocks resulting from the division of the image. These terms may be coding unit or CU, coding block or CB, luminance CB or block. CTU (coding tree unit) refers to a set of blocks, a set of units, or a set of coding units (CUs). In some embodiments, a CTU may be considered as a block, or a unit itself.

[0073] Each unit is encoded using, for example, intra-frame or inter-frame modes. When a unit is encoded in intra-frame mode, intra-frame prediction (260) is performed. In inter-frame mode, motion estimation (275) and compensation (270) are performed. Intra-frame and / or inter-frame modes may include several different sub-modes. For example, intra-frame modes may include directional intra-frame prediction, template-based intra-frame mode derivation prediction, intra-block copy prediction, or other modes that perform spatial domain prediction on the sample values ​​of the unit. Inter-frame modes may include skip modes, merge modes (in which motion information is derived from a list of motion candidates and no motion vector prediction residual is encoded), inter-frame modes (in which motion information is derived from a list of motion candidates and further refined by encoding the motion vector prediction residual or by template matching performed on both sides of the encoder and decoder), and other inter-frame modes are also possible. The encoder decides (205) which of the intra-frame or inter-frame modes to use to encode the unit. When different intra-frame and / or inter-frame modes are possible, the encoder determines (205) which of the intra-frame or inter-frame modes to use. The encoder indicates the intra-frame / inter-frame decision by, for example, one or more syntax elements indicating the prediction mode. The encoder can also mix (205) intra-frame and inter-frame prediction results, or mix results from different intra-frame / inter-frame prediction methods. For example, the prediction residual is calculated by subtracting (210) the prediction block from the original image block.

[0074] The motion refinement module (272) refines the motion field of a block using an existing reference frame, without referencing the original block. The motion field of a region can be viewed as the set of motion vectors of all pixels within that region. If the motion vectors are based on sub-blocks, the motion field can also be represented as the set of motion vectors of all sub-blocks within that region (all pixels within a sub-block have the same motion vector, and the motion vectors may differ from sub-blocks). If a single motion vector is used for the region, the motion field of that region can also be represented by that single motion vector (all pixels within that region have the same motion vector).

[0075] Then, the predicted residual is transformed (225) and quantized (230). The quantized transform coefficients, motion vectors, and other syntax elements are entropy encoded (245) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass the transform and quantization, i.e., directly encode the residual without applying transform or quantization.

[0076] The encoder decodes (reconstructs) the encoded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The image block is reconstructed by combining the decoded prediction residual with the prediction block (255). An in-loop filter (265) is applied to the reconstructed image to perform one or more of, for example, deblocking filtering, SAO (Sampling Adaptive Shift) filtering, and ALF (Adaptive Loop Filtering) filtering, thereby reducing coding artifacts. The filtered image is stored in a reference image buffer (280). Such a filtered image is also referred to as a reference image in the following description.

[0077] Figure 3 A block diagram of a video decoder 300 is shown. In decoder 300, the bitstream is decoded by decoder elements as described below. The video decoder 300 typically performs operations related to... Figure 2 The encoding process described herein is the inverse of the decoding process. Encoder 200 typically also performs video decoding as part of the video data encoding.

[0078] Specifically, the input to the decoder includes a video bitstream, which can be generated by the video encoder 200. The bitstream is first entropy decoded (330) to obtain transform coefficients, motion vectors, and other encoded information. Frame segmentation information indicates how the frame is segmented. The decoder can then segment (335) the frame based on the decoded frame segmentation information. The transform coefficients are dequantized (340) and inverse transformed (350) to decode the prediction residuals. The image blocks are reconstructed by combining the decoded prediction residuals with the prediction blocks (355).

[0079] Prediction blocks can be obtained from intra-frame prediction (360) or motion-compensated prediction (i.e., inter-frame prediction) (375). In a manner similar to that of the encoder, intra-frame prediction and / or inter-frame prediction can include several different sub-modes. The decoder obtains (370) predictor blocks based on one or more syntax elements indicating the prediction modes among the available intra-frame and inter-frame modes. The decoder can mix (370) intra-frame prediction results and inter-frame prediction results, or mix results from multiple intra-frame / inter-frame prediction methods. Before motion compensation, the motion field can be refined (372) using an already available reference frame. An in-loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference frame buffer (380). Note that for a given frame, the contents of the reference frame buffer 380 on the decoder 300 side are the same as the contents of the reference frame buffer 280 on the encoder 200 side for the same frame.

[0080] The decoded image can undergo further post-decoding processing (385), such as inverse color transformation (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping of the remapping process performed in pre-encoding processing (201), or resizing the reconstructed image (e.g., downsampling). Post-decoding processing can utilize metadata derived in pre-encoding processing and signaled in the bitstream.

[0081] Some embodiments described herein relate to encoding or decoding video, wherein temporal prediction of block partitions in an image is performed using partitioning information from a reference image derived from the video. In some embodiments described herein, the partitioning information from the reference image includes not only partition tree information but also sub-coding unit partitioning information. In some embodiments described herein, the predicted partitioning data includes at least one of the following: adding constraints to partitioning decisions based on partitioning information from the reference image, or encoding or decoding partitioning data based on partitioning information from the reference image.

[0082] The embodiments described herein can be implemented, for example, in the partitioning module of a video encoder or video decoder, or in a module for determining prediction modes or an entropy encoder / decoder module. For example, the embodiments described herein can be implemented... Figure 2 Implemented in the image segmentation module 202, prediction mode module 205, intra-frame prediction module 260, or entropy coding module 245 of the video encoder 200, or in Figure 3 Implemented in the partitioning module 335, intra-frame prediction module 360, prediction mode module 370 or entropy decoder 330 of the video decoder 300.

[0083] Division in VVC and ECM In VVC ( Universal Video Coding (VVC), ITU-T H.266 (ex H.VVC) | ISO / IEC 23090-3, 08 / 2020) and ECM ( M. Coban, R.-L. Liao, K. Naser, J. Ström, L. Zhang, "Algorithm Description of Enhanced Compression Model 10 (ECM 10) (Algorithm description), document JVET-AE2025, 32nd Meeting, Hannover, DE, 13–20 October 2023 In this process, the screen is first divided into Code Tree Units (CTUs). Next, recursive signaling can be used to divide each CTU into Code Units (CUs). For example... Figure 4 As shown, there are three types of CTU partitioning: quadtree (QT) partitioning 410, binary tree (BT) partitioning 420, and ternary tree (TT) partitioning 430. The latter two types (binary tree partitioning and ternary tree partitioning) can be horizontal or vertical. In the following text, BT and TT can be indiscriminately classified as multi-type trees (MTT).

[0084] Figure 5 It shows Figure 4 The example shown illustrates the recursive signaling of the current CTU (440), where QS, TS, BS, and x represent the signaling for quadtree partitioning (or quadtree splitting), ternary tree partitioning (or ternary tree splitting), binary tree partitioning (or binary tree splitting), and no partitioning, respectively. VVC and ECM also signal the minimum and maximum allowed partition depths for each partition type at the sequence level. For example, if the minimum and maximum partition depths for ternary tree partitioning are 1 and 3 respectively, then TS cannot be signaled when the partition depth is 0 or greater than 3. Figure 5 In the illustrated example, the maximum segmentation depth for all types is 4. If the segmentation depth has reached 4, no signal is needed to indicate a no-segmentation flag (in...). Figure 5 (500 locations in the text).

[0085] In VVC, the `split_cu_flag` syntax element is signaled or inferred at each coding tree node. If `split_cu_flag` is 0, the coding tree leaf node is reached and treated as a CU; otherwise, the `split_qt_flag` syntax element is signaled or inferred to select between QT and MTT splits. If `split_qt_flag` equals 1, QT splits are applied; otherwise, the `mtt_split_cu_vertical_flag` syntax element is signaled or inferred to select between horizontal and vertical split directions; and the `mtt_split_cu_binary_flag` syntax element is signaled or inferred to select between BT and TT splits. When signaling coding tree-related syntax elements, some additional conditions allow for reduced signaling redundancy. For example, if `split_qt_flag` is 0 for a CU whose size is no larger than the maximum CU size of a BT split but larger than the maximum CU size of a TT split, then `mtt_split_cu_binary_flag` is not signaled and is inferred to be 1. Another scenario involves inferring these four coding tree segmentation information flags based on syntax elements in the SPS or Picture Header (PH). For example, the maximum allowed CU size for BT segmentation and the maximum allowed CU size for TT segmentation can be derived from relevant parameters in the SPS or PH. In addition to reducing syntax redundancy using SPS and PH syntax elements, VVC also reduces syntax redundancy between BT and TT by removing redundant partitioning patterns through syntax constraints.

[0086] All four syntax elements are context-encoded using Context-Based Adaptive Binary Arithmetic Coding (CABAC). It can be expected that the partitioning decisions of the coding tree nodes are closely related to the partitioning structure of the current coding tree node and its neighboring nodes. Therefore, in VVC, 9, 6, 5, and 4 context variables are assigned to the entropy encoding of split_cu_flag, split_qt_flag, mtt_split_cu_vertical_flag, and mtt_split_cu_binary_flag, respectively. To derive the context index increment ctxInc of the current CB, information (width, height, QT depth) of the spatially adjacent CBs above and to the left of the current CB is used.

[0087] Time-domain partitioning prediction Storage of encoded information The video codec stores some reconstructed frames into the decoding frame buffer (DPB) (e.g.) Figure 2 and Figure 3 The reference frames in the DPB (280 and 380) are used as a reference for constructing inter-frame predictions.

[0088] In VVC and ECM, additional decoding information can be stored for each reference frame, such as the encoding mode (intra-frame or inter-frame) and intra-frame prediction mode in the IPM buffer, and motion information (motion vectors and reference indices) in the MI buffer.

[0089] exist G. Laroche, P. Onno, "AhG12: On ECM Temporal partitioning prediction (AhG12: prediction of ECM time domain partitioning), document JVET-AG0126, 33rd Meeting, by teleconference, 17–26 January 2024 Additional information also includes partitioning information (PI), QT depth, and MTT depth, which are stored at the center of each block aligned with the 16x16 grid to reduce the memory required to store these variables.

[0090] against Figure 4 Examples of QT depth and MTT depth stored in CTU(440) are shown below. Figure 18 As shown.

[0091] Time-domain partitioning prediction exist JVET-AG0126 In the process, the stored partitioning information is used to derive the following variables: • Minimum time-domain QT depth • Average time-domain QT depth • Maximum time-domain MTT depth • Average time-domain MTT depth.

[0092] These variables are derived from a temporal region. This temporal region is a juxtaposition region from the same reference frame (juxtaposed reference frame) used for the temporal motion predictor. The juxtaposition region of the current block is equal to the CTU size. This juxtaposition region is centered according to the center of the current block of the frame to be encoded or decoded. Within this juxtaposition region, each 16x16 cell on the grid of the juxtaposed reference frame is considered to determine these four variables. These variables allow for constraining or reducing the signaling of the current block's segmentation mode (QT, BT, TT). QT segmentation is allowed only when the current QT depth is strictly less than the temporal minimum QT depth minus 1. No segmentation, QT segmentation, and TT segmentation are allowed when the current QT depth is strictly less than the temporal average QT depth minus 1, and BT segmentation is allowed if TT segmentation has already been selected in the parent node.

[0093] These four variables can also be used to locally adjust the maximum MTT depth. The value of the maximum MTT depth increases when the current QT depth is equal to the average temporal QT depth minus 1, and the maximum temporal MTT depth is greater than the maximum MTT depth of the current frame, and the temporal average value of the maximum MTT depth is equal to half the maximum MTT depth of the reference frame's header, and if the reference frame is not an intra-frame reference frame, or when the current QT depth is equal to the average temporal QT depth, and the temporal maximum MTT depth is greater than the maximum MTT depth of the current frame, and the temporal average value of the MTT depth is greater than or equal to half the maximum MTT depth of the reference frame's header.

[0094] The value of the maximum MTT depth decreases when the maximum QT depth of the current frame is greater than the maximum MTT depth, and when the maximum temporal MTT depth is equal to the maximum MTT depth of the current frame, and when the average temporal QT depth is less than the QT depth of the current frame, or if the maximum temporal MTT depth is strictly less than the maximum multitree depth of the current node.

[0095] These four variables can also be used to adapt the order of the split syntax elements. When the current QT depth is strictly less than the average time-domain QT depth, the syntax elements (split_cu_flag, split_qt_flag) that signal whether the encoding unit has been split and whether the split is a quadtree split are encoded in this order. Otherwise, the syntax elements (split_qt_flag, split_cu_flag) are encoded in the order stated above.

[0096] Some CU encoding modes also provide CU partitioning (e.g., GPM, ISP, SBT), which allows CU to be divided into subCUs.

[0097] Geometric Partitioning (GPM) In VVC, geometric partitioning modes are supported for inter-frame prediction. A CU-level flag is used to signal this geometric partitioning mode as a merging mode, accompanying other merging modes. When using this mode, a geometrically positioned straight line divides the CU into two parts or partitions. The position of this partitioning line is mathematically derived based on the angle and offset parameters of the specific partition, such as... Figure 6 As shown. Angle φ i Quantization is performed between 0 and 360 degrees with a step size of 11.25 degrees (32 in total). Figure 6 The figure shows an angle φ i and distance ρ i Description of geometric segmentation.

[0098] In the CU (Cubic Convolutional Frame), each part of the geometric partition uses its own motion for inter-frame prediction or intra-frame prediction. In VVC (Variable Frame Convolutional Frame), each partition only allows unidirectional prediction, while ECM (Enhanced Frame Convolutional ...

[0099] Intra-frame subdivision (ISP) Using ISP for luma, the luma intra-prediction block is vertically or horizontally divided into 2 or 4 sub-partitions depending on the block size. In ECM, a syntax element ( ispMode The value is encoded as 0 or 1 (horizontal or vertical sub-partition). Figure 7 Examples of sub-partition possibilities are shown for 4×8 and 8×4 CUs (a) and CUs larger than 4×8 or 8×4 (b). Each partition is predicted using the same intra-frame mode, but they can use reconstructed samples from the top or left sub-partitions.

[0100] Subblock Transformation (SBT) In VVC and ECM, when using the SBT transform mode, the residual can be divided into two parts, and only one part is transformed and encoded, while the other residual part is set to zero, such as... J. Park, B. Jeon, "Non-CE6: Simplified SBT mode coding (Non-CE6) (document JVET-P0216) 16th Meeting: Geneva, CH, 1–11 October 2019) As stated in and as described in Figure 8 (For a transform unit having 1 / 2 the size of the current encoding unit) is shown.

[0101] The size of the encoding section can be half or a quarter of the size of the current encoding unit. The size of the encoding section is indicated by the following four flags: A value of 1 for cu_sbt_flag indicates that subblock transformation is used for the current coding unit. A value of 0 indicates that subblock transformation is not used for the current coding unit.

[0102] A value of 1 for cu_sbt_quad_flag indicates that for the current coding unit, the subblock transform includes a transform unit with a size of 1 / 4 that of the current coding unit.

[0103] A value of 0 for cu_sbt_quad_flag indicates that for the current coding unit, the sub-block transform includes a transform unit with a size of 1 / 2 that of the current coding unit.

[0104] A value of 1 for cu_sbt_horizontal_flag indicates that the current coding unit is horizontally divided into two transform units. A value of 0 for cu_sbt_horizontal_flag indicates that the current coding unit is vertically divided into two transform units.

[0105] A cu_sbt_pos_flag value of 1 indicates that the bitstream does not contain tu_cbf_luma, tu_cbf_cb, and tu_cbf_cr of the first transform unit in the current coding unit. A cu_sbt_pos_flag value of 0 indicates that the bitstream does not contain tu_cbf_luma, tu_cbf_cb, and tu_cbf_cr of the second transform unit in the current coding unit.

[0106] Sub-block temporal motion vector prediction (SbTMVP) Inter-frame prediction can construct motion candidates from reconstructed spatial neighbors, historical motion vector buffers, or juxtaposition positions in the MI buffer of juxtaposed frames. In one variant, displacement blocks in the MI buffer (displacements derived from the MV (motion vector) of an adjacent reconstructed CU, see [link to relevant documentation]) can be used. Figure 9 (A1 in the original text). The MI buffer memory contains block-aligned 4x4 motion information (MV and reference index). If the current block is larger than 4x4, the sub-block motion vectors can be used to predict the current prediction, such as... Figure 9 As shown.

[0107] In ECM, the SbTMVP vector can be selected from two different juxtaposed reference frames and several displacement blocks.

[0108] Some embodiments described herein provide a method for encoding or decoding video that uses temporal partitioning prediction, in which information from a partitioning tree from a reference image is used, as well as sub-CUs or CU-based partitioning information.

[0109] This paper proposes to enhance / improve temporal partitioning prediction and CU-based partitioning pattern prediction by storing additional information. Some embodiments include storing CU and sub-CU partitioning line information (PI), and / or partitioning type and depth, and using CU-based partitioning (e.g., partitioning provided by coding modes GPM, ISP, SBT) for temporal partitioning prediction of tree partitioning. In this document, tree partitioning refers to a partitioning pattern applied to a CTU to obtain a given CU or block to be encoded or decoded in the current image, while CU-based partitioning refers to a partitioning pattern applied to a CU that divides the CU into multiple sub-CUs or blocks.

[0110] In some embodiments, the stored PI is also used to predict sub-CU partitions (e.g., GPM partitions). Some variants extend the temporal storage / propagation to predict coded partitions of transform tree type or prediction unit type.

[0111] Some embodiments propose storing depth information or CU partitions, which allows for direct derivation of CU dimensions.

[0112] Storage of CU segmentation information and sub-CU segmentation line information In this embodiment, it is proposed to store the CU segmentation information and the final sub-CU segmentation line division information in a segmentation information (PI) buffer. In this embodiment, it is proposed to store information indicating that the CU has been further segmented into sub-units.

[0113] In another embodiment, information indicating the segmentation direction is proposed. More specifically, in these variations, the QT depth and MTT depth of the CUs determined for the reference image, stored in the PI buffer, are enriched by one or more data indicating the segmentation line direction for dividing the CU into smaller blocks (sub-CUs) or for obtaining the CU from the larger block (CTU).

[0114] In a further embodiment, the two embodiments described above may be combined to provide the two indications as separate index values, or provided in a combined index as described below.

[0115] The partitioning information of the CUs of the reference image is stored in the PI buffer of the reference image using a grid resolution (cell size, such as 4x4, 8x8, or 16x16) that may be larger than one sample. The partitioning information determined for the CU containing that cell is assigned to each cell.

[0116] An indication that a CU has been further subdivided can be obtained using QT depth or MTT depth values ​​(by incrementing these values ​​based on the further subdivisions applied to the CU, as described below). Figure 12 The data can be stored using the specified index value, or stored using a specific index value that indicates the further segmentation.

[0117] In one variant, the partitioning information stored in the PI buffer may include index values ​​indicating the partitioning direction. For example, the partitioning direction may correspond to vertical (e.g., 0), horizontal (e.g., 1), and several other values ​​(e.g., >1) ultimately used for other partitions (e.g., GPM non-vertical partition lines and non-horizontal partition lines). Figure 21 It shows the allocation from Figure 4 An example of the index value (index_dir) of the CU obtained from the CTU(440) segmentation. For example, this index value indicates the last direction of the CTU segmentation.

[0118] In a further variation, the index value may also include an indication of the segmentation direction for CU-based partitioning. For example, such as Figure 21 As shown, CU 2110 is encoded using a geometric partitioning pattern, and the index value for direction is set to the value of the partition line (gpm_dir) in the GPM pattern. As another example, such as... Figure 21As shown, the CU 2120 generated by the TT segmentation is further encoded using ISP, and the index value for orientation assigned to the CU is set to the (isp_V) value indicating the vertical ISP.

[0119] The index value can also indicate whether the current cell contains a segment, and the type of segment. The index value indicating the segment direction can be stored in the PI buffer in association with the MTT depth.

[0120] In another variation, the partitioning information stored in the PI buffer can also include CTU (depth) partitions and cell-level (typically, a cell can be a 4x4 sample, but other resolutions can be used) split sequence indicators. That is, each cell in the PI buffer can be assigned a value that can be computed to produce a concatenated binary mask of all partitioning patterns for the CU (Complex Unit) considering the cell. Assuming the partitioning operation sequence leading to this CU is denoted as {split[i], i=0,..,N-1}, the corresponding split series value is calculated as follows:

[0121] in: The array `splitMode[.]` represents the set of segmentation modes for generating the considered CU. It is available on both the encoder and decoder sides, and the `depth` value ranges from 0 to the depth of the considered CU. This is the number of bits used in memory to represent the partitioning pattern. For example, it can be equal to 5.

[0122] In this way, by comparing the corresponding values ​​of two adjacent 4x4 blocks... splitSeries The encoder and decoder can determine whether two 4x4 blocks belong to the same CU, thereby determining whether there is a dividing line between them and the direction of the dividing line (horizontal or vertical).

[0123] The stored in the PI buffer splitSeries Another alternative to this information approach is to use segmentation patterns specifically designed for identifying sub-CU segmentation patterns. That is, in the above equation... splitMode[depth] Some specific values ​​can indicate the GPM segmentation mode, ISP segmentation mode, SBT segmentation mode, or some motion segmentation mode (such as SbTMVP).

[0124] Another embodiment proposes storing depth information of alternative types in a temporal partitioning information buffer (PI buffer). Alternative type partitioning information can be stored in the PI buffer and further used for temporal prediction of CU-based segmented data.

[0125] In one variant, the composite depth value is defined as follows: The composite depth is set to zero for the root of the coding tree (i.e., the coding tree node corresponding to the CTU). When the CU is partitioned according to the QT (quadtree) partitioning mode, the composite depth is incremented by 2 and allocated to the child coding tree nodes. When the CU is partitioned according to the BT (binary tree) partitioning mode, the composite depth is incremented by 1 and allocated to the child coding tree nodes. When the CU is partitioned according to the TT (tritree) partitioning mode, the composite depth incremented by 1 is allocated to the child coding tree nodes corresponding to the middle child block region of the parent block region. Furthermore, the composite depth incremented by 2 is allocated to the two child coding tree nodes corresponding to the child block regions in the parent block region that are different from the middle child block. Figure 19 This demonstrates how to apply this variant. Figure 4 The composite depth is obtained by CTU(440).

[0126] As can be seen, the advantage of composite depth is that the composite depth value determined for a CU represents the size of the CU, while MTT depth does not always correspond one-to-one with block size. That is, the same MTT depth value may be assigned to two CUs of different sizes. Therefore, the composite depth metric defined above is more suitable than MTT depth values ​​for performing temporal prediction of block segmentation information and predicting whether blocks in the current frame have been segmented.

[0127] In further variations, the composite depth metric can be calculated not only for CU segmentation but also for segmentation within the CU itself (i.e., via SBT, GPM, and ISP modes). For example, in the case of ISP segmentation, if the segment is divided into 2 TUs, the composite depth increases by 1; if it is divided into 4 TUs, it increases by 2. In the case of SBT segmentation, if the segment is divided into 2 TUs of the same size, the composite depth increases by 1 / 2 and 3 / 2 respectively for the larger and smaller TUs. In the case of GPM segmentation into two motion partitions, the composite depth increases by 1 for each of the two PUs.

[0128] In another variation, instead of storing depth information and propagating it in the temporal domain as described above, some block size information can be used. That is, the size of a block in a given frame can be information stored in a temporal buffer of partitioning information. This can include the surface storing the block.

[0129] In another variation, composite depth values ​​can be defined for each direction (horizontal and vertical). For example, for a horizontal BT segmentation, only the horizontal composite value increments by 1, and for a vertical BT segmentation, only the vertical composite value increments by 1. For example, for QT, both the horizontal and vertical composite values ​​increment by 2. For example, for a horizontal TT segmentation, the horizontal composite value of the middle sub-CU increments by 1, and the horizontal composite value of all sub-CUs except the middle CU increments by 2; while for a vertical TT segmentation, the vertical composite value of the middle sub-CU increments by 1, and the vertical composite value of all sub-CUs except the middle CU increments by 2. Figure 20 This demonstrates how to apply this variant. Figure 4 The horizontal (H) and vertical (V) composite depths were obtained from the CTU (440).

[0130] In addition to QT depth and MTT depth or alternative depth values, the partitioning information mentioned in the above embodiments / variations can be stored in the PI buffer.

[0131] Figure 10 An example of a method 1000 for encoding video according to this embodiment using any of the above variations is shown. At 1010, partitioning information of at least one first block of a reference image for the current image of the video is stored in a PI buffer, wherein the partitioning information includes at least information indicating that the at least one first block is divided into smaller blocks. This partitioning information corresponds to partitioning data for encoding the at least one first block, which is available on both the encoder and decoder sides. The stored information includes any information mentioned in the above variations.

[0132] At 1020, at least one current block of the current image is encoded using a prediction of the partitioning of the at least one current block based on at least a portion of the stored partitioning information.

[0133] Figure 11 An example of a method 1110 for decoding video according to this embodiment using any of the above variations is shown. At 1110, partitioning information of at least one first block of a reference image for the current image of the video is stored in a PI buffer, wherein the partitioning information includes at least information indicating that the at least one first block is divided into smaller blocks. At 1120, at least one current block of the current image is decoded using a prediction of the partitioning of the at least one current block based on at least a portion of the stored partitioning information.

[0134] It can be seen that two types of partitioning information can be stored: one type corresponds to the partitioning pattern of the CTU that generates the current CU (the partitioning pattern that generates the CU and is applied sequentially to blocks larger than the CU), and the other type corresponds to the partitioning pattern applied to the current CU, for example, by using an encoding pattern that partitions the CU into sub-CUs (blocks smaller than the CU).

[0135] The first type of segmentation mode that generates the current CU can include, for example, quadtree segmentation, binary tree segmentation, or ternary tree segmentation. The second type of segmentation mode that divides the CU into smaller blocks can include geometric partitioning (GPM), intra-fractional sub-partitioning (ISP), sub-block transform segmentation (SBT) (according to which at least one sub-block is transformed and the other is set to zero), and sub-block temporal motion vector predictor segmentation (SbTMVP). The second type of segmentation mode can also include transform unit segmentation mode that divides the coding unit into transform units (TU), and predictor unit segmentation mode that divides the coding unit into predictor units (PU).

[0136] Depending on the variant used, encoding or decoding the current block based on predictions obtained from at least a portion of the stored segmentation information may include: determining a segmentation for the at least one current block based on predictions obtained from at least a portion of the stored segmentation information, and / or encoding / decoding a segmentation for the at least one current block based on predictions obtained from at least a portion of the stored segmentation information. (In conjunction with...) Figures 12 to 14 This describes further details and variations regarding temporal partitioning prediction for encoding / decoding the current block using CU-based (block-based) partitioning.

[0137] As described above, temporal partitioning prediction relies on information from the partition tree, which is solely controlled by the CTU to be partitioned into multiple CUs. In the variant described herein, CU-based partitioning, which determines how the control CUs can be further partitioned into several prediction portions or how the residuals can be further partitioned into several rectangular groups, is also used for temporal prediction of CTU partitioning. As mentioned above, CU-based partitioning can be provided by any coding scheme that provides the partitioning of CUs into smaller blocks.

[0138] Some examples of CU-based segmentation can include geometric partitioning mode (GPM) segmentation, intra-frame sub-partitioning (ISP), sub-block temporal motion vector prediction (SbTMVP), and / or sub-block transform (SBT) segmentation.

[0139] In one variant, when there is a CU-based segmentation, the stored QT depth and / or MTT depth values ​​can be incremented by 1 for further segmentation of the CU in the reference image. Figure 12An example of using this variant is shown. Figure 12 A method 1200 is shown for determining segmentation information for a block when a CU-based segmentation exists for a block of a reference image. This variation relates to the determination of segmentation information stored in a PI buffer, thus using the same process at both the encoder and decoder.

[0140] For example, when encoding or decoding a block of a reference image, method steps are performed such that all encoded data for that block is known at this stage. At 1210, the QT depth and MTT depth of the block are obtained based on the segmentation that produced it. At 1220, it is determined whether the block has been further segmented into sub-blocks for encoding. This can be determined by examining the encoding mode used to encode / decode the block. The encoding mode may involve a prediction mode (e.g., GPM, ISP, SbTMVP), or a mode for transforming residuals (e.g., SBT, or segmenting the block into TUs), or segmenting the block into multiple PUs for prediction. If this is the case, then at 1230, at least one of the QT depth and MTT depth is incremented.

[0141] In one variant, when one or more sub-blocks of a block are further subdivided into sub-cells, the QT or MTT depth assigned to the cell is incremented by an amount equal to the number of sub-cells that produced the block. In another variant, the QT depth is incremented when a quadtree is used to partition the block or sub-blocks, while the MTT depth is incremented when a binary or ternary tree-like partition is used. At 1240, the modified depth (QT or MTT) is stored in the PI buffer.

[0142] Figure 22 It shows when the application Figure 12 The method is for Figure 4 Examples of the resulting QT and MTT depths for the CTU(440). For example, for the CU 2210, which is assumed to be encoded using a geometric partitioning mode, the MTT depth is increased by 1. For the CU 2220, which is obtained from the CU's level TT and is assumed to be encoded using an ISP mode (divided into 4 sub-units), the MTT depth is increased by 2. Other examples and increment values ​​may exist.

[0143] The modified depths (QT and MTT) stored in the PI buffer can then be used to determine the CTU partitioning of the current image (e.g., using a combination of the above). JVET-AG0126 Further, similar methods can be used by defining average QT and MTT depths, minimum QT and MTT depths, and maximum QT and MTT depths. For example, signaling for CTU segmentation modes can be reduced (thus reducing the likelihood of segmentation), or the maximum or minimum QT and MTT depths provided by the picture header or SPS can be adjusted to provide, for example, more segmentation possibilities.

[0144] According to another embodiment, the stored partitioning information (CTU-based partitioning information and / or CU-based partitioning information) can be used to perform temporal prediction of CU-based partitioning and sub-CU partitioning.

[0145] In this embodiment, it is proposed that when encoding the partitioning of a CU, the stored partitioning information (PI) is used to predict the segmentation information for a given CU-based partitioning pattern. This prediction may correspond to a ranking of the most likely partitioning patterns or the selection of a cabac context for encoding / decoding the partitioning syntax elements. In some variations, intra-frame, the co-location information in the PI buffer can be used; while inter-frame, motion vectors can be used to pick up the stored partitioning information at the shifted position in the PI buffer of the reference frame. In one variation, motion vectors of adjacent blocks of the CU can be used.

[0146] Figure 14 An example of a method 1400 for encoding or decoding blocks of a current image using partitioning information from a reference image, according to a variant of this embodiment, is shown. In this variant, a stored PI is used to select a CABAC context for entropy encoding / decoding one or more syntax elements of a block. At 1410, for the current block to be encoded / decoded, the stored partitioning information for that block is obtained from the PI buffer. A context for a context-based arithmetic encoder / decoder is determined to indicate one or more syntax elements for partitioning the block, wherein the obtained stored partitioning information is used to determine the context. At 1420, once a CABAC context has been determined for one or more syntax elements, the determined CABAC context is used to entropy encode / decode the one or more syntax elements.

[0147] In one example, this variant can be used for syntax elements that indicate the horizontal or vertical subdivisions of the ISP encoding mode for the block. ispMode Perform entropy encoding / decoding. At 1410, determine whether the PI obtained for the current image block contains directional vertical segmentation data or horizontal segmentation data. This determination depends on how the PI obtained for that block is stored. For example, it can be determined regarding... Figure 21 The index value being discussed indicates whether it represents a vertical or horizontal segmentation. In another example, when using the binary mask described above... splitSeries At that time, it can be based on splitSeries The deepest segmentation pattern is determined by this. When using horizontal and vertical composite depth values, the orientation of the block is the direction that provides the highest composite depth.

[0148] When a block has multiple index values ​​available because it contains cells with different index values, a single index value can be obtained for the block by using the index value associated with the highest depth or the index value of the cell at the same or closest depth to the current block. Similarly, for splitSeries, the value associated with the highest depth or the value of the cell at the same or closest depth to the current block is used.

[0149] Alternatively, given the horizontal composite depth value and the vertical composite depth value, the average horizontal composite depth value and the average vertical composite depth value can be determined.

[0150] If PI indicates directional vertical segmentation of data, then it is used for syntax elements. ispMode The CABAC context used for encoding / decoding can be set to 0; otherwise, if PI indicates directional horizontal data splitting, it is used for syntax elements. ispMode The CABAC context used for encoding / decoding can be set to 1; otherwise, it is used for syntax elements. ispMode The number of CABAC contexts used for encoding / decoding can be set to two. Each CABAC context is associated with a different probabilistic model. In this way, the probabilistic model used to encode syntax elements can be adapted based on the partitioning information stored in the reference image.

[0151] This variant can be extended in a similar manner to any other syntax element used to encode / decode blocks and indicate other encoding patterns for horizontal or vertical segmentation of the blocks.

[0152] For example, when encoding / decoding a block using SBT mode, different syntax elements can be used depending on the PI segmentation data obtained for that block. cu_sbt_flag The CABAC context for entropy encoding / decoding. If the PI obtained for the block contains segmentation data, the first CABAC context is used to encode / decode the syntax element; otherwise, the second CABAC context is used.

[0153] Figure 13An example of method 1300 for encoding or decoding blocks of a current image using partitioning information from a reference image, according to another variation, is shown. In this variation, the most probable values ​​of syntax elements indicating the encoding mode are provided using the stored PI of the reference image, and a signal is used to indicate whether the syntax elements indicating the encoding mode correspond to the most probable values. At 1310, for the current block to be encoded / decoded, stored partitioning information for that block is obtained from the PI buffer. Using the obtained stored partitioning information, the most probable values ​​of one or more syntax elements indicating the partitioning for the current block are determined. At 1320, the one or more syntax elements are encoded / decoded using the most probable values.

[0154] In one example, when encoding / decoding the current block using this encoding mode, this variant can be used to encode / decode the syntax element indicating the SBT encoding mode (the SBT flags mentioned above). At 1310, it is determined whether the PI obtained for the current block indicates the use of segmented data (e.g., CU-based segmented data). This determination can be made by considering the segment depth (QT depth, MTT depth, composite depth, splitSeries) obtained for the block from the PI buffer. If the depth is higher than a given value, it is determined that the PI contains segmented data; otherwise, it does not. The given value can be 0 or other values. For example, the depth of the current block can be considered, and the depth of the current block can be compared with the depth obtained from the PI buffer. If the depth obtained from the PI buffer is higher than the depth of the current block, it is determined that the PI obtained for the block contains segmented data; otherwise, it does not. These are merely examples, and other conditions can be used instead or in addition to these.

[0155] When a block has multiple depth values ​​available because it contains cells with different depth values, a single depth value for the block can be obtained by using the highest depth value of the cell or by averaging the depth values ​​associated with the cell.

[0156] If the PI indicator uses split data, then the syntax element cu_sbt_flag The most likely value is 1, otherwise it is 0.

[0157] In another example, it can be determined whether the PI contains vertically or horizontally split data. If the PI contains horizontally split data, then the syntax element... cu_sbt_horizontal_flag The most likely value is 1, otherwise it is 0, and if PI contains vertically split data, then the syntax element... cu_sbt_horizontal_flag The most likely value is 0, otherwise it is 1.

[0158] Similarly, in another example, when encoding / decoding the current block using GPM, the most likely value of the segmentation direction can be inferred. For example, if the PI obtained for the current block contains a segmentation direction different from the horizontal or vertical direction, then the closest segmentation line of the GPM pattern (by...) Figure 6 The angle and offset definitions are set as the most likely dividing line. Then, a signal is used to notify the syntax element indicating whether the dividing line of the GPM pattern is the most likely value; if not, a signal is used to notify the index used to indicate the dividing line. In a variant, the set of possible GPM dividing lines is reordered according to their proximity to the dividing direction stored in the PI.

[0159] In the above variation, the PI obtained for the current block from the PI buffer can be obtained from the time domain region in the PI buffer that is juxtaposed with the current block.

[0160] In another variation, this can be adapted based on whether the current block is encoded using intra-frame mode or inter-frame mode. Advantageously, such as Figure 15 As shown, the temporal region in the PI buffer (1510) used to derive the PI of the current block (1520) is not limited to a juxtaposed region (1530) in a juxtaposed reference frame. Instead, in the case of inter-frame prediction of the current block, the temporal region in the PI buffer (1510) associated with the reference frame can be selected using motion vector information (1540) and the PI (1510) of the reference frame. For example, the motion vector information (MV) can be selected from the juxtaposed MI buffer, a decoded spatial candidate (e.g., similar to...). Figure 9 The value is obtained from either the current block's MV (if it has been decoded) or the MV of the current block.

[0161] This time domain region may not be limited to the current block size (1550), but may include a larger region (1560) surrounding the current block size.

[0162] In some video codecs, different from VVC or ECM (such as HEVC), some partitioning of coding units may include recursively dividing coding units into transform units (also known as transform trees). That is, all transform units belonging to the same CU typically share the same coding mode information and may also share the same prediction information. Partitioning a CU into TUs then involves encoding / decoding each TU by sequentially applying transform, quantization, and entropy coding of quantization coefficients to each TU in the CU under consideration. This transform partitioning of CUs was supported in previous HEVC video standards, and the transform partitioning was explicitly signaled in the bitstream. Such transformation partitioning into TUs may exist in future video standards after VVC. This variant proposes to extend the embodiments described herein to transform unit partitioning information. Transform unit partitioning information may be stored in the PI buffer of a reference frame. The PI stored in the PI buffer of the reference frame can be used to predict transform unit partitioning information for encoding / decoding blocks of the current image having that reference frame as a juxtaposed reference frame, regardless of whether TU partitioning information is stored in the PI buffer.

[0163] In some video codecs, different from VVC or ECM, some partitioning of coding units may include dividing coding units into so-called prediction units (PUs). That is, all prediction units belonging to the same CU typically share the same coding mode (intra-frame, inter-frame, intra-block copy) information. However, each PU carries its own prediction parameters (e.g., motion data). Typically, partitioning a CU into PUs can be a symmetric binary tree, an asymmetric binary tree, or a quadtree. This PU partitioning of CUs was supported in previous HEVC video standards, and this PU partitioning of CUs was explicitly signaled in the bitstream. It may exist in future video standards after VVC. This variant proposes extending the embodiments described herein to prediction unit partitioning information. Prediction unit partitioning information can be stored in the PI buffer of a reference frame. The PI stored in the PI buffer of the reference frame can be used to predict prediction unit partitioning information for encoding / decoding blocks of the current image with that reference frame as a juxtaposed reference frame, regardless of whether PU partitioning information is stored in the PI buffer.

[0164] In one embodiment, such as Figure 16As shown, in a transmission context between two remote devices A and B based on a communication network NET, device A includes a processor associated with RAM and ROM, configured to implement a method for encoding video according to any embodiment described herein, and device B includes a processor associated with RAM and ROM, configured to implement a method for decoding video according to any embodiment described herein. According to one example, the network is a broadcast network adapted to broadcast / transmit encoded video from device A to a decoding device, including device B.

[0165] Figure 17 An example of the syntax for signals transmitted via a packet-based transport protocol is shown. Each transmitted packet P includes a header H and a payload PAYLOAD. In some embodiments, the payload PAYLOAD may include data representing video encoded according to any of the embodiments described above. The payload may also include any signaling as described above.

[0166] Various implementations involve decoding. As used herein, "decoding" can include, for example, all or part of a process performed on a received encoded sequence to produce a final output suitable for display. In various embodiments, such processes include one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also include, or alternatively include, processes performed by a decoder of the various implementations described herein, such as entropy decoding of a sequence of binary symbols to reconstruct image or video data.

[0167] As a further example, in one embodiment, "decoding" refers only to entropy decoding; in another embodiment, "decoding" refers only to differential decoding; in yet another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding; and in yet another embodiment, "decoding" refers to the entire image reconstruction process including entropy decoding. Whether the phrase "decoding process" is intended to specifically refer to a subset of operations or to refer generally to a broader decoding process will be clear based on the specific context of the description and is believed to be fully understood by those skilled in the art.

[0168] Various implementations involve encoding. In a manner similar to the discussion of “decoding” above, “encoding,” as used herein, can include, for example, all or part of a process performed on an input video sequence to produce an encoded bitstream. In various embodiments, such processes include one or more processes typically performed by an encoder, such as segmentation, differential coding, transform, quantization, and entropy coding. In various embodiments, such processes also include, or alternatively include, processes performed by an encoder of the various implementations described herein, such as determining resampling filter coefficients and resampling the decoded frame.

[0169] As a further example, in one embodiment, "encoding" refers only to entropy encoding; in another embodiment, "encoding" refers only to differential encoding; and in yet another embodiment, "encoding" refers to a combination of differential and entropy encoding. Whether the phrase "encoding process" is intended to specifically refer to a subset of operations or to refer generally to a broader encoding process will be clear based on the specific context of the description and is believed to be fully understood by those skilled in the art.

[0170] Note that the syntax elements used here are descriptive terms. Therefore, the use of other syntax element names is not excluded.

[0171] This disclosure has described various types of information, such as syntax, that can be sent or stored. This information can be encapsulated or arranged in various ways, including those common in video standards, such as placing the information in SPS, PPS, NAL units, headers (e.g., NAL unit headers, picture headers, or slice headers), or SEI messages. Other methods are also available, including those common for system-level or application-level standards, such as placing the information in one or more of the following: a. SDP (Session Description Protocol), a format used to describe multimedia communication sessions for session announcements and invitations, such as the format described in RFC and used in conjunction with RTP (Real-Time Transport Protocol) transmission; b. DASH MPD (Media Rendering Description) descriptors, such as the DASHMPD descriptors used in DASH and transmitted via HTTP, which are associated with a rendering or a set of renderings to provide additional features to the content rendering; c. RTP header extensions, such as those used during RTP streaming; d. ISO basic media file formats, such as the format used in OMAF, use boxes, which are object-oriented building blocks defined by unique type identifiers and lengths, also referred to as "atoms" in some specifications; e. An HLS (HTTP Live Streaming) manifest transmitted over HTTP. The manifest can be associated, for example, with a version or set of versions of the content to provide characteristics of the version or set of versions.

[0172] When an accompanying figure is presented as a flowchart, it should be understood that a block diagram of the corresponding apparatus is also provided. Similarly, when an accompanying figure is presented as a block diagram, it should be understood that a flowchart of the corresponding method / process is also provided.

[0173] Some implementations involve rate-distortion optimization. Specifically, during the encoding process, a balance or trade-off between rate and distortion is typically considered, usually under constraints of computational complexity. Rate-distortion optimization is generally formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. Different approaches exist to address the rate-distortion optimization problem. For example, one approach might be based on extensive testing of all encoding options, including all considered modes or encoding parameter values, and a complete evaluation of their encoding costs and the correlated distortion of the reconstructed signal after encoding and decoding. Faster methods can also be used to save encoding complexity, particularly by calculating approximate distortion based on prediction or prediction of the residual signal (rather than the reconstructed signal). A hybrid of these approaches can also be used, such as using approximate distortion only for some possible encoding options and full distortion for others. Other methods evaluate only a subset of possible encoding options. More generally, many methods employ any of a variety of techniques to perform optimization, but this optimization is not necessarily a complete evaluation of both encoding costs and correlated distortion.

[0174] The implementations and aspects described herein can be implemented, for example, in methods or processes, apparatuses, software programs, data streams, or signals. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the features in question can also be implemented in other forms (e.g., apparatuses or programs). For example, an apparatus can be implemented with appropriate hardware, software, and firmware. The method can be implemented, for example, in a processor (which generally refers to a processing device), including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. The processor also includes communication devices, such as computers, cellular phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0175] References to "an embodiment," "an embodiment," "an implementation," or "an implementation," as well as other variations, mean that a particular feature, structure, characteristic, etc., described in connection with the embodiment is included in at least one embodiment. Therefore, the phrases "in an embodiment," "in an embodiment," "in an implementation," or "in an implementation," and any other variations appearing in various places throughout this application, do not necessarily refer to the same embodiment.

[0176] Furthermore, this application may involve "determining" various types of information. Determining information may include, for example, one or more of the following: estimation information, calculation information, prediction information, and information retrieved from memory.

[0177] Furthermore, this application may relate to "accessing" various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, and estimating information, or more of these.

[0178] Additionally, this application can refer to "receiving" various types of information. Like "accessing," receiving is intended to be a broad term. Receiving information can include, for example, accessing information and retrieving information (e.g., from memory). Furthermore, "receiving" is generally referred to in one or more ways during operations such as storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0179] It should be understood that the use of any of the following “ / ”, “and / or”, and “…at least one of” (e.g., in the cases of “A / B”, “A and / or B”, and “at least one of A and B”) is intended to cover the selection of only the first listed option (A), or only the second listed option (B), or the selection of both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C”, such wording is intended to include selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to multiple listed items.

[0180] Furthermore, as used herein, the term "signaling" refers, among other things, to instructing the corresponding decoder to provide certain information. Thus, in one embodiment, the same parameters are used on both the encoder and decoder sides. Therefore, for example, the encoder can send (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. Conversely, if the decoder already has specific parameters as well as other parameters, signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select specific parameters. Bit savings are achieved in various embodiments by avoiding the transmission of any actual functionality. It should be understood that signaling can be implemented in various ways. For example, in various embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. While the foregoing refers to the verb form of the term "signaling," the word "signal" can also be used as a noun herein.

[0181] As will be apparent to those skilled in the art, implementations can generate various signals formatted to carry information, such as information that can be stored or transmitted. This information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream of the described embodiment. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding the data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. It is well known that signals can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

[0182] Numerous embodiments have been described herein. The features of these embodiments may be provided individually or in any combination, covering a wide range of claim categories and types.

Claims

1. A method comprising: The system stores segmentation information for at least one first block of a reference image for the current image. This segmentation information includes at least information representing a first segmentation pattern that divides the at least one first block into smaller blocks. The first segmentation pattern is applied to the at least one first block via an encoding pattern used to decode the at least one first block. At least one current block of the current image is decoded using a prediction of the partition of the at least one current block based on at least a portion of the stored partition information.

2. An apparatus comprising one or more processors, said one or more processors being operable to: The system stores segmentation information for at least one first block of a reference image for the current image. This segmentation information includes at least information representing a first segmentation pattern that divides the at least one first block into smaller blocks. The first segmentation pattern is applied to the at least one first block via an encoding pattern used to decode the at least one first block. At least one current block of the current image is decoded using a prediction of the partition of the at least one current block based on at least a portion of the stored partition information.

3. A method comprising: The system stores segmentation information for at least one first block of a reference image for the current image. This segmentation information includes at least information representing a first segmentation pattern that divides the at least one first block into smaller blocks. The first segmentation pattern is applied to the at least one first block via an encoding pattern used to encode the at least one first block. At least one current block of the current image is encoded using a prediction of the partitioning of the at least one current block based on at least a portion of the stored partitioning information.

4. An apparatus comprising one or more processors, said one or more processors being operable to: The system stores segmentation information for at least one first block of a reference image for the current image. This segmentation information includes at least information representing a first segmentation pattern that divides the at least one first block into smaller blocks. The first segmentation pattern is applied to the at least one first block via an encoding pattern used to encode the at least one first block. At least one current block of the current image is encoded using a prediction of the partitioning of the at least one current block based on at least a portion of the stored partitioning information.

5. The method of claim 1 or the apparatus of claim 2, wherein decoding at least one current block comprises at least one of the following: - Determine the partition for the at least one current block based on the prediction obtained from at least a portion of the stored partition information, and - Decode the partition for the at least one current block based on the prediction obtained from at least a portion of the stored partition information.

6. The method or apparatus of claim 5, wherein decoding the partition for the at least one current block based on the prediction obtained from at least a portion of the stored partition information comprises: - Using at least a portion of the stored partition information, determine the most likely value of one or more syntax elements indicating the partition for the at least one current block. - Use the most likely value to decode the one or more syntax elements.

7. The method or apparatus of claim 5, wherein decoding the partition for the at least one current block based on the prediction obtained from the at least a portion of the stored partition information comprises: - To instruct one or more syntax elements for the partition of the at least one current block to determine a context for a context-based arithmetic decoder, the context is determined using at least a portion of the stored partition information. - Use the determined context to decode the one or more syntax elements.

8. The method according to any one of claims 1, 3 and 5 to 7, or the apparatus according to any one of claims 2 and 4 to 7, wherein the division information includes information indicating the direction of the dividing line, the dividing line dividing the at least one first block into smaller blocks or for obtaining the at least one first block from a larger block.

9. The method or apparatus of claim 8, wherein the information indicating the direction of the dividing line is an index value indicating at least one of a horizontal dividing line, a vertical dividing line, and a dividing line in a direction defined by a given angle other than the vertical or horizontal direction.

10. The method of any one of claims 1, 3 and 5 to 9, or the apparatus of any one of claims 2 and 4 to 9, wherein at least one piece of information representing a first segmentation pattern for dividing the at least one first block into smaller blocks is included in an index value, the index value further including an indication that at least one second segmentation pattern for dividing the larger blocks into larger blocks at a given depth is applied to the larger blocks.

11. The method according to any one of claims 1, 3 and 5 to 10, or the apparatus according to any one of claims 2 or 4 to 10, wherein the division of the current block comprises at least one of the following: one or more second segmentation patterns applied sequentially to generate a larger block of the current block, and one or more first segmentation patterns applied to the current block to divide the current block into smaller blocks.

12. The method or apparatus according to claim 10 or 11, wherein the at least one second segmentation mode includes at least one of quadtree segmentation, binary tree segmentation, ternary tree segmentation, and no segmentation.

13. The method according to claim 1 or 11, wherein the at least one first segmentation mode comprises at least one of the following: a geometric segmentation mode; an intra-frame sub-segmentation mode; a sub-block transform segmentation mode, wherein at least one of the sub-blocks is transformed and another of the sub-blocks is set to zero according to the sub-block transform segmentation mode; a sub-block temporal motion vector predictor segmentation mode; a transform unit segmentation mode that segments a coding unit into a transform unit; and a predictor unit segmentation mode that segments a coding unit into a predictor unit.

14. The method according to any one of claims 1, 3 and 8 to 13, wherein the information is stored in a partition buffer of each block at a given resolution of the reference image, the given resolution being at least one of a single sample, a 4x4 sample, an 8x8 sample, and a 16x16 sample.

15. The method according to any one of claims 1, 3, and 5 to 14, or the apparatus according to any one of claims 2 and 4, wherein storing partitioning information of at least one first block of a reference image for the current image comprises: - Obtain at least one depth of the second segmentation pattern of the at least one first block that generates the reference image. - In response to determining that the at least one first block is encoded using an encoding pattern that applies a first segmentation pattern that divides the at least one first block into smaller blocks, the depth of the at least one second segmentation pattern is increased by at least 1. - Store at least one depth of the second segmentation pattern in the segmentation information.

16. The method of any one of claims 1, 3 and 5 to 15, or the apparatus of any one of claims 2 and 4, wherein a region in the reference image is used to determine the prediction of the division of the at least one current block based on at least a portion of the stored division information, the region being juxtaposed with the at least one current block, or a motion-compensated region obtained using the motion vector of the at least one current block or the motion vector of a neighboring block of the current block.

17. The method according to any one of claims 1, 3 and 5 to 16, or the apparatus according to any one of claims 2 and 4, wherein the partitioning information includes a composite depth determined for the at least one first block, representing the size of the at least one first block.

18. The method or apparatus of claim 17, wherein the composite depth is recursively determined for the at least one first block according to one or more second segmentation patterns applied to generate one or more larger blocks of the at least one first block, and wherein the value for incrementing the composite depth at a given level depends on the type of the second segmentation pattern.

19. The method or apparatus of claim 17 or 18, wherein the composite depth is further determined for the at least one first block according to one or more first segmentation patterns applied to the at least one first block to divide the at least one first block into smaller blocks, and wherein the value for incrementing the composite depth at a given level depends on the type of the first segmentation pattern.

20. The method or apparatus according to any one of claims 17 to 19, wherein the composite depth comprises composite depth values ​​in each horizontal and vertical direction.

Images