Apparatus and method for video encoding and decoding

Abstract

A method for video encoding or decoding comprises determining a number of units to be assigned to a partition; indicating or inferring a number of explicit-sized partitions to be assigned; indicating a size or number of units in the explicit-sized partitions; and indicating or inferring a number of uniform-sized partitions to be assigned.

Description

Technical Field

[0001] The present invention relates to an apparatus, method and computer program for video encoding and decoding. Background Art

[0002] Video coding standards and specifications typically allow encoders to divide or partition coded pictures into subsets. In video coding, partitioning can be defined as dividing a picture or a sub-region of a picture into subsets (blocks) such that each element of the picture or a sub-region of a picture is located in exactly one of the subsets (blocks). For example, H.265 / HEVC introduces the concept of coding tree units (CTUs) with a default size of 64×64 pixels. Based on a quadtree structure, a CTU can contain a single coding unit (CU) or be recursively split into multiple smaller CUs of at least 8x8 pixels. H.265 / HEVC also confirms tiles (which are rectangular and contain an integer number of CTUs) and slices (defined based on slice segmentation, which contains an integer number of coding tree units, which are ordered consecutively in a tile scan and contained in a single NAL unit).

[0003] Versatile Video Coding (VVC) (MPEG-I Part 3) (aka ITU-T H.266) is a video compression standard developed by the Joint Video Experts Team (JVET) of the Moving Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) of the International Telecommunication Union (ITU) as the successor to HEVC / H.265 (formally ISO / IEC JTC1 SC29 WG11). The VVC partitioning scheme includes not only tiles, but also bricks, which can include one or more CTU rows within a tile. The introduction of bricks also affects the definition of slices.

[0004] Therefore, a rather complex syntax structure has been created for signaling the various options for tile and brick partitioning, which is sub-optimal in many respects, especially with regard to the bit rate required for the signaling. Summary of the invention

[0005] Now, in order to at least alleviate the above problems, an enhanced encoding method is introduced in this paper.

[0006] The method according to the first aspect includes: encoding an indication of tile columns and an indication of brick heights for one or more tile columns at a time in or along a bitstream; inferring potential tile rows when a tile row aligned by a picture is detected; inferring or indicating whether a boundary of a potential tile row is a boundary of a tile row; and encoding one or more pictures into a bitstream using the indicated tile columns, the indicated or inferred tile rows, and the indicated brick heights, wherein the one or more pictures are partitioned into a tile grid along the indicated tile columns and the indicated or inferred tile rows, the tiles in the tile grid comprising an integer number of coding tree units and being partitioned into one or more bricks, wherein a brick comprises an integer number of coding tree unit rows within the tile.

[0007] The method according to the second aspect comprises the following steps:

[0008] a) indicate the number of partitions to be assigned;

[0009] b) determining the number of cells to be assigned to the partition;

[0010] c) indicating whether the number of cells to be assigned are evenly assigned to the number of partitions; and if not,

[0011] d) indicates the number of cells to be assigned to the next partition, and

[0012] e) Repeat steps c) and d) until all cells have been assigned to partitions.

[0013] According to an embodiment, when the number of partitions that have not yet been assigned is greater than 1, said repeating steps c) and d) is performed.

[0014] According to an embodiment, the method further comprises: determining whether the number of unassigned cells is divisible by the number of unassigned partitions; and if so, indicating whether the number of unassigned cells is evenly assigned to the number of unassigned partitions.

[0015] According to an embodiment, if it is inferred or indicated that the number of not yet assigned cells is not evenly assigned to the number of not yet assigned partitions or the number of not yet assigned cells is not divisible by the number of not yet assigned partitions, step d) is followed by the following steps:

[0016] d1) reducing the number of cells not yet assigned to a partition by the indicated number; and

[0017] d2) Decrement the number of unassigned partitions by 1.

[0018] According to an embodiment, a partition is one or more of: a tile column, a tile row, a brick row of one or more tile columns, a brick row of tiles.

[0019] According to an embodiment, the unit is a rectangular block of samples of a picture.

[0020] The apparatus according to the third aspect comprises: a component for encoding, in a bitstream or along a bitstream at a time, an indication of a tile column and an indication of a brick height for one or more tile columns; a component for inferring a potential tile row after detecting a brick row aligned by a picture; a component for inferring that a boundary of a potential tile row is a boundary of a tile row or indicating whether a boundary of a potential tile row is a boundary of a tile row; and a component for encoding one or more pictures into a bitstream using the indicated tile columns, the indicated or inferred tile rows, and the indicated brick heights, wherein the one or more pictures are partitioned into a tile grid along the indicated tile columns and the indicated or inferred tile rows, the tiles in the tile grid comprising an integer number of coding tree units and being partitioned into one or more bricks, wherein a brick comprises an integer number of coding tree unit rows within a tile.

[0021] The apparatus according to the fourth aspect comprises: a component for indicating the number of partitions to be assigned; a component for determining the number of units to be assigned to the partitions; a component for indicating whether the number of units to be assigned are evenly assigned to the said number of partitions; a component for indicating the number of units to be assigned to the next partition in response to the said number of units to be assigned not being evenly assigned to the said number of partitions; and a component for repeating the indicating operation until all units have been assigned to the partitions.

[0022] The method according to the fifth aspect includes: decoding an indication of tile columns and an indication of brick heights for one or more tile columns from a bitstream or along a bitstream at a time; inferring potential tile rows after detecting a tile row aligned by a picture; inferring that a boundary of a tile row is a boundary of a tile row or indicating whether a boundary of a potential tile row is a boundary of a tile row; and decoding one or more pictures from a bitstream using the indicated tile columns, the indicated or inferred tile rows, and the indicated brick heights, wherein the one or more pictures are partitioned into a tile grid along the indicated tile columns and the indicated or inferred tile rows, the tiles in the tile grid comprising an integer number of coding tree units and partitioned into one or more bricks, wherein a brick comprises an integer number of coding tree unit rows within a tile.

[0023] The method according to the sixth aspect comprises the following steps:

[0024] a) decoding an indication of the number of partitions to be assigned;

[0025] b) determining the number of cells to be assigned to the partition;

[0026] c) determining whether the number of cells to be assigned are evenly assigned to the number of partitions; and if not,

[0027] d) determine the number of cells to be assigned to the next partition, and

[0028] e) Repeat steps c) and d) until all cells have been assigned to partitions.

[0029] The apparatus according to the seventh aspect comprises: a component for decoding an indication of a tile column and an indication of a brick height of one or more tile columns from a bitstream or along a bitstream at a time; a component for inferring a potential tile row after detecting a brick row aligned by a picture; a component for inferring that a boundary of a potential tile row is a boundary of a tile row or indicating whether a boundary of a potential tile row is a boundary of a tile row; and a component for decoding one or more pictures from a bitstream using the indicated tile columns, the indicated or inferred tile rows, and the indicated brick heights, wherein the one or more pictures are partitioned into a tile grid along the indicated tile columns and the indicated or inferred tile rows, the tiles in the tile grid comprising an integer number of coding tree units and being partitioned into one or more bricks, wherein a brick comprises an integer number of coding tree unit rows within a tile.

[0030] The apparatus according to the eighth aspect comprises: a component for decoding an indication of the number of partitions to be assigned; a component for determining the number of units to be assigned to the partitions; a component for determining whether the number of units to be assigned are evenly assigned to the said number of partitions; a component for determining the number of units to be assigned to the next partition in response to the said number of units to be assigned not being evenly assigned to the said number of partitions; and a component for repeating the determination operation until all units have been assigned to the partitions.

[0031] Other aspects relate to an apparatus comprising: at least one processor and at least one memory, wherein the at least one memory has codes stored thereon, which, when executed by the at least one processor, causes the apparatus to at least perform one or more embodiments of the above method and related embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] For a better understanding of the present invention, reference will now be made, by way of example, to the accompanying drawings, in which:

[0033] Figure 1 An electronic device using an embodiment of the present invention is schematically shown;

[0034] Figure 2 Schematically illustrates a user equipment suitable for adopting an embodiment of the present invention;

[0035] Figure 3Also schematically illustrated are electronic devices employing embodiments of the present invention, which use both wireless and wired network connections for connection;

[0036] Figure 4 Schematically illustrates an encoder suitable for implementing an embodiment of the present invention;

[0037] Figure 5a , Figure 5b , Figure 5c Some examples of partitioning a picture into coding tree units (CTUs), tiles, bricks, and slices are shown;

[0038] Figure 6 shows the syntax structure of the signaling of slice, picture and tile partitions according to H.266 / VVC draft 5;

[0039] Figure 7 A flowchart of an encoding method according to one aspect of the present invention is shown;

[0040] Figure 8 A flow chart showing an encoding method according to another aspect of the present invention;

[0041] Fig. 9 A flowchart of an encoding method according to an embodiment of the present invention is shown;

[0042] Fig.10 a. Fig.10 b. Fig.10 c shows some examples of tile and brick partitioning;

[0043] Fig.11 A schematic diagram of a decoder suitable for implementing an embodiment of the present invention is shown;

[0044] Fig.12 A flowchart of a decoding method according to an embodiment of the present invention is shown;

[0045] Fig.13 A flowchart showing a decoding method according to another embodiment of the present invention; and

[0046] Fig.14 A schematic diagram of an example multimedia communication system is shown in which various embodiments may be implemented. DETAILED DESCRIPTION

[0047] Suitable means and possible mechanisms for initiating viewpoint switching are described in further detail below. Figure 1 and Figure 2 ,in Figure 1A block diagram of a video encoding system according to an example embodiment is shown as a schematic block diagram of an exemplary apparatus or electronic device 50, which may include a codec according to an embodiment of the present invention. Figure 2 A layout of an apparatus according to an example embodiment is shown. Figure 1 and Figure 2 The elements will be explained next.

[0048] The electronic device 50 may be, for example, a mobile terminal or user equipment of a wireless communication system. However, it is to be understood that embodiments of the present invention may be implemented in any electronic device or apparatus that may require encoding and decoding or encoding or decoding video images.

[0049] The device 50 may include a housing 30 for containing and protecting the device. The device 50 may also include a display 32 in the form of a liquid crystal display. In other embodiments of the present invention, the display may be any suitable display technology suitable for displaying images or videos. The device 50 may also include a keypad 34. In other embodiments of the present invention, any suitable data or user interface mechanism may be adopted. For example, the user interface may be implemented as a virtual keyboard or data entry system as part of a touch-sensitive display.

[0050] The device may include a microphone 36 or any suitable audio input that may be a digital or analog signal input. The device 50 may also include an audio output device, which may be any one of a handset 38, a speaker, or an analog audio or digital audio output connection in embodiments of the present invention. The device 50 may also include a battery (or in other embodiments of the present invention, the device may be powered by any suitable mobile energy device such as a solar cell, a fuel cell, or a clockwork generator). The device may also include a camera capable of recording or capturing images and / or videos. The device 50 may also include an infrared port for short-range line-of-sight communication with other devices. In other embodiments, the device 50 may also include any suitable short-range communication solution, such as, for example, a Bluetooth wireless connection or a USB / Firewire wired connection.

[0051] The apparatus 50 may include a controller 56, a processor or processor circuitry for controlling the apparatus 50. The controller 56 may be connected to a memory 58 which, in embodiments of the invention, may store data in the form of image and audio data and / or may also store instructions for implementation on the controller 56. The controller 56 may also be connected to a codec circuitry suitable for performing encoding and decoding of audio and / or video data or assisting in encoding and decoding performed by the controller.

[0052] The apparatus 50 may also include a card reader 48 and a smart card 46, such as a UICC and a UICC reader, to provide user information and is suitable for providing authentication information for authenticating and authorizing the user at the network.

[0053] The device 50 may include a radio interface circuit system 52 connected to the controller and suitable for generating wireless communication signals for communicating with, for example, a cellular communication network, a wireless communication system, or a wireless local area network. The device 50 may also include an antenna 44 connected to the radio interface circuit system 52 to transmit radio frequency signals generated at the radio interface circuit system 52 to (multiple) other devices and receive radio frequency signals from (multiple) other devices.

[0054] The device 50 may include a camera capable of recording or detecting individual frames, which are then passed to a codec 54 or controller for processing. The device may receive video image data from another device for processing before transmission and / or storage. The device 50 may also receive images wirelessly or via a wired connection for encoding and decoding. The structural elements of the device 50 described above represent examples of components for performing corresponding functions.

[0055] Relative to Figure 3 , an example of a system in which embodiments of the present invention can be used is shown. System 10 includes multiple communication devices capable of communicating over one or more networks. System 10 may include any combination of wired or wireless networks, including but not limited to wireless cellular telephone networks (such as GSM, UMTS, CDMA networks, etc.), wireless local area networks (WLANs) such as defined by any IEEE 802.x standards, Bluetooth personal area networks, Ethernet local area networks, token ring local area networks, wide area networks, and the Internet.

[0056] System 10 may include wired and wireless communication devices and / or apparatuses 50 suitable for implementing embodiments of the present invention.

[0057] For example, Figure 3 The illustrated system shows a representation of the mobile telephone network 11 and the Internet 28. Connectivity to the Internet 28 may include, but is not limited to, long range wireless connections, short range wireless connections, and various wired connections including, but not limited to, telephone lines, cable lines, power lines, and similar communications pathways.

[0058] The example communication devices shown in system 10 may include, but are not limited to, electronic devices or apparatuses 50, a combination of a personal digital assistant (PDA) and a mobile phone 14, a PDA 16, an integrated messaging device (IMD) 18, a desktop computer 20, a notebook computer 22. The apparatus 50 may be stationary or mobile when carried by a moving individual. The apparatus 50 may also be located in a mode of transportation, including, but not limited to, a car, a truck, a taxi, a bus, a train, a ship, an airplane, a bicycle, a motorcycle, or any similar suitable mode of transportation.

[0059] Embodiments may also be implemented in a set-top box (i.e., a digital TV receiver that may or may not have a display or wireless capabilities), in a tablet or (laptop) personal computer (PC) (with a combination of hardware or software or encoder / decoder implementations), in various operating systems, and in chipsets, processors, DSPs, and / or embedded systems that provide hardware / software based encoding.

[0060] Some or other devices may send and receive calls and messages and communicate with service providers via a wireless connection 25 to a base station 24. The base station 24 may be connected to a network server 26 that allows communication between the mobile phone network 11 and the Internet 28. The system may include additional communication devices and various types of communication devices.

[0061] The communication device may communicate using various transmission technologies, including but not limited to code division multiple access (CDMA), global system for mobile communications (GSM), universal mobile telecommunications system (UMTS), time division multiple access (TDMA), frequency division multiple access (FDMA), transmission control protocol-internet protocol (TCP-IP), short message service (SMS), multimedia messaging service (MMS), email, instant messaging service (IMS), Bluetooth, IEEE 802.11 and any similar wireless communication technology. The communication device involved in implementing various embodiments of the present invention may communicate using various media, including but not limited to radio, infrared, laser, cable connection and any appropriate connection.

[0062] In telecommunications and data networks, a channel can refer to a physical channel or a logical channel. A physical channel can refer to a physical transmission medium such as a wire, while a logical channel can refer to a logical connection on a multiplexed medium capable of conveying multiple logical channels. A channel can be used to convey information signals (e.g., bit streams) from one or more senders (or transmitters) to one or more receivers.

[0063] An MPEG-2 Transport Stream (TS), specified in ISO / IEC 13818-1 or equivalently in ITU-T Recommendation H.222.0, is a format for carrying audio, video and other media, as well as program metadata or other metadata in a multiplexed stream. A packet identifier (PID) is used to identify elementary streams within a TS (aka packetized elementary streams). Thus, a logical channel within an MPEG-2 TS can be considered to correspond to a specific PID value.

[0064] Available media file format standards include ISO Base Media File Format (ISO / IEC 14496-12, which may be abbreviated as ISOBMFF) and File Format for NAL Unit Structured Video (ISO / IEC 14496-15), which is derived from ISOBMFF.

[0065] A video codec consists of an encoder that transforms the input video into a compressed representation suitable for storage / transmission and a decoder that can decompress the compressed video representation back into a viewable form. The video encoder and / or video decoder can also be separate from each other, i.e., they do not need to form a codec. Typically, an encoder discards some information in the original video sequence in order to represent the video in a more compact form (i.e., at a lower bit rate).

[0066] A typical hybrid video encoder (e.g., many encoder implementations of ITU-T H.263 and H.264) encodes video information in two stages. First, the pixel values ​​in a certain picture area (or "block") are predicted, for example, by motion compensation (finding and indicating an area in one of the previously encoded video frames that closely corresponds to the block being encoded) or by spatial means (using the pixel values ​​around the block to be encoded in a specified manner). Second, the prediction error (i.e., the difference between the predicted pixel block and the original pixel block) is encoded. This is usually done by transforming the difference in pixel values ​​using a specified transform (e.g., a discrete cosine transform (DCT) or a variant thereof), quantizing the coefficients, and entropy encoding the quantized coefficients. By varying the fidelity of the quantization process, the encoder is able to control the balance between the accuracy of the pixel representation (picture quality) and the size of the resulting encoded video representation (file size or transmission bit rate).

[0067] In temporal prediction, the prediction source is a previously decoded picture (aka reference picture). In intra-block copy (IBC; aka intra-block copy prediction), prediction is applied similarly to temporal prediction, but the reference picture is the current picture, and only previously decoded samples can be referenced in the prediction process. Inter-layer or inter-view prediction may be similarly applied to temporal prediction, but the reference picture is a decoded picture from another scalable layer or from another view, respectively. In some cases, inter-frame prediction may refer only to temporal prediction, while in other cases, inter-frame prediction may collectively refer to temporal prediction and any one of intra-block copy, inter-layer prediction, and inter-view prediction, provided that they are performed in the same or similar process as temporal prediction. Inter-frame prediction or temporal prediction may sometimes be referred to as motion compensation or motion compensated prediction.

[0068] Inter prediction (also known as temporal prediction, motion compensation or motion compensated prediction) reduces temporal redundancy. In inter prediction, the prediction source is a previously decoded picture. Intra prediction exploits the fact that adjacent pixels within the same picture are likely to be correlated. Intra prediction can be performed in the spatial or transform domain, i.e., sample values ​​or transform coefficients can be predicted. Intra prediction is typically utilized in intra coding where inter prediction is not applied.

[0069] One result of the encoding process is a set of coded parameters, such as motion vectors and quantized transform coefficients. Many parameters can be more efficiently entropy coded if they are first predicted from spatially or temporally neighboring parameters. For example, a motion vector can be predicted from spatially neighboring motion vectors, and only the difference with respect to the motion vector predictor can be coded. Prediction of coding parameters and intra-frame prediction may be collectively referred to as intra-picture prediction.

[0070] Figure 4 A block diagram of a video encoder suitable for employing embodiments of the present invention is shown. Figure 4 An encoder for two layers is proposed, but it is to be appreciated that the proposed encoder can be similarly extended to encode more than two layers. Figure 4 An embodiment of a video encoder is illustrated, which includes a first encoder section 500 for a base layer and a second encoder section 502 for an enhancement layer. Each of the first encoder section 500 and the second encoder section 502 may include similar elements for encoding an incoming picture. The encoder sections 500, 502 may include a pixel predictor 302, 402, a prediction error encoder 303, 403, and a prediction error decoder 304, 404. Figure 4Also shown are embodiments of pixel predictors 302, 402, which include inter-predictors 306, 406, intra-predictors 308, 408, mode selectors 310, 410, filters 316, 416, and reference frame memories 318, 418. The pixel predictor 302 of the first encoder section 500 receives 300 base layer images of a video stream for encoding at the inter-predictor 306 (determines the difference between the image and the motion compensated reference frame 318) and the intra-predictor 308 (determines the prediction of the image block based only on the processed portion of the current frame or picture). The outputs of both the inter-predictor and the intra-predictor are passed to the mode selector 310. The intra-predictor 308 can have more than one intra-prediction mode. Therefore, each mode can perform intra-prediction and provide the predicted signal to the mode selector 310. The mode selector 310 also receives a copy of the base layer picture 300. Correspondingly, the pixel predictor 402 of the second encoder section 502 receives 400 enhancement layer images of the video stream for encoding at both the inter-frame predictor 406 (determining the difference between the image and the motion compensated reference frame 418) and the intra-frame predictor 408 (determining the prediction of the image block based only on the processed portion of the current frame or picture). The outputs of both the inter-frame predictor and the intra-frame predictor are passed to the mode selector 410. The intra-frame predictor 408 can have more than one intra-frame prediction mode. Therefore, each mode can perform intra-frame prediction and provide the predicted signal to the mode selector 410. The mode selector 410 also receives a copy of the enhancement layer picture 400.

[0071] Depending on which coding mode is selected to encode the current block, the output of the inter-frame predictor 306, 406 or the output of one of the selectable intra-frame predictor modes or the output of the surface encoder within the mode selector is passed to the output of the mode selector 310, 410. The output of the mode selector is passed to the first summing device 321, 421. The first summing device can subtract the output of the pixel predictor 302, 402 from the base layer picture 300 / enhancement layer picture 400 to generate a first prediction error signal 320, 420 that is input to the prediction error encoder 303, 403.

[0072] The pixel predictor 302, 402 also receives a combination of a predicted representation of the image block 312, 412 and an output 338, 438 of the prediction error decoder 304, 404 from the preliminary reconstructor 339, 439. The preliminary reconstructed image 314, 414 may be passed to the intra predictor 308, 408 and the filter 316, 416. The filter 316, 416 receiving the preliminary representation may filter the preliminary representation and output a final reconstructed image 340, 440 which may be stored in a reference frame memory 318, 418. The reference frame memory 318 may be connected to the inter predictor 306 to be used as a reference image for comparison with future base layer pictures 300 in inter prediction operations. According to some embodiments, in the case where the base layer is selected and indicated as a source for inter-layer sample prediction and / or inter-layer motion information prediction of the enhancement layer, the reference frame memory 318 may also be connected to the inter-frame predictor 406 to be used as a reference image for comparison with future enhancement layer pictures 400 in inter-frame prediction operations. Moreover, the reference frame memory 418 may be connected to the inter-frame predictor 406 to be used as a reference image for comparison with future enhancement layer pictures 400 in inter-frame prediction operations.

[0073] According to some embodiments, in case a base layer is selected and indicated as the source of filter parameters for predicting an enhancement layer, filter parameters of the filter 316 from the first encoder section 500 may be provided to the second encoder section 502 .

[0074] The prediction error encoder 303, 403 comprises a transform unit 342, 442 and a quantizer 344, 444. The transform unit 342, 442 transforms the first prediction error signal 320, 420 into a transform domain. The transform is, for example, a DCT transform. The quantizer 344, 444 quantizes the transform domain signal (eg, DCT coefficients) to form quantized coefficients.

[0075] The prediction error decoder 304, 404 receives the output from the prediction error encoder 303, 403 and performs the reverse process of the prediction error encoder 303, 403 to produce a decoded prediction error signal 338, 438, which when combined with the prediction representation of the image block 312, 412 at the second summing device 339, 439 produces a preliminary reconstructed image 314, 414. The prediction error decoder can be considered to include a dequantizer 361, 461 that dequantizes the quantized coefficient values ​​(e.g., DCT coefficients) to reconstruct the transformed signal and an inverse transform unit 363, 463 that performs an inverse transform on the reconstructed transformed signal, wherein the output of the inverse transform unit 363, 463 contains (multiple) reconstructed blocks. The prediction error decoder may also include a block filter that may filter the (multiple) reconstructed blocks based on further decoded information and filter parameters.

[0076] The entropy encoder 330, 430 receives the output of the prediction error encoder 303, 403 and may perform appropriate entropy encoding / variable length encoding on the signal to provide error detection and correction capabilities. The output of the entropy encoder 330, 430 may be inserted into the bitstream, for example, via a multiplexer 508.

[0077] Entropy coding / decoding can be performed in many ways. For example, context-based coding / decoding can be applied, in which both the encoder and the decoder modify the context state of the coding parameters based on the coding parameters of the previous coding / decoding. Context-based coding can be, for example, context-adaptive binary arithmetic coding (CABAC) or context-based variable length coding (CAVLC) or any similar entropy coding. Entropy coding / decoding can be performed alternatively or additionally using a variable length coding scheme, such as Huffman coding / decoding or Exp-Golomb coding / decoding, etc. The decoding of the coding parameters of the bit stream or codeword from the entropy coding can be referred to as parsing.

[0078] The H.264 / AVC standard was developed by the Joint Video Team (JVT) of the Video Coding Experts Group (VCEG) of the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) and the Moving Picture Experts Group (MPEG) of the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC). The H.264 / AVC standard was published by the two parent standardization organizations and is known as ITU-T Recommendation H.264 and ISO / IEC International Standard 14496-10, also known as MPEG-4 Part 10 Advanced Video Coding (AVC). There are multiple versions of the H.264 / AVC standard, with new extensions or features integrated into the specification. These extensions include Scalable Video Coding (SVC) and Multi-view Video Coding (MVC).

[0079] Version 1 of the High Efficiency Video Coding (H.265 / HEVC, aka HEVC) standard was developed by the Joint Collaboration Team - Video Coding (JCT-VC) of VCEG and MPEG. The standard was published by the two parent standardization organizations, and it is known as ITU-T Recommendation H.265 and ISO / IEC International Standard 23008-2, also known as MPEG-H Part 2 High Efficiency Video Coding (HEVC). Later versions of H.265 / HEVC include Scalable, Multi-view, Fidelity Range, 3D, and Screen Content Coding extensions, which can be abbreviated as SHVC, MV-HEVC, REXT, 3D-HEVC, and SCC, respectively.

[0080] SHVC, MV-HEVC and 3D-HEVC use a common base specification specified in Annex F of Version 2 of the HEVC standard. This common base includes, for example, high-level syntax and semantics (such as inter-layer dependencies) that specify some features of each layer of the bitstream, as well as decoding processes, such as reference picture list construction, including inter-layer reference pictures and picture order count derivation for multi-layer bitstreams. Annex F may also be used in potential subsequent multi-layer extensions of HEVC. It is to be understood that even though video encoders, video decoders, encoding methods, decoding methods, bitstream structures and / or embodiments may be described below with reference to specific extensions such as SHVC and / or MV-HEVC, they are generally applicable to any multi-layer extension of HEVC, and even more generally to any multi-layer video coding scheme.

[0081] Versatile Video Coding (VVC) (MPEG-I Part 3) (also known as ITU H.266) is a video compression standard developed by the MPEG Alliance and the ITU's Joint Video Exploration Team (JVET) as the successor to HEVC / H.265.

[0082] Some key definitions, bitstream and coding structures and concepts of H.264 / AVC, HEVC and VVC and some of their extensions are described in this section as examples of video encoders, decoders, encoding methods, decoding methods and bitstream structures, in which embodiments can be implemented. Various aspects of the present invention are not limited to H.264 / AVC or HEVC or VCC or their extensions, but rather the description is given for a possible basis, on which the present embodiment can be partially or completely implemented. Whenever VVC or any draft version thereof is referenced below, it is necessary to understand that the description matches the VVC draft specification, that later draft versions and (multiple) final versions of VVC may change, and that the description and embodiments can be adjusted to match (multiple) final versions of VVC.

[0083] Video coding standards may specify bitstream syntax and semantics and the decoding process for error-free bitstreams, while the encoding process may not be specified, but the encoder may only need to generate a compliant bitstream. Bitstream and decoder compliance can be verified using a hypothetical reference decoder (HRD). These standards may contain coding tools that help deal with transmission errors and losses, but using this tool in encoding may be optional, and the decoding process for erroneous bitstreams may not be specified.

[0084] A syntax element may be defined as a data element represented in a bitstream. A syntax structure may be defined as zero or more syntax elements that appear together in a bitstream in a specified order.

[0085] Each syntax element can be described by its name and a descriptor for its coded representation. The convention that syntax element names consist of all lowercase letters with underscore characters can be used. The decoding process of a video decoder can be represented based on the value of the syntax element and the values ​​of previously decoded syntax elements.

[0086] When describing H.264 / AVC, HEVC, VCC, and example embodiments, the following descriptors and / or descriptions may be used to specify the parsing process of each syntax element.

[0087] -u(n): Use an unsigned integer of n bits. When n is "v" in the syntax table, the number of bits varies in a way that depends on the values ​​of other syntax elements. The pairing process for this descriptor is specified by the n next bits from the bitstream, interpreted as the binary representation of the unsigned integer, with the most significant bit written first.

[0088] -ue(v): unsigned integer exponential Golomb coding (also known as exp Golomb coding) syntax element, where the left bit comes first.

[0089] The Exponential Golomb bit string can be converted to a code number (codeNum), for example using the following table:

[0090] Bit string codeNum 1 0 0 1 0 1 0 1 1 2 0 0 1 0 0 3 0 0 1 0 1 4 0 0 1 1 0 5 0 0 1 1 1 6 0 0 0 1 0 0 0 7 0 0 0 1 0 0 1 8 0 0 0 1 0 1 0 9 … …

[0091] In some cases, the syntax table may use the values ​​of other variables derived from the syntax element values. A variable naming convention may be used where lowercase and uppercase letters are mixed and there are no underscore characters. Variables starting with an uppercase letter may be derived for use in decoding the current syntax structure and all related syntax structures. Variables starting with an uppercase letter may be used in later syntax structures during the decoding process without referring to the original syntax structure of the variable. A convention may be used where variables starting with a lowercase letter may be used only within the context in which they are derived.

[0092] In some cases, "mnemonic" names for syntax element values ​​or variable values ​​and their numerical values ​​may be used interchangeably. Sometimes "mnemonic" names are used without any associated numerical value.

[0093] A flag may be defined as a variable or a single bit syntax element that can take one of two possible values: 0 and 1.

[0094] Arrays can be syntactic elements or variables. Square brackets can be used to index arrays. One-dimensional arrays can be called lists. Two-dimensional arrays can be called matrices.

[0095] A function can be described by its name. A convention may be used whereby a function name starts with an uppercase letter, contains a mixture of lowercase and uppercase letters without any underscore characters, and ends with an opening and closing parenthesis, and consists of zero or more variable names (for definitions) or values ​​(for uses) separated by commas (if there is more than one variable).

[0096] The function Ceil(x) can be defined to return the smallest integer greater than or equal to x. The function Log2(x) can be defined to return the base 2 logarithm of x.

[0097] Procedures for describing the decoding of syntactic elements may be specified. Procedures may have separate specifications and invocations. It may be specified that all syntactic elements and uppercase variables belonging to the current syntactic structure and related syntactic structures are available in procedure specifications and invocations, and it may also be specified that procedure specifications may also have lowercase variables explicitly specified as inputs. Each procedure specification may explicitly specify one or more outputs, each of which may be a variable, which may be either an uppercase variable or a lowercase variable.

[0098] The syntax, semantics, and processes may be described using arithmetic, logical, relational, bitwise, and assignment operators similar to those used in the C programming language. Specifically, the operator / is used to indicate integer division (with truncation), and the operator % is used to indicate modulus (i.e., the remainder of a division).

[0099] Numbering and counting conventions can start from 0, for example, "first" is equivalent to number 0, "second" is equivalent to number 1, etc.

[0100] The basic unit for encoder input and decoder output, respectively, is usually a picture. A picture given to an encoder as input may also be referred to as a source picture, and a picture decoded by decoding may be referred to as a decoded picture or a reconstructed picture.

[0101] The source picture and the decoded picture each consist of one or more sample arrays, such as one of the following sample array sets:

[0102] - Luminance (Y) only (monochrome).

[0103] - Luma and two chrominances (YCbCr or YCgCo).

[0104] - Green, Blue and Red (GBR, also known as RGB).

[0105] - An array representing otherwise unspecified monochromatic or tristimulus color samples (e.g., YZX, also called XYZ).

[0106] In the following, these arrays may be referred to as luma (or L or Y) and chroma, where the two chroma arrays may be referred to as Cb and Cr; regardless of which actual color representation method is used. The actual color representation method used can be indicated, for example, in the coded bitstream using the Video Usability Information (VUI) syntax of HEVC or the like. A component may be defined as an array or a single sample from one of the three sample arrays (luma and two chroma) or an array or a single sample of an array constituting a monochrome format picture.

[0107] A picture can be defined as a frame or a field. A frame consists of a matrix of luma samples and possibly corresponding chroma samples. A field is a set of alternating sample rows of a frame and can be used as encoder input when the source signal is interlaced. The chroma sample array may not exist (and therefore monochrome sampling may be used) or may be subsampled when compared to the luma sample array.

[0108] Some chroma formats can be summarized as follows:

[0109] - In monochrome sampling, there is only one sample array, which nominally can be thought of as the brightness array.

[0110] - In 4:2:0 sampling, each of the two chroma arrays has half the height and half the width of the luma array.

[0111] - In 4:2:2 sampling, each of the two chroma arrays has the same height and half the width of the luma array.

[0112] - In 4:4:4 sampling without using separate color planes, each of the two chroma arrays has the same height and width as the luma array.

[0113] The coding format or standard may allow encoding the sample arrays as separate color planes into the bitstream and decoding the separate encoded color planes from the bitstream separately. When separate color planes are used, each of them is processed separately as a picture (by the encoder and / or decoder) using monochrome sampling.

[0114] Partitioning can be defined as dividing a set into subsets so that each element of the set is exactly in one of the subsets. In video coding, partitioning can be defined as dividing a picture or a sub-region of a picture into subsets so that each element of the picture or a sub-region of a picture is exactly in one of the subsets. For example, in partitioning associated with HEVC encoding and / or decoding and / or with VVC encoding and / or decoding, the following terms may be used. A coding block may be defined as an NxN block of samples for some values ​​of N, so that the division of a coding tree block into coding blocks is a partition. A coding tree block (CTB) may be defined as an NxN block of samples for some values ​​of N, so that the division of a component into coding tree blocks is a partition. A coding tree unit (CTU) may be defined as a coding tree block of luminance samples, two corresponding coding tree blocks of chrominance samples of a picture with three sample arrays, or a coding tree block of samples of a monochrome picture or a picture encoded using three separate color planes and a syntax structure (used to encode the samples). A coding unit (CU) may be defined as a coding block of luma samples, two corresponding coding blocks of chroma samples for a picture with three sample arrays, or a coding block of samples for a monochrome picture or a picture encoded using three separate color planes and a syntax structure used to encode the samples. A CU with a maximum allowed size may be named LCU (largest coding unit) or coding tree unit (CTU), and a video picture is divided into non-overlapping LCUs.

[0115] In HEVC, a CU consists of one or more prediction units (PUs) that define the prediction process for samples within the CU and one or more transform units (TUs) that define the prediction error coding process for samples in the CU. Typically, a CU consists of square blocks of samples, whose size can be selected from a predefined set of possible CU sizes. Each PU and TU can also be split into smaller PUs and TUs to increase the granularity of the prediction and prediction error coding processes, respectively. Each PU has prediction information associated with it that defines which prediction will be applied to the pixels within the PU (e.g., motion vector information for PUs for inter-prediction and intra-prediction directionality information for PUs for intra-prediction).

[0116] Each TU can be associated with information describing the prediction error decoding process for the samples within the TU (including, for example, DCT coefficient information). It is usually signaled at the CU level whether prediction error coding is applied to each CU. In the case where there is no prediction error residual associated with a CU, it can be considered that there is no TU for the CU. The division of an image into CUs and the division of a CU into PUs and TUs is usually signaled in the bitstream, allowing a decoder to reproduce the expected structure of these units.

[0117] In the draft version of H.266 / VVC, the following partitions apply. It should be noted that the content described here may still evolve in later draft versions of H.266 / VVC before the standard is finalized. Although the maximum CTU size has been increased to 128x128, similar to HEVC, the picture is partitioned into CTUs. The coding tree unit (CTU) is first partitioned by a quadtree (also known as a quadtree) structure. The leaf nodes of the quadtree can then be further partitioned by a multi-type tree structure. There are four types of splits in the multi-type tree structure, namely, vertical binary splitting, horizontal binary splitting, vertical ternary splitting, and horizontal ternary splitting. The leaf nodes of the multi-type tree are called coding units (CUs). CU, PU, ​​and TU have the same block size unless the CU is too large for the maximum transform length. The segmentation structure of CTU is a quadtree with nested multi-type trees, which uses binary and ternary splitting, that is, no separate CU, PU, ​​and TU concepts are used, except for CUs that need to be too large for the maximum transform length. CU can have a square or rectangular shape.

[0118] The basic unit of encoder output of some coding formats (such as VVC,v) and input of decoders of some coding formats (such as VVC) is the Network Abstraction Layer (NAL) unit. For transmission over packet-oriented networks or storage in structured files, NAL units can be encapsulated into packets or similar structures.

[0119] The byte stream format may be specified for NAL unit streams for use in transport or storage environments that do not provide a framing structure. The byte stream format separates NAL units from each other by appending a start code in front of each NAL unit. To avoid false detection of NAL unit boundaries, the encoder runs a byte-oriented start code emulation prevention algorithm that adds emulation prevention bytes to the NAL unit payload if the start code would otherwise appear. To enable direct gateway operation between packet-oriented and stream-oriented systems, start code emulation prevention may always be performed regardless of whether the byte stream format is in use.

[0120] A NAL unit may be defined as a syntactic structure containing an indication of the type of data to follow and the bytes containing that data, in the form of an RBSP interspersed with emulation prevention bytes where necessary. A Raw Byte Sequence Payload (RBSP) may be defined as a syntactic structure containing an integer number of bytes that are encapsulated in a NAL unit. An RBSP is either empty or has the form of a data bit string containing the syntax elements, followed by an RBSP stop bit and then zero or more subsequent bits equal to 0.

[0121] The NAL unit consists of a header and a payload. The NAL unit header indicates the type of the NAL unit, etc.

[0122] NAL units can be classified into Video Coding Layer (VCL) NAL units and non-VCL NAL units. A VCL NAL unit is typically a coded slice NAL unit.

[0123] A non-VCL NAL unit may be, for example, one of the following types: a sequence parameter set, a picture parameter set, a supplemental enhancement information (SEI) NAL unit, an access unit delimiter, an end of a sequence NAL unit, an end of a bitstream NAL unit, or a padding data NAL unit. Parameter sets may be required to reconstruct a decoded picture, while many other non-VCL NAL units are not required to reconstruct decoded sample values.

[0124] Some coding formats specify parameter sets, which may carry parameter values ​​needed to decode or reconstruct a decoded picture. Parameters may be defined as syntax elements of a parameter set. A parameter set may be defined as a syntax structure that contains parameters and may be referenced or activated from another syntax structure, for example using an identifier.

[0125] Some types of parameter sets are briefly described below, but it should be understood that other types of parameter sets may exist, and embodiments may be applied but not limited to the described parameter set types. Parameters that remain unchanged through the encoded video sequence may be included in a sequence parameter set (SPS). In addition to parameters that may be required for the decoding process, the sequence parameter set may optionally include video usability information (VUI), which includes parameters that may be important for buffering, picture output timing, rendering, and resource reservation. The picture parameter set (PPS) contains such parameters that may not change in multiple encoded pictures. The picture parameter set may include parameters that can be referenced by the encoded image segment of one or more encoded pictures. The header parameter set (HPS) has been proposed to include such parameters that can be changed based on the picture.

[0126] A bitstream can be defined as a sequence of bits, which in some coding formats or standards may be in the form of a NAL unit stream or a byte stream, which forms a representation of coded pictures and associated data (forming one or more coded video sequences). A first bitstream may be followed by a second bitstream in the same logical channel, such as in the same file or in the same connection of a communication protocol. An elementary stream (in the context of video coding) may be defined as a sequence of one or more bitstreams. In some coding formats or standards, the end of the first bitstream may be indicated by a specific NAL unit, which may be referred to as the end of the bitstream (EOB) NAL unit and is the last NAL unit of the bitstream.

[0127] A bitstream portion may be defined as a continuous subset of a bitstream. In some contexts, a bitstream portion may be required to consist of one or more complete syntactic structures and have no incomplete syntactic structures. In other contexts, a bitstream portion may include any continuous segment of a bitstream and may contain incomplete (multiple) syntactic structures.

[0128] The phrases along a bitstream (e.g., indicated along a bitstream) or along a coding unit of a bitstream (e.g., indicated along a coding tile) may be used in the claims and described embodiments to refer to transmission, signaling, or storage by way of "out-of-band" data associated with the bitstream or coding unit, respectively, but not included within the bitstream or coding unit. The phrases decoding along a bitstream or along a coding unit of a bitstream, etc. may refer to decoding reference out-of-band data associated with the bitstream or coding unit, respectively (which may be obtained from an out-of-band transmission, signaling, or storage). For example, the phrase along a bitstream may be used when the bitstream is contained in a container file, such as a file conforming to the ISO base media file format, and certain file metadata is stored in the file in a manner that associates the metadata to the bitstream, such as a box in a sample entry of a track containing the bitstream, a sample group of a track containing the bitstream, or a timed metadata track associated with a track containing the bitstream.

[0129] A coded video sequence (CVS) may be defined as a sequence of coded pictures in decoding order that is independently decodable and that is followed by the end of another coded video sequence or bitstream. A coded video sequence may additionally or alternatively be specified to end when a specific NAL unit, which may be referred to as an end-of-sequence (EOS) NAL unit, appears in the bitstream.

[0130] An image can be split into independently encodable and decodable image segments (e.g., slices and / or tiles and / or tile groups). This image segmentation can enable parallel processing, and a "slice" in this description may refer to an image segment consisting of a certain number of basic coding units processed in a default encoding or decoding order, while a "tile" may refer to an image segment that has been defined as a rectangular image area along a tile grid. A tile group may be defined as a group of one or more tiles. Image segments may be encoded as separate units in a bitstream, such as VCL NAL units in H.264 / AVC and HEVC and VVC. A coded image segment may include a header and a payload, wherein the header contains parameter values ​​required to decode the payload. The payload of a slice may be referred to as slice data.

[0131] In HEVC, pictures can be partitioned into rectangular tiles and contain an integer number of LCUs. In HEVC, the partitioning of tiles forms a regular grid, where the height and width of the tiles differ from each other by at most one LCU. In HEVC, a slice is defined as an integer number of coding tree units contained in an independent slice segment and all subsequent subordinate slice segments (if any) before the next independent slice segment (if any) in the same access unit. In HEVC, a slice segment is defined as an integer number of coding tree units that are ordered consecutively in a tile scan and contained in a single NAL unit. The division of each picture into slice segments is partitioning. In HEVC, an independent slice segment is defined as a slice segment whose values ​​of the syntax elements of the slice segment header are not inferred from the values ​​of the previous slice segment, and a subordinate slice segment is defined as a slice segment whose values ​​of some syntax elements of the slice segment header are inferred from the values ​​of the previous independent slice segment in decoding order. In HEVC, a slice header is defined as a slice segment header of an independent slice segment that is the current slice segment or an independent slice segment preceding the current subordinate slice segment, and a slice segment header is defined as a portion of a coded slice segment that contains data elements related to the first or all coding tree units represented in the slice segment. If tiles are not used, CUs are scanned in the raster scan order of LCUs within tiles or within a picture. Within an LCU, CUs have a specific scanning order.

[0132] Therefore, video coding standards and specifications may allow the encoder to divide a coded picture into coded slices, etc. Intra-picture prediction is typically disabled across slice boundaries. Therefore, slices can be viewed as a way to split a coded picture into independently decodable fragments. In H.264 / AVC and HEVC, intra-picture prediction can be disabled across slice boundaries. Therefore, slices can be viewed as a way to split a coded picture into independently decodable fragments, and slices are therefore typically viewed as basic units for transmission. In many cases, the encoder can indicate in the bitstream which types of intra-picture prediction are turned off across slice boundaries, and the decoder operation takes this information into account, for example, when deriving which prediction sources are available. For example, if a neighboring CU resides in a different slice, samples from a neighboring CU may be considered unavailable for intra prediction.

[0133] In the latest version of VVC (ie, VVC Draft 5), the partitioning of a picture into slices, tiles, and bricks is defined as follows.

[0134] The picture is divided into one or more tile rows and one or more tile columns. The partitioning of the picture into tiles forms a tile grid, which may be characterized by a list of tile column widths (in CTUs) and a list of tile row heights (in CTUs).

[0135] A tile is a sequence of Coding Tree Units (CTUs) that covers a "cell" in the tile grid, i.e., a rectangular area of ​​the picture. A tile is divided into one or more bricks, each consisting of a number of CTU rows within the tile. Tiles that are not partitioned into multiple bricks are also called bricks. However, bricks that are a proper subset of a tile are not called tiles.

[0136] A slice contains multiple tiles of a picture or multiple bricks of a tile. A slice is a VCL NAL unit.

[0137] Two slicing modes are supported, namely, raster scan slicing mode and rectangular slicing mode. In raster scan slicing mode, a slice contains a sequence of tiles in a tile raster scan of a picture. In rectangular slicing mode, a slice contains multiple bricks of a picture that together form a rectangular region of the picture. The bricks within a rectangular slice follow the brick raster scan order of the slice.

[0138] Brick scan can be defined as a specific sequential ordering of CTUs that partition a picture, where CTUs are ordered consecutively in a CTU raster scan within a brick, bricks within a tile are ordered consecutively in a raster scan of bricks of tiles, and tiles within a picture are ordered consecutively in a raster scan of tiles of a picture. It may be required, for example, in a coding standard that for the first CTU of each coded slice NAL unit, the coded slice NAL units should be in order of increasing CTU addresses in brick scan order, where CTU addresses can be defined as increasing in a CTU raster scan within a picture. Raster scan can be defined as mapping a rectangular two-dimensional pattern to a one-dimensional pattern such that the first entry in the one-dimensional pattern is from the first row of the two-dimensional pattern scanned from left to right, followed by the second row, third row, etc. of the pattern (downwards) similarly scanned from left to right, respectively.

[0139] Figure 5a An example of raster scan slice partitioning of a picture is shown, where the picture is divided into 12 tiles and 3 raster scan slices. Figure 5b An example of rectangular slice partitioning of a picture is shown (with 18x12 CTUs), where the picture is divided into 24 tiles (6 tile columns and 4 tile rows) and 9 rectangular slices. Figure 5c An example of a picture partitioned into tiles, bricks, and rectangular slices is shown, where the picture is divided into 4 tiles (2 tile columns and 2 tile rows), 11 bricks (the upper left tile contains 1 brick, the upper right tile contains 5 bricks, the lower left tile contains 2 bricks, and the lower right tile contains 3 bricks), and 4 rectangular slices.

[0140] Partitioning of tiles, bricks and rectangular slices is specified in the picture parameter set (PPS). Figure 6The syntax for indicating partitioning of tiles and bricks is shown, which is performed in two stages: the tile grid (ie, tile column widths and tile row heights) is provided as a first stage, and thereafter the indicated tiles are further partitioned into bricks.

[0141] There are two modes of indicating the tile grid: uniform (indicated by the syntax element uniform_tile_spacing_flag with value equal to 1) and explicit. In uniform tile spacing, tiles have equal width (possibly except for the rightmost tile column) and equal height (possibly except for the bottommost tile row). In explicit tile spacing, the width and height (respectively) of tile columns and rows are indicated (in units of CTUs), except for the rightmost column and bottommost row (respectively).

[0142] Similar to how the tile grid is indicated, there are two modes for indicating how the tile is split into bricks, ie, each tile can indicate either uniform or explicit brick spacing. The signaling is similar to that of tile rows.

[0143] When rectangular slices are used, the following is provided for each slice using a syntax structure that includes and follows the syntax element num_slices_in_pic_minus1:

[0144] - Top left tile index (except for the first slice which is inferred to have index 0)

[0145] - The difference brick index of the bottom right brick of the slice relative to the top left brick index.

[0146] Figure 6 The semantics of the syntax elements have been specified in VVC draft 5 as follows:

[0147] single_tile_in_pic_flag equal to 1 specifies that there is only one tile reference PPS per picture. single_tile_in_pic_flag equal to 0 specifies that there is more than one tile reference PPS per picture. Note – If there is no further brick splitting within a tile, the entire tile is called a brick. When a picture contains only a single tile without further brick splitting, it is called a single brick. It is a requirement for bitstream conformance that the value of single_tile_in_pic_flag should be the same for all PPSs activated within a CVS.

[0148] uniform_tile_spacing_flag equal to 1 specifies that tile column boundaries and likewise tile row boundaries are uniformly distributed across the picture, and is signaled using the syntax elements tile_cols_width_minus1 and tile_rows_height_minus1. uniform_tile_spacing_flag equal to 0 specifies that tile column boundaries and likewise tile row boundaries may or may not be uniformly distributed across the picture, and is signaled using the syntax elements num_tile_columns_minus1 and num_tile_rows_minus1 and the list of syntax elements tile_column_width_minus1[i] and tile_row_height_minus1[i]. When not present, the value of uniform_tile_spacing_flag is inferred to be equal to 1.

[0149] tile_cols_width_minus1 plus 1 specifies the width of a tile column in CTBs (excluding the rightmost tile column of the picture) when uniform_tile_spacing_flag is equal to 1. The value of tile_cols_width_minus1 should be in the range of 0 to PicWidthInCtbsY-1, inclusive. When not present, the value of tile_cols_width_minus1 is inferred to be equal to PicWidthInCtbsY–1.

[0150] tile_rows_height_minus1 plus 1 specifies the height of tile rows in CTBs (excluding the bottom tile row of the picture) when uniform_tile_spacing_flag is equal to 1. The value of tile_rows_height_minus1 should be in the range of 0 to PicHeightInCtbsY-1, inclusive. When not present, the value of tile_rows_height_minus1 is inferred to be equal to PicHeightInCtbsY–1.

[0151] num_tile_columns_minus1 plus 1 specifies the number of tile columns to partition the picture by when uniform_tile_spacing_flag is equal to 0. The value of num_tile_columns_minus1 should be in the range of 0 to PicWidthInCtbsY-1, inclusive. If single_tile_in_pic_flag is equal to 1, the value of num_tile_columns_minus1 is inferred to be equal to 0. Otherwise, when uniform_tile_spacing_flag is equal to 1, the value of num_tile_columns_minus1 is inferred to be specified during CTB raster scan, tile scan, and brick scan.

[0152] num_tile_rows_minus1 plus 1 specifies the number of tile rows to partition the picture by when uniform_tile_spacing_flag is equal to 0. The value of num_tile_rows_minus1 should be in the range of 0 to PicHeightInCtbsY-1, inclusive. If single_tile_in_pic_flag is equal to 1, the value of num_tile_rows_minus1 is inferred to be equal to 0. Otherwise, when uniform_tile_spacing_flag is equal to 1, the value of num_tile_rows_minus1 is inferred to be specified during the CTB raster scan, tile scan, and brick scan. The variable NumTilesInPic is set equal to (num_tile_columns_minus1+1)*(num_tile_rows_minus1+1). When single_tile_in_pic_flag is equal to 0, NumTilesInPic should be greater than 1.

[0153] tile_column_width_minus1[i] plus 1 specifies the width of the i-th tile column in CTB units.

[0154] tile_row_height_minus1[i] plus 1 specifies the height of the i-th tile row in CTB units.

[0155] brick_splitting_present_flag equal to 1 specifies that one or more tiles of a picture referencing a PPS may be split into two or more tiles. brick_splitting_present_flag equal to 0 specifies that tiles of a picture not referencing a PPS may be split into two or more tiles.

[0156] brick_split_flag[i] equal to 1 specifies that the i-th tile is split into two or more bricks. brick_split_flag[i] equal to 0 specifies that the i-th tile is not split into two or more bricks. When not present, the value of brick_split_flag[i] is inferred to be equal to 0.

[0157] uniform_brick_spacing_flag[i] equal to 1 specifies that horizontal brick borders are uniformly distributed over the i-th tile, and is signaled using the syntax element brick_height_minus1[i]. uniform_brick_spacing_flag[i] equal to 0 specifies that horizontal brick borders may or may not be uniformly distributed over the i-th tile, and is signaled using the syntax element num_brick_rows_minus1[i] and the list of syntax elements brick_row_height_minus1[i][j]. When not present, the value of uniform_brick_spacing_flag[i] is inferred to be equal to 1.

[0158] brick_height_minus1[i] plus 1 specifies the height of a brick row in CTBs (excluding the bottom brick in the i-th tile) when uniform_brick_spacing_flag[i] is equal to 1. When present, the value of brick_height_minus1 should be in the range of 0 to RowHeight[i]-2, inclusive. When not present, the value of brick_height_minus1[i] is inferred to be equal to RowHeight[i]–1.

[0159] num_brick_rows_minus1[i] plus 1 specifies the number of bricks used to partition the i-th tile when uniform_brick_spacing_flag[i] is equal to 0. When present, the value of num_brick_rows_minus1[i] should be in the range of 1 to RowHeight[i]-1, inclusive. If brick_split_flag[i] is equal to 0, the value of num_brick_rows_minus1[i] is inferred to be equal to 0. Otherwise, when uniform_brick_spacing_flag[i] is equal to 1, the value of num_brick_rows_minus1[i] is inferred to be specified during CTB raster scan, tile scan, or brick scan.

[0160] brick_row_height_minus1[i][j] plus 1 specifies the height of the j-th brick in the i-th tile in CTB units when uniform_tile_spacing_flag is equal to 0.

[0161] The following variables are exported and the values ​​of num_tile_columns_minus1 and num_tile_rows_minus1 are inferred when uniform_tile_spacing_flag is equal to 1, and the value of num_brick_rows_minus1[i] is inferred by calling the CTB raster scan, tile scan, brick scan process when uniform_brick_spacing_flag[i] is equal to 1 for each i ranging from 0 to NumTilesInPic-1 (inclusive):

[0162] - List RowHeight[j] specifies the height of the jth tile row in CTBs for j ranging from 0 to num_tile_rows_minus1 (inclusive),

[0163] - List CtbAddrRsToBs [ctbAddrRs] specifies the conversion from CTB addresses in the CTB raster scan of the picture to CTB addresses in the brick scan for ctbAddrRs ranging from 0 to PicSizeInCtbsY-1 (inclusive),

[0164] - List CtbAddrBsToBs[ctbAddrBs] specifies the conversion from CTB addresses in the brick scan to CTB addresses in the CTB raster scan of the picture for ctbAddrBs ranging from 0 to PicSizeInCtbsY-1 (inclusive),

[0165] - List BrickId[ctbAddrBs] specifies the conversion from CTB addresses in the brick scan to brick IDs for ctbAddrBs ranging from 0 to PicSizeInCtbsY-1 (inclusive),

[0166] - List NumCtusInBrick[brickIdx] specifies the conversion from brick index to number of CTUs in the brick for brickIdx ranging from 0 to NumBricksInPic-1 (inclusive),

[0167] - List FirstCtbAddrBs[brickIdx] specifies the conversion from brick ID to CTB address in the brick scan of the first CTB in a brick for brickIdx ranging from 0 to NumBricksInPic-1 (inclusive).

[0168] single_brick_per_slice_flag equal to 1 specifies that each slice referencing this PPS includes one brick. single_brick_per_slice_flag equal to 0 specifies that a slice referencing this PPS may include more than one brick. When not present, the value of single_brick_per_slice_flag is inferred to be equal to 1.

[0169] rect_slice_flag equal to 0 specifies that the bricks within each slice are in raster scan order and the slice information is not signaled in the PPS. rect_slice_flag equal to 1 specifies that the bricks within each slice cover a rectangular area of ​​the picture and the slice information is signaled in the PPS. When single_brick_per_slice_flag is equal to 1, rect_slice_flag is inferred to be equal to 1.

[0170] num_slices_in_pic_minus1 plus 1 specifies the number of slices in each picture referencing the PPS. The value of num_slices_in_pic_minus1 should be in the range of 0 to NumBricksInPic-1, inclusive. When not present and single_brick_per_slice_flag is equal to 1, the value of num_slices_in_pic_minus1 is inferred to be equal to NumBricksInPic-1.

[0171] top_left_brick_idx[i] specifies the brick index of the brick located at the top left corner of the i-th slice. For any i not equal to j, the value of top_left_brick_idx[i] shall not be equal to the value of top_left_brick_idx[j]. When not present, the value of top_left_brick_idx[i] is inferred to be equal to i. The length of the top_left_brick_idx[i] syntax element is Ceil(Log2(NumBricksInPic) bits.

[0172] bottom_right_brick_idx_delta[i] specifies the difference between the brick index of the brick located at the bottom right corner of the i-th slice and top_left_brick_idx[i]. When single_brick_per_slice_flag is equal to 1, the value of bottom_right_brick_idx_delta[i] is inferred to be equal to 0. The length of the bottom_right_brick_idx_delta[i] syntax element is Ceil(Log2(NumBricksInPic-top_left_brick_idx[i])) bits.

[0173] The requirement for bitstream conformance is that a slice should consist of a contiguous sequence of multiple complete tiles or just complete bricks of a tile. The variables NumBricksInSlice[i] and BricksToSliceMap[j] that specify the number of bricks in the i-th slice and the mapping of bricks to slices are derived as follows:

[0174]

[0175] Therefore, a rather complex syntax structure is designed to signal tile and brick partitioning. It is suboptimal in many aspects, such as the number of syntax elements, lines in the syntax, the number of operation modes (i.e., separate unified and explicit modes for both tiles and bricks, and separate indicated tile and brick boundaries), and bit count when signaling.

[0176] Improved methods for signaling tile and brick partitions are now introduced.

[0177] Figure 7 The method for encoding according to the first aspect shown includes: encoding (700) a bitstream at a time, the bitstream including an indication of tile columns and an indication of brick heights for one or more tile columns, or encoding an indication of tile columns and an indication of brick heights for one or more tile columns at a time in or along the bitstream; after detecting a brick row aligned by a picture, inferring (702) a potential tile row; inferring (704) that a boundary of a potential tile row is a boundary of a tile row or indicating whether a boundary of a potential tile row is a boundary of a tile row; and encoding (706) one or more pictures into the bitstream using the indicated tile columns, the indicated or inferred tile rows, and the indicated brick heights, wherein the one or more pictures are partitioned into a tile grid along the indicated tile columns and the indicated or inferred tile rows, the tiles in the tile grid comprising an integer number of coding tree units and being partitioned into one or more bricks, wherein a brick comprises an integer number of coding tree unit rows within the tile.

[0178] A method for decoding according to a first aspect comprises: decoding from a bitstream or along a bitstream at a time an indication of a tile column and an indication of a brick height for one or more tile columns; upon detecting a brick row aligned by a picture, inferring a potential tile row; inferring whether a boundary of a potential tile row is a boundary of a tile row or decoding whether a boundary of a potential tile row is a boundary of a tile row; and decoding one or more pictures from a bitstream using the indicated tile columns, the indicated or inferred tile rows, and the indicated brick heights, wherein the one or more pictures are partitioned into a tile grid along the indicated tile columns and the indicated or inferred tile rows, the tiles in the tile grid comprising an integer number of coding tree units and being partitioned into one or more bricks, wherein a brick comprises an integer number of coding tree unit rows within a tile.

[0179] Thus, tile column and tile column direction brick heights are indicated by the encoder and / or decoded by the decoder, excluding tile row heights. Potential tile row boundaries are inferred to be those tile row boundaries where the brick boundaries are aligned (horizontally) across the picture. Thus, for a potential tile row boundary, it can be inferred that the potential tile row boundary is a tile row boundary. Inferring that a potential tile row boundary is a tile row boundary can be based on a conclusion made based on other information available in the syntax structure or the absence of certain information in the syntax structure (e.g., the absence of a particular flag).

[0180] Alternatively, it can be indicated by the encoder and / or decoded by the decoder whether a potential tile row boundary is a tile row boundary. The indication may be based on one or more flags present in the syntax structure, for example.

[0181] Therefore, by indicating only the tile column and tile column direction brick heights and inferring the potential tile row, partitioning can be signaled without signaling the tile row height. Thus, coding efficiency is improved and the bit rate required for the signaling is reduced.

[0182] In the following, a number of exemplary embodiments of the first aspect are provided, i.e., syntax and semantics for indicating tile columns and tile column directions and excluding tile row heights. The embodiments are also applicable to encoding of bitstream portions that generate compliance with the syntax and semantics and decoding of bitstream portions that decode according to the syntax and semantics.

[0183] Example 1 of syntax and semantics:

[0184]

[0185]

[0186] uniform_tile_col_spacing_flag equal to 1 specifies that tile column boundaries are uniformly distributed across the picture, and is signaled using the syntax element tile_cols_width_minus1. uniform_tile_spacing_flag equal to 0 specifies that tile column boundaries may or may not be uniformly distributed across the picture, and is signaled using the syntax element num_tile_columns_minus1 and a list of syntax elements tile_column_width_minus1[i]. When not present, the value of uniform_tile_col_spacing_flag is inferred to be equal to 1.

[0187] The semantics of tile_cols_width_minus1, num_tile_columns_minus1, and tile_column_width_minus1[i] are specified identically to the semantics of the syntax elements with the same names in VVC-Draft 5.

[0188] If uniform_tile_col_spacing_flag is equal to 1, then NumTileColsInPic is set equal to PicWidthInCtbsY / (tile_cols_width_minus1+1)+PicWidthInCtbsY%(tile_cols_width_minus1+1). Otherwise, NumTileColsInPic is set equal to num_tile_columns_minus1+1.

[0189] uniform_brick_spacing_flag[i] equal to 1 specifies that the horizontal brick borders are uniformly distributed over the i-th tile column, and is signaled using the syntax element brick_height_minus1[i]. uniform_brick_spacing_flag[i] equal to 0 specifies that the horizontal brick borders may or may not be uniformly distributed over the i-th tile column, and is signaled using the syntax element num_brick_rows_minus1[i] and the list of syntax elements brick_row_height_minus1[i][j]. When not present, the value of uniform_brick_spacing_flag[i] is inferred to be equal to 1.

[0190] brick_height_minus1[i] plus 1 specifies the height of the brick row in CTBs when uniform_brick_spacing_flag[i] is equal to 1 (excluding the bottom brick in the i-th tile column).

[0191] num_brick_rows_minus1[i] plus 1 specifies the number of bricks to partition the i-th tile column when uniform_brick_spacing_flag[i] is equal to 0.

[0192] brick_row_height_minus1[i][j] plus 1 specifies the height of the j-th brick in the i-th tile column in CTBs when uniform_tile_spacing_flag is equal to 0.

[0193] Example 2 of syntax and semantics

[0194] Example 2 is similar to Example 1, but additionally indicates by the encoder and / or decodes by the decoder whether the partitioning of the current tile column into bricks is the same as the partitioning of the previous tile column.

[0195]

[0196]

[0197] The semantics of the syntax elements are the same as in Example 1, but the semantics of copy_previous_col_flag[i] is added as follows:

[0198] copy_previous_col_flag[i] equal to 1 specifies all of the following:

[0199] - uniform_brick_spacing_flag[i] is inferred to be equal to uniform_brick_spacing_flag[i–1].

[0200] When brick_height_minus1[i–1] is present, brick_height_minus1[i] is inferred to be equal to brick_height_minus1[i–1].

[0201] When num_brick_rows_minus1[i–1] is present, num_brick_rows_minus1[i] is inferred to be equal to num_brick_rows_minus1[i–1], and brick_row_height_minus1[i][j] is inferred to be equal to brick_row_height_minus1[i-1][j] for every value of j in the range of 0 to num_brick_rows_minus1[i]–1 (inclusive).

[0202] On the other hand, the suboptimal syntax structure problem of VVC draft 5 can be alleviated by a method where the tile column width, tile row height or brick height can be indicated by the encoder and / or decoded by the decoder in some predefined scanning order until it is indicated or decoded that the remaining tile columns, tile rows or bricks (respectively) have equal sizes.

[0203] The method for encoding according to the second aspect is Figure 8 , wherein the method comprises: step a) indicating (800) the number of partitions to be assigned; b) determining (802) the number of cells to be assigned to the partitions; c) indicating (804) whether the number of cells to be assigned is evenly assigned to the said number of partitions; and if not, d) indicating (806) the number of cells to be assigned to the next partition, and e) repeating (808) steps c) and d) until all cells have been assigned to the partitions.

[0204] The method for decoding according to the second aspect comprises: step a) decoding the number of partitions to be assigned; b) determining the number of units to be assigned to the partitions; c) decoding whether the number of units to be assigned are evenly assigned to the said number of partitions; and if not, d) decoding the number of units to be assigned to the next partition, and e) repeating steps c) and d) until all units have been assigned to the partitions.

[0205] According to an embodiment applicable to encoding and / or decoding, the number of units to be assigned to the partition is one of: the picture width in CTUs (for example, when the partition is a tile column), the picture height in CTUs (for example, when the partition is a tile row, or when the partition is a brick row indicated for one or more complete tile columns at a time), the number of CTU rows in a tile (for example, when the partition is a brick of tiles).

[0206] According to an embodiment applicable to encoding and / or decoding, the partition is one or more of: tile columns, tile rows, brick rows.

[0207] According to an embodiment applicable to encoding and / or decoding, a unit is a rectangular block of samples of a picture. Figure 8 The unit in the second aspect shown may be a coding treeblock.

[0208] Hence, by indicating the number of cells to be assigned to a partition in a predefined scanning order, a significant saving in the number of required syntax elements and the bit rate required for the signaling can be achieved, in particular if the number of cells not yet assigned should be evenly assigned to the remaining partitions.

[0209] Fig. 9 Shows Figure 8 An example of how the method may be implemented according to an embodiment. Thus, the number of partitions to be assigned (such as tile columns and / or tile rows) is first indicated (900) and the number of units NU (such as units of coding tree blocks (CTBs)) to be assigned to the partitions is determined (902). In order to create a loop for checking that all units have been assigned to one partition, it is checked (904) whether the number of partitions to be assigned NP is greater than 1. If not, i.e. NP=1, it is inferred or indicated (910) that all remaining units that have not yet been assigned should be assigned to the remaining partitions.

[0210] However, if NP>1, it is checked whether the number of units to be assigned NU is divisible by the number of partitions NP. If so, it is determined (908) whether the number of units NU should be evenly assigned to the remaining partitions. If so, it is inferred or indicated (910) that all remaining units NU that have not yet been assigned should be evenly assigned to the remaining partition(s). If it is noted that the number of units to be assigned NU is not divisible by the number of partitions NP (906), or it is determined (908) that the number of units NU should not be evenly assigned to the remaining partitions, the number of units assigned to the next partition in a predefined scanning order is indicated (912). The number of units to be assigned NU is reduced (914) by the indicated number of units to be assigned to the next partition, and the number of partitions to be assigned NP is decremented by 1 (916). Then if the number of partitions to be assigned NP is greater than 1, the loop is returned to check (904).

[0211] Fig. 9The method can be similarly implemented for decoding according to the embodiments described below. First, the number of partitions to be assigned (such as tile columns and / or tile rows) is decoded from the bitstream or along the bitstream, and the number NU of units to be assigned to the partitions (such as units of coding tree blocks (CTBs)) is determined. In order to create a loop for checking that all units have been assigned to one partition, it is checked whether the number NP of partitions to be assigned is greater than 1. If not, that is, NP=1, all remaining units that have not yet been assigned are inferred or decoded from the bitstream or along the bitstream should be assigned to the remaining partitions.

[0212] However, if NP>1, it is checked whether the number of units to be assigned NU is divisible by the number of partitions NP. If so, whether the number of units NU decoded from the bitstream or along the bitstream should be evenly assigned to the remaining partitions. If it is noted that the number of units to be assigned NU is not divisible by the number of partitions NP, or the number of units decoded from the bitstream or along the bitstream should not be evenly assigned to the remaining partitions, the number of units assigned to the next partition in a predefined scanning order is decoded from the bitstream or along the bitstream. The number of units to be assigned NU is reduced by the indicated number of units to be assigned to the next partition, and the number of partitions to be assigned NP is decremented by 1. Then if the number of partitions to be assigned NP is greater than 1, the loop is returned to check.

[0213] In the following, exemplary embodiments of the second aspect, i.e., syntax and semantics for unified signaling of explicit / unified tile / brick partitioning, are provided. The embodiments are equally applicable to encoding of bitstream portions that generate conforming syntax and semantics and decoding of bitstream portions that decode according to syntax and semantics.

[0214] In this example, unified signaling is used to specify tile column width and tile row height, while the signaling of bricks is unchanged compared to VVC draft 5.

[0215]

[0216] rem_tile_col_equal_flag[i] equal to 0 specifies that tile columns with indices in the range 0 to i (inclusive) may or may not have equal widths in CTBs. rem_tile_col_equal_flag[i] equal to 1 specifies that tile columns with indices in the range 0 to i (inclusive) have equal widths in CTBs, and tile_column_width_minus1[j] is inferred to be equal to remWidthInCtbsY / (i+1) for each value of j in the range 0 to i (inclusive).

[0217] rem_tile_row_equal_flag[i] equal to 0 specifies that tile rows with indices in the range 0 to i (inclusive) may or may not have equal heights in CTBs. rem_tile_row_equal_flag[i] equal to 1 specifies that tile rows with indices in the range 0 to i (inclusive) have equal heights in CTBs, and tile_row_width_minus1[j] is inferred to be equal to remHeightInCtbsY / (i+1) for each value of j in the range 0 to i (inclusive).

[0218] The semantics of other syntax elements may be specified identically to the semantics of the syntax elements of the same name in VVC-Draft 5.

[0219] In the following, exemplary embodiments of yet another aspect are provided, namely, syntax and semantics for both unified signaling including tile height indicating tile columns and tile column directions and explicit / unified tile / brick partitioning. The embodiments are equally applicable to encoding of bitstream portions that generate compliance with the syntax and semantics and decoding of bitstream portions that decode according to the syntax and semantics.

[0220] The exemplary embodiment for encoding can be summarized as follows, while the exemplary embodiment can be applied to decoding by replacing the term indication with the term decoding.

[0221] The tile columns are indicated as follows:

[0222] -The number of tile columns is indicated (num_tile_columns_minus1).

[0223] - The following is indicated in a loop that traverses the tile columns from right to left until all tile columns have been traversed or until the remaining tile columns are indicated to be of equal width:

[0224] o If the remaining width (in CTBs) is divisible by the number of tile columns that have not yet been assigned, indicates whether the remaining tile columns have equal width (rem_tile_col_equal_flag[i]).

[0225] o If the remaining tile columns are not equal in width, the tile column width is indicated (tile_column_width_minus1[i]).

[0226] The following is indicated in a loop that traverses the tile columns from left to right for a rectangular tile and contains only one loop entry, which specifies the tile row height for the raster scanned tile:

[0227] - Flag if the brick partitioning of a tile column is the same as the brick partitioning of the previous tile column. This flag is not present for the leftmost tile column (copy_previous_col_flag[i]). Note that this flag can be omitted in this exemplary embodiment, or there are other alternatives discussed further below that can be used to implement functionality similar to that implemented by this flag.

[0228] - When the tile brick partitioning of a tile column is different from the previous tile column, the bricks of the tile column are indicated as follows:

[0229] o The number of bricks in a tile column is indicated (num_bricks_minus1[i]).

[0230] ○ The following are indicated in a loop that traverses the tiles from right to left until all tiles of a tile column have been traversed or until the remaining tiles of a tile column are indicated to be of equal height:

[0231] ■ If the remaining height (in CTBs) is divisible by the number of bricks that have not yet been assigned, indicate whether the remaining bricks have equal height (rem_brick_height_equal_flag[i][j]).

[0232] ■ If the heights of the remaining bricks are not equal, the brick height is indicated (brick_height_minus1[i][j]).

[0233] The following syntax may be used in this exemplary embodiment:

[0234]

[0235]

[0236] num_tile_columns_minus1 plus 1 specifies the number of tile columns to partition the picture when uniform_tile_spacing_flag is equal to 0. The value of num_tile_columns_minus1 should be in the range of 0 to PicWidthInCtbsY-1, inclusive. When single_tile_in_pic_flag is equal to 1, the value of num_tile_columns_minus1 is inferred to be equal to 0.

[0237] rem_tile_col_equal_flag[i] equal to 0 specifies that tile columns with indices in the range 0 to i (inclusive) may or may not have equal widths in CTBs. rem_tile_col_equal_flag[i] equal to 1 specifies that tile columns with indices in the range 0 to i (inclusive) are inferred to have equal widths in CTBs. When not present, the value of rem_tile_equal_flag[i] is inferred to be equal to 0.

[0238] tile_column_width_minus1[i] plus 1 specifies the width of the i-th tile column in CTB units.

[0239] copy_previous_col_flag[i] equal to 0 specifies that num_bricks_minus1[i] is present. copy_previous_col_flag[i] equal to 1 specifies all of the following:

[0240] - num_bricks_minus1[i] is inferred to be equal to num_bricks_minus1[i–1],

[0241] - For all such values ​​of j in the range 1 to num_bricks_minus1[i] (inclusive), rem_brick_height_equal_flag[i][j] is inferred to be equal to rem_brick_height_equal_flag[i-1][j], where the value of rem_brick_height_equal_flag[i-1][j] exists or is inferred.

[0242] - For all such values ​​of j in the range 1 to num_bricks_minus1[i] (inclusive), brick_height_minus1[i][j] is inferred to be equal to brick_height_minus1[i-1][j], where that value of brick_height_minus[i-1][j] exists.

[0243] copy_previous_col_flag[i] equal to 0 specifies that num_bricks_minus1[i] is present. copy_previous_col_flag[i] equal to 1 specifies all of the following:

[0244] - num_bricks_minus1[i] is inferred to be equal to num_bricks_minus1[i–1],

[0245] - For all such values ​​of j in the range 1 to num_bricks_minus1[i] (inclusive), rem_brick_height_equal_flag[i][j] is inferred to be equal to rem_brick_height_equal_flag[i-1][j], where the value of rem_brick_height_equal_flag[i-1][j] exists or is inferred.

[0246] - For all such values ​​of j in the range 1 to num_bricks_minus1[i] (inclusive), brick_height_minus1[i][j] is inferred to be equal to brick_height_minus1[i-1][j], where that value of brick_height_minus[i-1][j] exists.

[0247] rem_brick_height_equal_flag[i][j] equal to 0 specifies that bricks with indices in the range 0 to j (inclusive) within the i-th tile column may or may not have equal heights in CTBs. rem_brick_height_equal_flag[i][j] equal to 1 specifies that bricks with indices in the range 0 to j (inclusive) within the i-th tile column are inferred to have equal heights in CTBs. When not present, the value of rem_brick_height_equal_flag[i][j] is inferred to be equal to 0.

[0248] brick_height_minus1[i][j] plus 1 specifies the height of the jth brick in the i-th tile column in CTB units.

[0249] The semantics of other syntax elements may be specified identically to the semantics of syntax elements with the same names in VVC-Draft 5.

[0250] The decoding process can use variables defined as follows or similar to the following definitions:

[0251] i ranges from 0 to num_tile_columns_minus1 (inclusive), and the list colWidth[i] specifying the width of the i-th tile column in CTB units is derived as follows:

[0252]

[0253] i ranges from 0 to num_tile_columns_minus1 (inclusive), and j ranges from 0 to num_bricks_minus1[i] (inclusive), a list colBrickHeight[i][j] specifying the height of the j-th brick row in CTB units within the i-th tile column, j ranges from 0 to NumTileRows-1 (inclusive), a list RowHeight[j] specifying the height of the j-th tile row in CTB units, j ranges from 0 to NumTileRows (inclusive), a list tileRowBd[j] specifying the position of the j-th tile row boundary in CTB units, the value of NumTileRows, and the value of NumTilesInPic are derived as follows:

[0254]

[0255]

[0256]

[0257] When single_tile_in_pic_flag is equal to 0, NumTilesInPic should be greater than 1. i ranges from 0 to num_tile_columns_minus1+1 (inclusive), and the list tileColBd[i] specifying the position of the i-th tile column boundary in CTB units is derived as follows:

[0258] for(tileColBd[0]=0, i=0; i<=num_tile_columns_minusl; i++)

[0259] tileColBd[i+1]=tileColBd[i]+colWidth[i]

[0260] The variable NumBricksInPic specifying the number of bricks in the picture referencing the PPS, and brickIdx ranging from 0 to NumBricksInPic-1 (inclusive), the lists BrickColBd[brickIdx], BrickRowBd[brickIdx], BrickWidth[brickIdx], and BrickHeight[brickIdx] specifying the position of the vertical brick boundary in units of CTBs, the position of the horizontal brick boundary in units of CTBs, the width of the brick column in units of CTBs, and the height of the brick column in units of CTBs are derived, and for each i from 0 to NumTilesInPic-1 (inclusive), when uniform_brick_spacing_flag[i] is equal to 1, the value of num_brick_rows_minus1[i] is inferred as follows:

[0261]

[0262] According to an embodiment, a method for encoding includes: steps a) determining the number of units to be assigned to a partition; b) indicating or inferring the number of explicit-sized partitions to be assigned; c) indicating the size or number of units of the explicit-sized partitions; and d) indicating or inferring the number of uniform-sized partitions to be assigned.

[0263] According to an embodiment, a method for decoding includes: steps a) determining the number of units to be assigned to a partition; b) decoding or inferring the number of explicit-sized partitions to be assigned; c) decoding the size or number of units of the explicit-sized partitions; and d) decoding or inferring the number of uniform-sized partitions to be assigned.

[0264] According to an embodiment applicable to encoding and / or decoding, the number of units to be assigned to the partition is one of: the picture width in CTUs (for example, when the partition is a tile column), the picture height in CTUs (for example, when the partition is a tile row, or when the partition is a brick row indicated for one or more complete tile columns at a time), and the number of CTU rows in a tile (for example, when the partition is a brick of tiles).

[0265] According to an embodiment applicable to encoding and / or decoding, the partition is one or more of: tile columns, tile rows, brick rows.

[0266] According to an embodiment applicable to encoding and / or decoding, step d comprises: determining the number of cells that have not yet been assigned to the partition by decrementing the number of cells in the partition of the explicit size from the number of cells to be assigned to the partition, and the method further comprises:

[0267] - assigning partitions to explicit-sized partitions according to the size or number of cells in the explicit-sized partitions and according to a predefined or indicated / decoded scan order;

[0268] - Assigning partitions to even-sized partitions by dividing cells that have not yet been assigned to a partition by the number of even-sized partitions and according to a predefined or indicated / decoded scanning order.

[0269] In accordance with an embodiment applicable to encoding and / or decoding, the number of explicit-sized partitions is indicated and / or decoded in a higher-level syntax structure (e.g., SPS), while the size of the explicit-sized partitions and / or the number of uniform-sized partitions to be assigned may be indicated and / or decoded in a lower-level syntax structure (e.g., PPS).

[0270] According to an embodiment, the number of explicitly sized partitions is inferred (eg predetermined in a coding standard) to be equal to one.

[0271] According to an embodiment applicable to encoding and / or decoding, step d comprises:

[0272] - determining a set or list of partition sizes that are divisible by the number of unassigned cells;

[0273] - If the number of items in a set or list is equal to 1, then the number of evenly sized partitions is inferred to be equal to 1;

[0274] - Indicating and / or decoding an index (or similar) corresponding to an item in a set or list, where the index indicates the number of uniformly sized partitions to be assigned.

[0275] According to an embodiment applicable to encoding and / or decoding, the set or list of partition sizes that are divisible by the number of units that have not yet been assigned is constrained by excluding partition sizes that are smaller than a threshold, where the threshold may be predefined or indicated / decoded in the coding standard, for example. For example, the minimum tile column width in a CTU may be predefined or indicated / decoded.

[0276] According to an embodiment, the index corresponding to the items in the set or list is encoded using a fixed-length codeword, such as u(v), where the length of the codeword is determined by the number of items in the set or list.

[0277] Indicates whether the border of a tile row is the border of a potential tile row

[0278] In some embodiments, when horizontal tile boundaries are aligned across a picture, tile row boundaries are inferred. This section presents an embodiment for signaling tile row boundaries. This embodiment can be applied with any embodiment where tile boundaries are indicated before tile row boundaries in the syntax.

[0279] This embodiment may include one or more of the following steps (some of which have been described earlier):

[0280] - Potential tile row boundaries are inferred to be those where the brick boundaries align (horizontally) across the picture.

[0281] - Inferred by the encoder in or along the bitstream and / or decoded by the decoder from or along the bitstream whether all aligned tile boundaries form a tile row boundary. For example, a flag in the bitstream syntax may be used.

[0282] - If all aligned tile boundaries do not form a tile row boundary, then for each aligned tile boundary, whether the boundary is a tile row boundary is indicated by the encoder in or along the bitstream and / or decoded by the decoder from or along the bitstream. For example, there may be a flag in the bitstream syntax for each aligned tile boundary (excluding aligned tile boundaries that are picture boundaries).

[0283] For example, the following syntax can be used:

[0284] pic_parameter_set_rbsp(){ Descriptors … if(rect_slice_flag){ explicit_tile_rows_flag u(1) if (explicit_tile_rows_flag) for(i=1;i<NumAlignedBrickRows;i++) tile_row_flag[i] u(1) } …

[0285] In another embodiment, NumAlignedBrickRows may be derived like NumTileRows.

[0286] The semantics of the proposed syntactic elements can be specified as follows:

[0287] explicit_tile_rows_flag equal to 0 specifies that tile row boundaries are inferred whenever horizontal tile boundaries are aligned across pictures. explicit_tile_rows_flag equal to 1 specifies that the tile_row_flag[i] syntax element is present.

[0288] tile_row_flag[i] equal to 0 specifies that the i-th such horizontal boundary that a horizontal tile boundary is aligned across a picture is not a tile row boundary. tile_row_flag[i] equal to 1 specifies that the i-th such horizontal boundary that a horizontal tile boundary is aligned across a picture is a tile row boundary. The 0th such horizontal boundary that a horizontal tile boundary is aligned on a picture is the top boundary of the picture.

[0289] Indicates a tile column that is partitioned identically to a brick

[0290] In some example embodiments, the syntax includes an indication that a tile column is partitioned into bricks in the same manner as the previous tile column in the loop entry order (e.g., scanning the tile column from left to right). It should be understood that the embodiments apply similarly to the absence of an indication or any similar indication. For example, the order in which the tile columns are scanned may be from right to left, and thus it may be indicated that the brick partition of the current tile column is copied from the tile column to the right. In another example, it is indicated or inferred that all tile columns of equal width have the same brick partition. In yet another example, the index of the brick column to which the brick partition is copied is indicated. It should also be understood that the embodiments apply similarly when another way of partitioning a tile column into bricks is derived based on an earlier indication. This section presents some related embodiments.

[0291] In an embodiment, the encoder indicates in or along the bitstream and / or the decoder decodes from or along the bitstream whether all tile columns with the same width (e.g. in CTBs) have the same brick partitioning. For example, a syntax element called same_brick_spacing_in_equally_wide_tile_cols_flag may be used.

[0292] In an embodiment, the encoder indicates in or along the bitstream and / or the decoder decodes from or along the bitstream the number of adjacent tile columns with the same brick partitioning in loop entry order (e.g. scanning tile columns from left to right). For example, a syntax element such as num_tile_cols_with_same_brick_partitioning_minus1[i] may be u(v) encoded, where v is determined by the remaining tile columns for which brick partitioning has not yet been indicated.

[0293] In an embodiment, an encoder may indicate in or along a bitstream and / or a decoder may decode from or along a bitstream whether or not there is(s) syntax element(s) associated with indicating that tile columns are identically partitioned into bricks (e.g. copy_previous_col_flag[i]). In an embodiment, the indication is in a sequence level syntax structure, such as an SPS. In another embodiment, the indication is in a picture level syntax structure, such as a PPS. In an embodiment, it is, for example, predefined in a coding standard, that the absence of(s) syntax element(s) associated with indicating that tile columns are identically partitioned into bricks results in the brick partitioning being indicated and / or decoded for all tile columns one by one. In an embodiment, it is, for example, predefined in a coding standard, that the absence of(s) syntax element(s) associated with indicating that tile columns are identically partitioned into bricks results in the brick partitioning being indicated and / or decoded for one tile column and being inferred to be the same for all tile columns. In an embodiment, a method for handling the absence of (multiple) syntax elements related to indicating that tile columns are identically partitioned into bricks is indicated by an encoder in or along a bitstream, such as in an SPS, and / or decoded by a decoder from or along a bitstream, such as from an SPS. The method may be indicated and / or decoded in a predefined set of procedures, which may include but may not be limited to i) brick partitioning is to be indicated and / or decoded for all tile columns one by one, and ii) brick partitioning is to be indicated and / or decoded for one tile column and is inferred to be the same for all tile columns.

[0294] Thus, the syntax and semantics for signaling tile and brick partitioning according to an embodiment provides significant savings in the number of syntax elements and syntax lines required to perform the signaling. Thus, significant savings are achieved in the number of bits required to indicate tile and brick partitioning.

[0295] These benefits are illustrated by the following examples, where Fig.10 a. Fig.10 b and Fig.10 The three different tile and brick partitions shown in c are used to compare the performance of tile and brick partition according to VVC draft 5 and tile and brick partition according to an embodiment.

[0296] Fig.10 a and Fig.10 b Tile and brick partitioning are proposed to implement the 6K Effective Equirectangular Projection (ERP) and Cubemap Projection (CMP) schemes (respectively) described in the Omnidirectional Media Format (OMAF, ISO / IEC 23090-2) clauses D.6.3 and D.6.4 (respectively). These schemes have been recommended in the VR Industry Forum Guidelines. Fig.10 The solution proposed in c is similar to Fig.10 The solution in b is equivalent, but uses a different image aspect ratio.

[0297] The following features are derived from VVC draft 5 and an embodiment combining both unified signaling of tile height indicating tile columns and tile column direction and explicit / unified tile / brick partitioning:

[0298] - Number of syntax elements used to indicate tile and brick partitions

[0299] - Number of syntax lines used to indicate tile and brick partitions

[0300] - The number of bits required to represent the tile and brick partitioning of the scheme included in the following figure

[0301] - The bit count saving percentage provided by the embodiments when compared to VVC draft 5 for the scheme proposed in the figure below.

[0302] The characteristics derived using a 128×128 luma CTB size are presented in the following table.

[0303]

[0304] Therefore, it can be seen that the number of syntax elements is reduced from 13 to 7, and the number of required syntax lines is reduced from 26 to 20. Compared with VVC Draft 5, the embodiment is Fig.10 a to Fig.10 Each of the tile and brick partitions of c provides a bit count savings of more than 50%. It is also noted that the semantics and derivation process also become shorter with the proposal.

[0305] Indicates an uncoded tile or brick

[0306] In some applications, it may be reasonable to assign content to be encoded and / or decoded to tiles and / or bricks in such a way that only a subset of the tiles and / or bricks are occupied. For example, in a viewport-dependent stream of a 360-degree video, only a subset of independently coded picture areas (such as tiles) may be received. In another example, patch-based encoding of volumetric or point cloud video is applied and the patches only occupy a subset of the tiles and / or bricks of the picture.

[0307] According to an embodiment for encoding, uncoded tiles or bricks are indicated in a syntax structure above the slice data in the bitstream or along the bitstream. Syntax elements of the uncoded tiles or bricks are not encoded as slice data. The uncoded tiles or bricks are reconstructed (e.g., reconstructed as a decoded reference picture) using a predefined or indicated method (such as setting reconstructed sample values ​​to 0 in a sample array).

[0308] According to an embodiment for decoding, an uncoded tile or brick is decoded from a syntax structure above the slice data in or along the bitstream. Syntax elements of the uncoded tile or brick are not decoded from the slice data. The uncoded tile or brick is decoded (e.g., decoded as a decoding reference picture) using a predefined or indicated method (such as setting reconstructed sample values ​​to 0 in a sample array).

[0309] According to an embodiment applicable to encoding and / or decoding, the number of uncoded bricks is indicated and / or decoded for a tile column, for example using a variable length codeword, such as ue(v). The bricks in a tile column are traversed in a predefined scan order (e.g., from bottom to top). For each traversed brick, a flag is indicated and / or decoded to derive whether the brick is uncoded. If the brick is uncoded, the number of bricks left to be assigned as uncoded is decremented by 1. The process is continued until there are no remaining bricks to be assigned as uncoded.

[0310] Fig.11 A block diagram of a video decoder suitable for employing embodiments of the present invention is shown. Fig.11 The structure of a two-layer decoder is depicted, but it is to be understood that the decoding operations can be similarly used in a single-layer decoder.

[0311] The video decoder 550 includes a first decoder section 552 for the base layer and a second decoder section 554 for the prediction layer. Block 556 illustrates a demultiplexer for delivering information about the base layer picture to the first decoder section 552 and delivering information about the prediction layer picture to the second decoder section 554. Reference numeral P'n represents a predicted representation of an image block. Reference numeral D'n represents a reconstructed prediction error signal. Blocks 704, 804 illustrate a preliminary reconstructed image (I'n). Reference numeral R'n represents a final reconstructed image. Blocks 703, 803 illustrate an inverse transform (T -1 Blocks 702 and 802 illustrate inverse quantization (Q -1 ). Boxes 701 and 801 illustrate entropy decoding (E -1 ). Blocks 705, 805 illustrate a reference frame memory (RFM). Blocks 706, 806 illustrate prediction (P) (inter prediction or intra prediction). Blocks 707, 807 illustrate filtering (F). Blocks 708, 808 may be used to combine the decoded prediction error information with the predicted base layer / prediction layer image to obtain a preliminary reconstructed image (I'n). The preliminary reconstructed and filtered base layer image may be output 709 from the first decoder section 552, and the preliminary reconstructed and filtered base layer image may be output 809 from the first decoder section 554.

[0312] In this document, a decoder should be interpreted as covering any operating unit capable of performing decoding operations, such as a player, a receiver, a gateway, a demultiplexer and / or a decoder.

[0313] Fig.12 A flow chart of the operation of a decoder according to an embodiment of the present invention is shown. The decoding operation of the embodiment is similar to the encoding operation in other respects, except that the decoder decodes indications from them. Thus, the decoding method includes: decoding (1200) a portion of a bitstream at a time, the portion of the bitstream including indications of tile columns and indications of brick heights for one or more tile columns; upon detecting a brick row aligned by a picture, inferring (1202) a potential tile row; inferring (1204) whether a boundary of a potential tile row is a boundary of a tile row or decoding an indication of whether a boundary of a potential tile row is a boundary of a tile row; and decoding (1206) one or more pictures from the bitstream using the indicated tile columns, the indicated or inferred tile rows, and the indicated brick heights, wherein the one or more pictures are partitioned into a tile grid along the indicated tile columns and the indicated or inferred tile rows, the tiles in the tile grid comprising an integer number of coding tree units and partitioned into one or more bricks, wherein a brick comprises an integer number of coding tree unit rows within a tile.

[0314] Fig.13 A flow chart showing the operation of a decoder according to another embodiment of the present invention. The decoding method comprises the steps of a) decoding (1300) an indication of the number of partitions to be assigned; b) determining (1302) the number of cells to be assigned to the partitions; c) determining (1304) whether the number of cells to be assigned is evenly assigned to the number of partitions; and if not, d) determining (1306) the number of cells to be assigned to the next partition, and e) repeating (1308) steps c) and d) until all cells have been assigned to the partitions.

[0315] Indicates the partitioning of the image into rectangular slices

[0316] Embodiments of improved methods for signaling encoding and / or decoding rectangular slices are introduced in the following paragraphs. These embodiments can be applied together with or independently of the embodiments for tile and brick partitioning. The embodiments are based on the definitions and characteristics of tiles, bricks and rectangular slices specified in VVC Draft 5. With the embodiments, the bit count required to indicate a rectangular slice (e.g., to indicate or derive the position, width and height of a rectangular slice) is reduced.

[0317] The encoding method according to the first aspect includes: indicating or inferring the position of the upper left brick of the rectangular slice in or along the bitstream; deriving from the position whether the rectangular slice includes one or more bricks of the tile; if the rectangular slice includes one or more bricks of the tile, indicating or inferring the number of bricks in the rectangular slice in or along the bitstream.

[0318] The decoding method according to the first aspect includes: decoding or inferring the position of the upper left brick of the rectangular slice from the bitstream or along the bitstream; deriving from the position whether the rectangular slice includes one or more bricks of the tile; if the rectangular slice includes one or more bricks of the tile, decoding or inferring the number of bricks in the rectangular slice from the bitstream or along the bitstream.

[0319] The position of a tile may be, for example, the tile index in a tile scan of the image.

[0320] In an embodiment applicable to encoding and / or decoding, when the position of the upper left brick of the rectangular slice is not the upper left brick of any tile, it is concluded that the rectangular slice includes one or more bricks of the tile.

[0321] In an embodiment applicable to encoding and / or decoding, when the position of the upper left brick of the rectangular slice is the upper left brick of the tile and the tile includes multiple bricks, it is derived that the rectangular slice may include bricks of the tile or a complete tile. In an embodiment applicable to encoding, if it is derived that the rectangular slice may include bricks of the tile or a complete tile, it is indicated in or along the bitstream whether the rectangular slice includes bricks of the tile or a complete tile. In an embodiment applicable to decoding, if it is derived that the rectangular slice may include bricks of the tile or a complete tile, it is decoded from or along the bitstream whether the rectangular slice includes bricks of the tile or a complete tile.

[0322] In an embodiment applicable to encoding and / or decoding, if it is inferred or indicated (as part of encoding) or decoding that a rectangular slice includes bricks of a tile, a variable (e.g., called numDeltaValues) is set equal to the number of bricks following the position of the upper left brick of the rectangular slice and within the same tile. If numDeltaValues ​​is equal to 1, then it is inferred that the rectangular slice contains exactly one brick. Otherwise, the variable numDeltaValues ​​is used to derive the syntax element length of a first syntax element indicating the lower right brick of the rectangular slice, or a second syntax element indicating the number of bricks in the rectangular slice, or a third syntax element indicating the height of the rectangular slice (in bricks), or any similar syntax element. For example, the first syntax element, or the second syntax element, or the third syntax element, or any similar syntax element can be u(v) encoded and have a length of Ceil(Log2(numDeltaValues)) bits.

[0323] In the example embodiment, the following syntax is used:

[0324] pic_parameter_set_rbsp(){ Descriptors … if(rect_slice_flag&&!single_brick_per_slice_flag){ num_slices_in_pic_minus1 ue(v) for(i=0;i<=num_slices_in_pic_minus1;i++){ if(i>0) top_left_brick_idx[i] u(v) if(numDeltaValues>0) bottom_right_brick_idx_delta[i] u(v) } }

[0325] The semantics of the syntax elements can be specified as described earlier, except that bottom_right_brick_idx_delta[i] specifies the difference between the brick index of the brick located at the bottom right corner of the i-th slice and top_left_brick_idx[i]. When single_brick_per_slice_flag is equal to 1, the value of bottom_right_brick_idx_delta[i] is inferred to be equal to 0. When not present, the value of bottom_right_brick_idx_delta[i] is inferred to be equal to 0. The variable numDeltaValues, which specifies the number of values ​​that bottom_right_brick_idx_delta[i] can have, is derived as follows:

[0326]

[0327] The length of the bottom_right_brick_idx_delta[i] syntax element is Ceil(Log2(numDeltaValues)) bits. The variable NumBricksInTile[brickIdx] specifies the number of bricks in the tile that contains the brick with index brickIdx in the brick scan of the picture. When a brick is identified by index brickIdx in the brick scan of the picture, the variable BrickIdxInTile[brickIdx] specifies the index of the brick within the tile that contains the brick.

[0328] In an embodiment applicable to encoding and / or decoding, the position of the upper left brick of a rectangular slice is inferred. Initially, all brick positions are marked as empty. A loop that assigns bricks to rectangular slices is included in or along the bitstream, or is decoded from or along the bitstream. For each loop entry, the upper left brick of the rectangular slice is inferred to be the next empty brick position in a predefined, indicated or decoded scan order. For example, it may be predefined in a coding standard in which a brick scan order within a picture is used. The lower right brick of the rectangular slice may be inferred, indicated or decoded, for example as described above, and the bricks forming a rectangle enclosed by the upper left brick and the lower right brick are marked as assigned. The same or similar process is repeated for each loop entry.

[0329] In an embodiment applicable to encoding and / or decoding, when it is derived (as part of encoding or decoding) or indicated (as part of encoding) or decoded (as part of decoding) that a rectangular slice includes a complete tile, a syntax element indicating a lower right tile of the rectangular slice is derived from one or more of the following:

[0330] - A set of possible bottom right brick positions is derived. The set may include only those brick positions that are the last brick positions within a brick, that are at or below the tile row containing the top left brick of the rectangular slice, that are at or to the right of the tile column containing the top left brick of the rectangular slice, and that enclose the rectangular set of empty brick positions (not yet assigned to any rectangular slice).

[0331] - The entries in the set of possible bottom right brick positions are indexed or enumerated.

[0332] - The length of the syntax element indicating the u(v) encoding of the bottom right brick of a rectangular slice is derived from the number of entries in the set of possible bottom right brick positions. If the number of entries numEnt is equal to 1, the bottom right brick index does not need to be indicated or decoded. Otherwise, the length of the syntax element is equal to Ceil(Log2(numEnt)) bits.

[0333] - The syntax element indicating the bottom right brick of a rectangular slice is an index into an enumerated set of possible bottom right brick positions.

[0334] The encoding method according to the second aspect comprises

[0335] - indicating or inferring the position of the top left brick of the rectangular slice in or along the bitstream;

[0336] - from this position it is derived whether the rectangular slice includes one or more bricks of the tile;

[0337] - if the rectangular slice comprises one or more tiles of a tile, the width of the rectangular slice is inferred to be equal to one tile column, otherwise if the top left tile is on the rightmost tile column, the width of the rectangular slice is inferred to be equal to one tile column, and otherwise the width of the rectangular slice in tile columns is indicated in or along the bitstream;

[0338] - If the rectangular slice comprises one or more bricks of tiles, the number of bricks in the rectangular slice is indicated or inferred in the bitstream or along the bitstream; otherwise if the top left brick is on the bottommost tile row, the height of the rectangular slice is inferred to be equal to one tile row, and otherwise the height of the rectangular slice in tile rows is indicated in the bitstream or along the bitstream.

[0339] The decoding method according to the second aspect comprises

[0340] - decode or infer the position of the top left brick of the rectangular slice from or along the bitstream;

[0341] - from this position it is derived whether the rectangular slice includes one or more bricks of the tile;

[0342] - if the rectangular slice includes one or more tiles of a tile, the width of the rectangular slice is inferred to be equal to one tile column, otherwise if the top left tile is on the rightmost tile column, the width of the rectangular slice is inferred to be equal to one tile column, and otherwise the width of the rectangular slice in tile columns is decoded from or along the bitstream;

[0343] - If the rectangular slice includes one or more tiles of a tile, the number of tiles in the rectangular slice is decoded or inferred from the bitstream or along the bitstream; otherwise if the top left tile is on the bottommost tile row, the height of the rectangular slice is inferred to be equal to one tile row, and otherwise the height of the rectangular slice in tile rows is decoded from the bitstream or along the bitstream.

[0344] In an embodiment applicable to encoding and / or decoding and applicable to the first aspect and / or the second aspect, when it is derived or indicated or decoded that a rectangular slice contains bricks of a tile and the tile contains two bricks and the current brick (i.e., the upper left brick of the rectangular slice) is the top brick of the tile, or the current brick is the bottom-most brick of the tile, the number of bricks in the rectangular slice is inferred to be equal to 1 (in units of bricks).

[0345] In the example embodiment, the following syntax is used:

[0346]

[0347] The variables and semantics of the syntactic elements can be specified as described earlier, with the addition of the following:

[0348] - tlBrickIdx may be designated as the next empty brick position in a predefined scan order, such as the brick scan in the picture, as described earlier. tlBrickIdx is derived anew for each value of i, ie, for each loop entry.

[0349] -numFreeColumnsOnTheRight[brickIdx] is a variable indicating the number of tile columns on the right side of the brick, where the index is brickIdx.

[0350] -slice_width_minus1[i] plus 1 specifies the width of the i-th rectangular slice in the tile column. When not present, slice_width_minus[i] is inferred to be equal to 0.

[0351] -full_tiles_in_slice_flag[i] equal to 0 specifies that the i-th rectangular slice contains one or more bricks of a single tile. full_tiles_in_slice_flag[i] equal to 1 specifies that the i-th rectangular slice contains one or more full tiles. full_tiles_in_slice_flag[i] is inferred to be equal to 0 when BrickIdxInTile[tlBrickIdx] is greater than 0 (i.e., when the top-left brick of the rectangular slice is not the top-left brick of any tile). full_tiles_in_slice_flag[i] is inferred to be equal to 1 when slice_width_minus1[i] is greater than 0 or when BrickIdxInTile[tlBrickIdx] is equal to 0 and NumBricksInTile[tlBrickIdx] is equal to 1.

[0352] - If full_tiles_in_slice_flag[i] is equal to 0, then numFreeRowsBelow[brickIdx] is a variable indicating the number of bricks in the tile below the brick with index brickIdx in the same tile. Otherwise, numFreeRowsBelow[brickIdx] is a variable indicating the number of tile rows below the tile containing the brick with index brickIdx.

[0353] - If full_tiles_in_slice_flag[i] is equal to 0, then slice_height_minus1[i] plus 1 specifies the height of the i-th rectangular slice in the tile. Otherwise, slice_height_minus1[i] plus 1 specifies the height of the i-th rectangular slice in the tile row. When not present, slice_height_minus[i] is inferred to be equal to 0.

[0354] In an embodiment suitable for encoding and / or decoding, the length of the slice width syntax element (e.g., slice_width_minus1) is derived from the number of possible values ​​based on the top left brick position of the rectangular slice. Using the above variables and syntax elements, the length of slice_width_minus1 is equal to Ceil(Log2(numFreeColumnsOnTheRight[tlBrickIdx]+1)).

[0355] In an embodiment suitable for encoding and / or decoding, the length of the slice height syntax element (e.g., slice_height_minus1) is derived from the number of possible values ​​based on the top left tile position of the rectangular slice and whether the rectangular slice includes a single tile of the tile or a complete tile. Using the above variables and syntax elements, the length of slice_height_minus1 is equal to Ceil(Log2(numDeltaValues)), where numDeltaValues ​​is derived as follows:

[0356] Indicates an unencoded rectangular slice

[0357] In some applications, it may be reasonable to assign content to be encoded and / or decoded to rectangular slices in a way that only a subset of the rectangular slices is occupied. For example, in a viewport-dependent stream of a 360-degree video, only a subset of rectangular slices may be received. In another example, patch-based encoding of a volumetric or point cloud video is applied and the patches only occupy a subset of the rectangular slices of the picture.

[0358] According to an embodiment for encoding, the uncoded rectangular slice is indicated in or along the bitstream (e.g., in the PPS). The uncoded rectangular slice is not encoded into the bitstream as a VCL NAL unit. The uncoded rectangular slice is reconstructed (e.g., reconstructed as a decoding reference picture) using a predefined or indicated method (such as setting the reconstructed sample values ​​to 0 in the sample array).

[0359] According to an embodiment for decoding, the uncoded rectangular slice is decoded from the bitstream or along the bitstream (e.g., from the PPS). The uncoded rectangular slice is not decoded from the bitstream from the VCL NAL unit. Instead, the uncoded rectangular slice is decoded (e.g., decoded as a decoded reference picture) using a predefined or indicated method (such as setting the decoded sample values ​​to 0 in the sample array).

[0360] According to an embodiment applicable to encoding and / or decoding, a flag is indicated and / or decoded for each rectangular slice, the flag indicating whether the rectangular slice is not encoded.

[0361] According to an embodiment, the number of uncoded rectangular slices is indicated and / or decoded using a variable length codeword, such as ue(v). The rectangular slices are traversed in a predefined scan order (in the reverse raster scan order of the upper left CTU of the rectangular slice). For each traversed rectangular slice, a flag is indicated and / or decoded to derive whether the rectangular slice is uncoded. If the rectangular slice is uncoded, the number of rectangular slices left to be assigned as uncoded is decremented by 1. The process is continued until there are no remaining rectangular slices to be assigned as uncoded.

[0362] Fig.14 It is a graphical representation of an example multimedia communication system in which various embodiments can be implemented. Data source 1510 provides source signals in analog, uncompressed digital or compressed digital formats or any combination of these formats. Encoder 1520 may include preprocessing or be connected to preprocessing, such as data format conversion and / or filtering of source signals. Encoder 1520 encodes source signals into coded media bitstreams. It should be noted that the bitstream to be decoded can be directly or indirectly received from a remote device that is actually located in any type of network. Additionally, the bitstream can be received from local hardware or software. Encoder 1520 may be able to encode more than one media type, such as audio and video, or more than one encoder 1520 may be required to encode source signals of different media types. Encoder 1520 can also obtain synthetically generated inputs, such as graphics and text, or it may be able to produce coded bitstreams of synthetic media. In the following, only the processing of a coded media bitstream of a media type is considered to simplify the description. However, it should be noted that, in general, a real-time broadcast service includes multiple streams (usually at least one audio, video and text subtitle stream). It should also be noted that the system may include many encoders, but in the figures, only one encoder 1520 is represented to simplify the description without lack of generality. It should also be understood that although the text and examples contained herein may specifically describe the encoding process, those skilled in the art will understand that the same concepts and principles also apply to the corresponding decoding process, and vice versa.

[0363] The coded media bitstream may be transferred to a storage device 1530. The storage device 1530 may include any type of mass storage to store the coded media bitstream. The format of the coded media bitstream in the storage device 1530 may be a basic independent bitstream format, or one or more coded media bitstreams may be encapsulated into a container file, or the coded media bitstream may be encapsulated into a segmented format suitable for DASH (or a similar streaming system) and stored as a segmented sequence. If one or more media bitstreams are encapsulated in a container file, a file generator (not shown in the figure) may be used to store the one or more media bitstreams in a file and create file format metadata, which may also be stored in the file. The encoder 1520 or the storage device 1530 may include a file generator, or the file generator may be operably attached to the encoder 1520 or the storage device 1530. Some systems operate in "real time", i.e., the storage device is omitted and the coded media bitstream is transferred directly from the encoder 1520 to the transmitter 1540. The coded media bitstream may then be transferred to the transmitter 1540 (also referred to as a server) as needed. The format used in the transmission may be an elementary independent bitstream format, a packetized stream format, a segmented format suitable for DASH (or similar streaming systems), or one or more coded media bitstreams may be encapsulated into a container file. The encoder 1520, storage device 1530, and transmitter 1540 may reside in the same physical device, or they may be included in separate devices. The encoder 1520 and transmitter 1540 may operate with real-time content, in which case the coded media bitstreams are typically not stored permanently, but are buffered for a short period of time in the content encoder 1520 and / or transmitter 1540 to smooth out processing delays, transfer delays, and variations in the coded media bitrate.

[0364] The transmitter 1540 sends the coded media bitstream using a communication protocol stack. The stack may include, but is not limited to, one or more of a real-time transport protocol (RTP), a user datagram protocol (UDP), a hypertext transfer protocol (HTTP), a transmission control protocol (TCP), and an Internet protocol (IP). When the communication protocol stack is packet-oriented, the transmitter 1540 encapsulates the coded media bitstream into packets. For example, when RTP is used, the transmitter 1540 encapsulates the coded media bitstream into RTP packets according to the RTP payload format. Typically, each media type has a dedicated RTP payload format. It should be noted again that the system may include more than one transmitter 1540, but for simplicity, the following description only considers one transmitter 1540.

[0365] If the media content is encapsulated in a container file for storage device 1530 or for inputting data to transmitter 1540, transmitter 1540 may include or be operably attached to a "send file parser" (not shown in the figures). Specifically, if the container file is not transmitted as such, and at least one of the contained coded media bitstreams is encapsulated for transmission via a communication protocol, the send file parser locates the appropriate portion of the coded media bitstream to be communicated via the communication protocol. The send file parser may also help create the correct format for the communication protocol, such as packet headers and payloads. The multimedia container file may contain encapsulation instructions, such as a hint track in ISOBMFF, to encapsulate at least one of the contained media bitstreams on the communication protocol.

[0366] The sender 1540 may or may not be connected to the gateway 1550 via a communication network, which may be, for example, a CDN, the Internet, and / or a combination of one or more access networks. The gateway may also or alternatively be referred to as a middle box. For DASH, the gateway may be an edge server (of the CDN) or a web proxy. It should be noted that the system may generally include any number of gateways, etc., but for simplicity, the following description only considers one gateway 1550. The gateway 1550 may perform different types of functions, such as shifting packet streams from one communication protocol stack to another communication protocol stack, merging and branching of data streams, and manipulating data streams according to downlink and / or receiver capabilities, such as controlling the bit rate of the forwarded stream according to the prevailing downlink network conditions. In various embodiments, the gateway 1550 may be a server entity.

[0367] The system includes one or more receivers 1560, which are generally capable of receiving, demodulating and decapsulating the transmitted signal into a coded media bitstream. The coded media bitstream can be transferred to a recording storage device 1570. The recording storage device 1570 can include any type of mass storage to store the coded media bitstream. The recording storage device 1570 can alternatively or additionally include a computing memory, such as a random access memory. The format of the coded media bitstream in the recording storage device 1570 can be a basic independent bitstream format, or one or more coded media bitstreams can be encapsulated into a container file. If there are multiple coded media bitstreams associated with each other, such as audio streams and video streams, a container file is usually used, and the receiver 1560 includes or is attached to a container file generator, which generates a container file from an input stream. Some systems operate "real time", that is, the recording storage device 1570 is omitted, and the coded media bitstream is directly transferred from the receiver 1560 to the decoder 1580. In some systems, only the most recent portion of the recorded stream (eg, an excerpt of the most recent 10 minutes of the recorded stream) is maintained in the recorded storage 1570 , while any earlier recorded data is discarded from the recorded storage 1570 .

[0368] The coded media bitstream may be transferred from the recording storage device 1570 to the decoder 1580. If there are many coded media bitstreams (such as audio streams and video streams) associated with each other and encapsulated into a container file or a single media bitstream is encapsulated in a container file, for example, for easier access, a file parser (not shown in the figure) is used to decapsulate each coded media bitstream from the container file. The recording storage device 1570 or the decoder 1580 may include a file parser, or the file parser is attached to the recording storage device 1570 or the decoder 1580. It should also be noted that the system may include many decoders, but only one decoder 1580 is discussed here to simplify the description without lack of generality.

[0369] The coded media bitstream may also be processed by a decoder 1580, the output of which is one or more uncompressed media streams. Finally, a renderer 1590 may regenerate the uncompressed media streams, for example using a speaker or display. The receiver 1560, the recording storage device 1570, the decoder 1580, and the renderer 1590 may reside in the same physical device, or they may be included in separate devices.

[0370] In the above, some embodiments have been described with reference to and / or using the terminology of VVC / H.266. It should be understood that the embodiments may be similarly implemented using any video encoder and / or video decoder.

[0371] In the above, some example embodiments have been described with reference to specific syntactic structures and / or syntactic elements. It is to be understood that the embodiments may be similarly implemented using any syntactic structures and / or syntactic elements. For example, when the embodiments have been described with reference to syntactic elements in PPS syntax, it is to be understood that the embodiments may be implemented using the same or similar syntactic elements in another syntactic structure (such as SPS).

[0372] In the above, some embodiments have been described with reference to the term indication. It should be understood that the term indication may be understood as encoding or generating one or more syntax elements in one or more syntax structures in or along a bitstream.

[0373] In the above, some embodiments have been described with reference to the term decoding. It should be understood that the term decoding may be understood as decoding or parsing one or more syntax elements from one or more syntax structures from or along a bitstream.

[0374] In the above, where example embodiments have been described with reference to an encoder, it is to be understood that the resulting bitstream and decoder may have corresponding elements therein. Similarly, where example embodiments have been described with reference to a decoder, it is to be understood that the encoder may have a structure and / or computer program for generating a bitstream to be decoded by a decoder. For example, some embodiments related to generating prediction blocks have been described as part of encoding. By generating prediction blocks as part of decoding, the embodiments can be implemented similarly, with the difference that encoding parameters such as horizontal offset and vertical offset are decoded from the bitstream compared to those determined by the encoder.

[0375] The embodiments of the present invention described above describe the codec with respect to separate encoder and decoder devices to assist in understanding the processes involved. However, it is to be understood that the device, structure and operation can be implemented as a single encoder-decoder device / structure / operation. In addition, the encoder and decoder may share some or all common elements.

[0376] Although the above examples describe embodiments of the present invention operating in a codec within an electronic device, it is to be understood that the present invention defined in the claims can be implemented as a part of any video codec. Thus, for example, embodiments of the present invention can be implemented in a video codec that can implement video encoding over a fixed or wired communication path.

[0377] Hence, the user equipment may comprise a video codec such as described above in embodiments of the invention.It should be appreciated that the term user equipment is intended to cover any suitable type of wireless user equipment, such as a mobile phone, a portable data processing device or a portable web browser.

[0378] Furthermore, elements of a public land mobile network (PLMN) may also include the above-mentioned video codec.

[0379] In general, various embodiments of the present invention may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, although the present invention is not limited thereto. Although various aspects of the present invention may be illustrated and described as block diagrams, flow charts, or using some other graphical representation, it is fully understood that the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.

[0380] Embodiments of the present invention may be implemented by computer software executable by a data processor of a mobile device (such as in a processor entity), or by hardware, or by a combination of software and hardware. Further, in this regard, it should be noted that any box of the logic flow in the accompanying drawings may represent program steps or interconnected logic circuits, boxes and functions or a combination of program steps and logic circuits, boxes and functions. The software may be stored in such physical media as a memory chip, or a memory block implemented in a processor, a magnetic medium such as a hard disk or a floppy disk, and an optical medium such as, for example, a DVD and its data variant CD.

[0381] The memory may be of any type suitable for the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. The data processor may be of any type suitable for the local technical environment and may include, as non-limiting examples, one or more of a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), and a processor based on a multi-core processor architecture.

[0382] Embodiments of the present invention may be practiced in various components such as integrated circuit modules. The design of integrated circuits is generally a highly automated process. Complex and powerful software tools are available to convert a logic level design into a semiconductor circuit design ready to be etched and formed on a semiconductor substrate.

[0383] Programs, such as those offered by Synopsys Technologies Inc. of Mountain View, Calif., and Cadence Design Inc. of San Jose, Calif., use well-established design rules and a library of pre-stored design modules to automatically route conductors and locate components on a semiconductor chip. Once the design of a semiconductor circuit has been completed, the resulting design in a standardized electronic format (e.g., Opus, GDSII, etc.) can be sent to a semiconductor fabrication facility or "fab" for fabrication.

[0384] The foregoing description provides a complete and informative description of exemplary embodiments of the present invention by way of illustrative and non-limiting examples. However, in view of the foregoing description, various modifications and adaptations may become apparent to those skilled in the relevant arts when read in conjunction with the accompanying drawings and the appended claims. Nevertheless, all such modifications and similar modifications of the teachings of the present invention will still fall within the scope of the present invention.

Claims

1. A method for encoding, comprising: Determining a number of units to be assigned to a partition, wherein the number of units to be assigned to the partition is one of: a picture width in coding tree units (CTUs), a picture height in CTUs, a number of CTU rows in a tile; Indicates the number of partitions of explicit size to be assigned; indicating the size or number of cells in the partition of the explicit size; as well as inferring the number of evenly sized partitions to be assigned; Inferring the number of evenly sized partitions to be assigned also includes: determining the number of cells that have not yet been assigned to the partition by decrementing the number of cells in the partition of the explicit size from the number of cells to be assigned to the partition; assigning partitions to the explicitly sized partitions according to a size or number of cells in the explicitly sized partitions and according to a predefined or indicated scan order; as well as Partitions are assigned to the even-sized partitions by dividing the cells that have not been assigned to the partitions by the number of the even-sized partitions and according to a predefined or indicated scanning order.

2. A coding device, comprising: means for determining a number of units to be assigned to a partition, wherein the number of units to be assigned to the partition is one of: a picture width in coding tree units (CTUs), a picture height in CTUs, a number of CTU rows in a tile; means for indicating the number of partitions of explicit size to be assigned; means for indicating the size or number of cells in said explicitly sized partition; as well as means for inferring a number of uniformly sized partitions to be assigned; Wherein said means for inferring the number of evenly sized partitions to be assigned further comprises: means for determining the number of said cells not yet assigned to said partition by decrementing the number of said cells in said partition of said explicit size from the number of said cells to be assigned to said partition; means for assigning partitions to said explicitly sized partitions according to the size or number of cells in said explicitly sized partitions and according to a predefined or indicated scan order; and Means for assigning partitions to said even-sized partitions by dividing said cells not yet assigned to said partitions by the number of said even-sized partitions and according to a predefined or indicated scanning order.

3. The apparatus according to claim 2, further comprising: means for indicating in a higher level syntax structure the number of partitions of said explicit size; as well as Means for indicating the size of the explicit-sized partitions and / or the number of uniform-sized partitions in a lower level syntax structure.

4. A method for decoding, comprising: Determining a number of units to be assigned to a partition, wherein the number of units to be assigned to the partition is one of: a picture width in coding tree units (CTUs), a picture height in CTUs, a number of CTU rows in a tile; Decode the number of explicit-sized partitions to be assigned; decoding a size or number of cells in the explicitly sized partition; as well as inferring the number of evenly sized partitions to be assigned; Inferring the number of evenly sized partitions to be assigned also includes: determining the number of cells that have not yet been assigned to the partition by decrementing the number of cells in the partition of the explicit size from the number of cells to be assigned to the partition; assigning partitions to the explicitly sized partitions according to a size or number of cells in the explicitly sized partitions and according to a predefined or decoded scan order; as well as Partitions are assigned to the even-sized partitions by dividing the cells that have not been assigned to the partitions by the number of the even-sized partitions and according to a predefined or decoded scanning order.

5. A decoding device, comprising: means for determining a number of units to be assigned to a partition, wherein the number of units to be assigned to the partition is one of: a picture width in coding tree units (CTUs), a picture height in CTUs, a number of CTU rows in a tile; means for decoding the number of explicit-sized partitions to be assigned; means for decoding the size or number of cells in said explicitly sized partition; as well as means for inferring a number of uniformly sized partitions to be assigned; Wherein said means for inferring the number of evenly sized partitions to be assigned further comprises: means for determining the number of said cells not yet assigned to said partition by decrementing the number of said cells in said partition of said explicit size from the number of said cells to be assigned to said partition; means for assigning partitions to said explicitly sized partitions according to the size or number of cells in said explicitly sized partitions and according to a predefined or decoded scan order; and Means for assigning partitions to the even-sized partitions by dividing the cells not yet assigned to the partitions by the number of the even-sized partitions and according to a predefined or decoded scan order.

6. The apparatus of claim 5, wherein the partitions are one or more of: tile columns, tile rows, brick rows.

7. The device according to claim 5 or 6, further comprising: means for decoding the number of said explicitly sized partitions from a higher level syntactic structure; as well as Means for decoding the size of said explicit-sized partitions and / or the number of uniform-sized partitions from a lower level syntax structure.

8. The device according to any one of claims 5 to 6, further comprising: Means for inferring that the number of partitions of said explicit size is equal to one.

9. The apparatus of claim 8, wherein the means for inferring the number of uniformly sized partitions to be assigned further comprises: means for determining a set or list of partition sizes that are divisible by the number of said cells that have not yet been assigned; means for inferring that the number of evenly-sized partitions is equal to one if the number of items in the set or the list is equal to one; as well as Means for indicating an index corresponding to an item in said set or list, wherein said index indicates a number of said uniformly sized partitions to be assigned.

10. The apparatus according to claim 9, further comprising: Means for constraining said set or said list of partition sizes divisible by the number of said cells not yet assigned by excluding partition sizes smaller than a threshold, wherein said threshold is predefined or decoded.

11. The device according to claim 9 or 10, further comprising: Means for decoding said index corresponding to an item in said set or list using a fixed length codeword, wherein said length of said codeword is determined by said number of items in said set or list.

Images