Method and apparatus for encoding and decoding image data, computer readable medium, and computer program product

Abstract

A method and apparatus for encoding and decoding image data, a computer readable medium, and a computer program product are provided. The invention also provides a method of processing image data of one or more images, each image consisting of one or more tiles and being divisible into one or more image portions, where the image is divisible into one or more sub-pictures, where the sub-pictures are divided into one or more sub-pictures. And the method comprises: determining one or more image portions included in the sub-picture; and processing the one or more images using information obtained from the determination.

Description

[0001] (This application is a divisional application of application No. 2020800888327, filed December 17, 2020, entitled “Video Coding Supporting Sub-Pictures, Slices, and Blocks”.) Technical Field

[0002] The present invention relates to partitioning of images, and encoding or decoding of images or image sequences comprising images. Embodiments of the invention are particularly, but not exclusively, used in the context of encoding or decoding an image sequence using a first partitioning of the image into one or more sub-pictures and a second partitioning of the image into one or more slices. Background Art

[0003] Video coding involves encoding images (an image is equivalent to a single frame, or picture, of a video). In video coding, before some coding tools, such as motion compensation / prediction (e.g., inter-frame prediction) or intra-frame prediction, can be used on an image, the image is first partitioned (e.g., split) into one or more image parts so that the coding tools can be used on the image parts. The present invention particularly relates to partitioning an image into two types of image parts, namely, sub-pictures and slices, which have been studied by the Video Coding Experts Group / Moving Picture Experts Group (VCEG / MPEG) standardization group and are being considered for use in the Versatile Video Coding (VVC) standard.

[0004] Sub-pictures are a new concept introduced in VVC to enable bitstream extraction and merging operations of independent spatial regions (or image parts) from different bitstreams. "Independent" here means that these regions (or image parts) are encoded / decoded without reference to information obtained from encoding / decoding other regions or image parts. For example, independent spatial regions (i.e., regions or image parts that are encoded / decoded without reference to encoding / decoding of other regions / image parts from the same image) are used for region of interest (ROI) streaming (e.g., during 3D video streaming) or for streaming of omnidirectional video content (e.g., image sequences streamed using the Omnidirectional Media Format (OMAF) standard), in particular when viewport-dependent streaming methods are used for streaming. Individual images from omnidirectional video content are split into independent regions that are encoded with different versions of them (e.g., in terms of image quality or resolution). The client terminal (e.g., a device with a display, such as a mobile phone, etc.) is then able to select the appropriate version of the independent region to obtain a high-quality version of the independent region in the primary viewing direction, while still being able to use lower-quality versions of the remaining regions for the remainder of the omnidirectional video content to improve coding efficiency.

[0005] High Efficiency Video Coding (HEVC or H.265) provides for the signaling of motion restricted block sets (e.g., the bitstream includes data specifying or determining a set of blocks for which motion prediction is restricted so as to make it "independent" of other areas of the picture) to indicate independently coded regions. In HEVC, this signaling is done in the Supplemental Enhancement Information (SEI) message and is only optional. However, in HEVC, the signaling of slices is made independent of the SEI message so that the partitioning of a picture into one or more slices is defined independently of the partitioning of the same picture into one or more block sets. This means that slice partitions do not have the same motion prediction restrictions imposed on them.

[0006] A proposal based on an older draft of the Versatile Video Coding Draft 4 (VVC4) includes signaling block group partitions that rely on the signaling of sub-pictures. A block group is an integer number of complete blocks of a picture that are exclusively contained in a single Network Abstraction Layer (NAL) unit. This proposal (JVET-N0107: AHG 12: Sub-picture-based coding for VVC, Huawei) is a syntax change proposal for introducing the sub-picture concept of VVC. The sub-picture position is signaled using the luma sample position in the sequence parameter set (SPS). Next, a flag in the SPS indicates whether motion prediction is constrained for each sub-picture, but the block group partition (i.e., the picture is partitioned into one or more block groups) is signaled in the picture parameter set (PPS), where each PPS is defined for each sub-picture. Since a PPS is provided for each sub-picture, the block group partition is signaled for each sub-picture in JVET-N0107.

[0007] However, the latest Universal Video Coding Draft 7 (VVC7) no longer has this block group partition concept. VVC7 uses the signal in the SPS to notify the sub-picture layout in units of CTU. The flag in the SPS indicates whether motion prediction is constrained for sub-pictures. The SPS syntax elements for these are as follows:

[0008]

[0009]

[0010] In VVC7, stripe partitions are defined in PPS based on block partitions as follows:

[0011]

[0012]

[0013] This means that in VVC7, slice partitioning is defined independently of sub-picture partitioning. The VVC7 syntax for slice partitioning is based on the block structure without reference to sub-pictures. This is because this independence of sub-pictures avoids any special handling of sub-pictures during the encoding / decoding process, making the handling of sub-pictures simpler. Summary of the Invention

[0014] As mentioned above, VVC7 provides several tools to partition a picture into pixel regions (or component samples). Some examples of these tools are sub-pictures, slices, and blocks. In order to accommodate all of these tools while maintaining their functionality, VVC7 imposes some constraints on the partitioning of pictures into these regions. For example, a block must have a rectangular shape, and the blocks must form a grid. A slice can be an integer number of blocks or a fragment of a block (i.e., a slice includes only a portion of a block or a "partial block" or a "fragment block"). A sub-picture is a rectangular area that must contain one or more slices. However, in VVC7, the signaling of sub-picture partitioning is independent of the signaling of slices and block grids. Therefore, such signaling in VVC7 requires the decoder to check and ensure that the picture partitioning complies with the constraints of VVC7, which can be complex and lead to unnecessary time or resource consumption at the decoder end.

[0015] Embodiments of the present invention aim to address one or more problems or shortcomings of the aforementioned partitioning of images and encoding or decoding of images or image sequences comprising the same. For example, one or more embodiments of the present invention aim to improve and optimize the signaling of picture partitioning (e.g., within the context of VVC7) while ensuring that at least some of the constraints that need to be checked in VVC7 are achieved / satisfied by design during the signaling or encoding process.

[0016] According to various aspects of the present invention, there are provided apparatus / devices, methods, programs, computer-readable storage media, and carrier media / signals as set forth in the accompanying claims. Further features of the present invention will be apparent from the dependent claims and the description. According to other aspects of the present invention, there are provided systems as set forth in the accompanying claims, methods for controlling such systems, apparatus / devices for performing methods, apparatus / devices for processing, media storage devices storing signals as set forth in the accompanying claims, computer-readable storage media or non-transitory computer-readable storage media storing programs as set forth in the accompanying claims, and bitstreams generated using the encoding methods as set forth in the accompanying claims. Further features of the present invention will be apparent from the dependent claims and the subsequent description.

[0017] According to a first aspect of the present invention, a method for processing image data of one or more images is provided, each image consisting of one or more blocks and capable of being divided into one or more image parts, wherein the image is capable of being divided into one or more sub-pictures, and the method comprises: determining one or more image parts included in the sub-picture; and using information obtained from the determination to process the one or more images.

[0018] According to a second aspect of the present invention, a method for partitioning one or more images is provided, the method comprising: partitioning the image into one or more blocks; partitioning the image into one or more sub-pictures; and partitioning the image into one or more image parts by processing image data of the image according to the first aspect.

[0019] According to a third aspect of the invention, there is provided a method of signaling partitioning of one or more images, the method comprising: processing image data of one or more images according to the first aspect; and signaling information for determining the partitioning in a bitstream.

[0020] For the aforementioned aspects of the present invention, the following features may be provided according to embodiments of the present invention. Suitably, the image portion can include a portion of a block. Suitably, the image portion is encoded in a single logical unit (e.g., in a network abstraction layer unit or a NAL unit) or decoded from a single logical unit (e.g., signaled, transmitted, provided in a single logical unit, or obtained from a single logical unit). Suitably, the block and / or sub-picture is not encoded in a single logical unit (e.g., in a NAL unit) or decoded from a single logical unit (e.g., signaled, transmitted, provided in a single logical unit, or obtained from a single logical unit).

[0021] According to a fourth aspect of the present invention, there is provided a method for processing image data of one or more images, each image consisting of one or more blocks and capable of being partitioned into one or more image parts, wherein the image part can include a portion of a block (partial block), and the image can be partitioned into one or more sub-pictures, and the method comprises: determining one or more image parts included in a sub-picture; and processing the one or more images using information obtained from the determination. The portion of the block (partial block) can be an integer number of consecutive complete coding tree unit (CTU) rows within the block.

[0022] For the aforementioned aspects of the present invention, the following features may be provided according to embodiments of the present invention. Suitably, the determining comprises defining the one or more image parts using one or more of the following: an identifier of the sub-picture; a size, width, or height of the sub-picture; whether only a single image part is included in the sub-picture; and the number of image parts included in the sub-picture.

[0023] Suitably, when the number of image parts included in a sub-picture is greater than 1, each image part is determined based on the number of blocks included therein.

[0024] Suitably, when the picture portion comprises one or more parts of a block (partial blocks), the picture portion is determined based on the number of rows or columns of coding tree units (CTUs) to be comprised therein.

[0025] Suitably, the processing comprises providing in a picture parameter set (PPS) or obtaining from the PPS information for determining an image portion based on the number of blocks; and when the image portion comprises one or more parts of blocks (partial blocks), providing in a header of one or more logical units of encoded data comprising the image portion or obtaining from the header information for identifying the image portion comprising the one or more parts of blocks (partial blocks).

[0026] Suitably, the image portion consists of a sequence of tiles in a tile raster scan order.

[0027] Suitably, the processing comprises providing in the bitstream or obtaining from the bitstream information for determining one or more of the following: whether only a single image portion is included in the sub-picture; the number of image portions included in the sub-picture. Suitably, the processing comprises providing in the bitstream or obtaining from the bitstream whether use of an image portion comprising a portion of a tile (partial tile) is permitted when processing the one or more images.

[0028] Suitably, the information provided in or obtained from the bitstream includes: information indicating whether sub-pictures are used in the video sequence; and when the information indicates that sub-pictures are not used in the video sequence, one or more image portions of the video sequence are determined to be not allowed to include a portion of a block (partial block).

[0029] Suitably, the information used for the determination is provided in a Picture Parameter Set (PPS) or obtained from the PPS.

[0030] Suitably, the information used for the determination is provided in a sequence parameter set (SPS) or obtained from the SPS.

[0031] Suitably, when the information used for determination indicates that the number of image parts included in the sub-picture is 1, the sub-picture is composed of a single image part that does not include a part of the tile (partial tile).

[0032] Suitably, the sub-picture comprises two or more image parts, each image part comprising one or more parts of a block (part-blocks).

[0033] Suitably, one or more parts of a block (partial blocks) are from the same single block.

[0034] Suitably, the two or more image parts can comprise one or more parts of a block (partial blocks) from more than one block.

[0035] Suitably, the image portion consists of a plurality of blocks, said image portion forming a rectangular area in the image.

[0036] Suitably, the image parts are stripes (one or more image parts are one or more stripes).

[0037] According to a fifth aspect of the present invention, a method for encoding one or more images is provided, the method comprising any one of the following items: processing image data according to the first aspect or the fourth aspect; partitioning according to the second aspect; and / or signaling according to the third aspect.

[0038] Suitably, the method further comprises: receiving an image; processing image data of the received image according to the first aspect or the fourth aspect; and encoding the received image and generating a bitstream.

[0039] Suitably, the method further comprises providing one or more of the following items in the bitstream: information in a picture parameter set (PPS) for determining an image portion based on the number of blocks, and, when an image portion comprises one or more parts (partial blocks) of a block, information in a header of one or more logical units of coded data comprising the image portion, a slice segment header or a slice header, for identifying the image portion comprising one or more parts (partial blocks) of the block; in the PPS, information for determining whether only a single image portion is included in a sub-picture; in the PPS, information for determining the number of image portions included in a sub-picture; in a sequence parameter set (SPS), information for determining whether use of an image portion comprising a part (partial block) of a block is allowed when processing one or more images; and in the SPS, information indicating whether a sub-picture is used in a video sequence.

[0040] According to a sixth aspect of the present invention, a method for decoding one or more images is provided, the method comprising any one of the following items: processing image data according to the first aspect or the fourth aspect; partitioning according to the second aspect; and / or signaling according to the third aspect.

[0041] Suitably, the method further comprises: receiving a bitstream; decoding information from the received bitstream and processing image data according to any one of the first aspect or the fourth aspect; and obtaining an image using the decoded information and the processed image data.

[0042] Suitably, the method further comprises obtaining one or more of the following items from the bitstream: information from a picture parameter set (PPS) for determining an image portion based on the number of blocks; and, when the image portion comprises one or more parts (partial blocks) of a block, information from a header of one or more logical units of coded data comprising the image portion, a slice segment header or a slice header, for identifying the image portion comprising one or more parts (partial blocks) of a block; information from the PPS for determining whether only a single image portion is included in a sub-picture; information from the PPS for determining the number of image portions included in a sub-picture; information from a sequence parameter set (SPS) for determining whether use of an image portion comprising a part (partial block) of a block is permitted when processing one or more images; and information from the SPS indicating whether a sub-picture is used for a video sequence.

[0043] According to a seventh aspect of the present invention, there is provided an apparatus for processing image data of one or more images, the apparatus being configured to perform the method according to any one of the first, fourth, second and third aspects.

[0044] According to an eighth aspect of the present invention, there is provided an apparatus for encoding one or more images, the apparatus comprising the processing apparatus according to the seventh aspect. Suitably, the apparatus is configured to perform the method according to the fifth aspect.

[0045] According to a ninth aspect of the present invention, there is provided an apparatus for decoding one or more images, the apparatus comprising the processing apparatus according to the seventh aspect. Suitably, the apparatus is configured to perform the method according to the sixth aspect.

[0046] According to a tenth aspect of the present invention, there is provided a program which, when executed on a computer or a processor, causes the computer or the processor to execute the method according to the first, fourth, second or third, fifth or sixth aspect.

[0047] According to an eleventh aspect of the present invention, there is provided a carrier medium or a computer-readable storage medium carrying / storing the program of the tenth aspect.

[0048] According to a twelfth aspect of the present invention, there is provided a signal carrying an information data set for an image, the image being encoded using a method according to the fifth aspect and represented by a bit stream, the image consisting of one or more blocks and being capable of being divided into one or more image parts, wherein the image part is capable of including a portion of a block (partial block), and the image is capable of being divided into one or more sub-pictures, wherein the information data set includes data for determining one or more image parts included in the sub-picture.

[0049] Further aspects of the present invention relate to programs that, when executed by a computer or processor, cause the computer or processor to perform any of the methods of the aforementioned aspects. The program may be provided separately, or may be on, carried by, or carried in a carrier medium. The carrier medium may be non-transitory, such as a storage medium, in particular a computer-readable storage medium. The carrier medium may also be transient, such as a signal or other transmission medium. The signal may be transmitted via any suitable network, including the Internet.

[0050] Further aspects of the present invention relate to a camera comprising an apparatus according to any one of the aforementioned apparatus aspects. According to yet another aspect of the present invention, there is provided a mobile device comprising an apparatus according to any one of the aforementioned apparatus aspects and / or a camera embodying the aforementioned camera aspects.

[0051] Any feature in one aspect of the present invention may be applied to other aspects of the present invention in any appropriate combination. In particular, method aspects may be applied to device aspects, and vice versa. In addition, features implemented in hardware may be implemented in software, and vice versa. Any reference to software and hardware features herein should be interpreted accordingly. Any device features as described herein may also be provided as method features, and vice versa. As used herein, component plus function features may be expressed alternatively according to their corresponding structures (such as appropriately programmed processors and associated memories). It should also be understood that specific combinations of the various features described and defined in any aspect of the present invention may be independently implemented and / or provided and / or used.

[0052] Further features, aspects, and advantages of the present invention will become apparent from the following description of the embodiments with reference to the accompanying drawings. The various embodiments of the present invention described below may be implemented individually or in combination with a plurality of embodiments. In addition, features from different embodiments may be combined where necessary or where a combination of elements or features from various embodiments is beneficial in a single embodiment. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Embodiments of the present invention will now be described, by way of example only, with reference to the following drawings, in which:

[0054] Figure 1 Shows partitioning of a picture into blocks and slices according to an embodiment of the present invention;

[0055] Figure 2 shows sub-picture partitioning of an image according to an embodiment of the present invention;

[0056] Figure 3 shows a bit stream according to an embodiment of the present invention;

[0057] Figure 4 is a flowchart illustrating an encoding process according to an embodiment of the present invention;

[0058] Figure 5 is a flowchart illustrating a decoding process according to an embodiment of the present invention;

[0059] Figure 6 is a flow chart illustrating determination steps used when signaling stripe partitioning according to an embodiment of the present invention;

[0060] Figure 7 shows an example of sub-picture and slice partitioning according to an embodiment of the present invention;

[0061] Figure 8 shows an example of sub-picture and slice partitioning according to an embodiment of the present invention;

[0062] Figure 9a is a flow chart illustrating the steps of an encoding method according to an embodiment of the present invention;

[0063] Figure 9b is a flowchart illustrating the steps of a decoding method according to an embodiment of the present invention;

[0064] Figure 10 is a block diagram illustrating steps of an encoding method according to an embodiment of the present invention;

[0065] Figure 11 is a block diagram illustrating steps of a decoding method according to an embodiment of the present invention;

[0066] Figure 12 is a block diagram schematically illustrating a data communication system in which one or more embodiments of the present invention may be implemented;

[0067] Figure 13 is a block diagram illustrating components of a processing device that may implement one or more embodiments of the present invention;

[0068] Figure 14is a diagram illustrating a network camera system in which one or more embodiments of the present invention may be implemented; and

[0069] Figure 15 is a diagram illustrating a smartphone in which one or more embodiments of the present invention may be implemented. DETAILED DESCRIPTION

[0070] The embodiments of the invention described below are directed to improving the encoding and decoding of images (or pictures).

[0071] In this specification, "signaling" may refer to inserting (providing / including / encoding) information about one or more parameters or syntax elements into a bitstream or extracting / obtaining (decoding) the information from a bitstream, such as any one or more information for determining an identifier of a sub-picture, the size / width / height of a sub-picture, whether only a single image portion (e.g., a slice) is included in a sub-picture, whether a slice is a rectangular slice, and / or the number of slices included in a sub-picture. In this specification, "processing" may refer to any type of operation performed on data, such as encoding or decoding image data of one or more images / pictures.

[0072] In this specification, the term "slice" is used as an example of a picture portion (another example of such a picture portion would be a picture portion comprising one or more coding tree units). It should be understood that embodiments of the present invention may also be implemented based on picture portions instead of slices, and with appropriately modified parameters / values / syntax, such as picture portion headers instead of slice headers or slice segment headers. It should also be understood that various information described herein as being signaled in a slice header, slice segment header, sequence parameter set (SPS), or picture parameter set (PPS) may be signaled elsewhere as long as it can provide the same functionality as provided by signaling the information in these mediums. It should also be understood that any of a slice, block group, block, coding tree unit (CTU) / largest coding unit (LCU), coding tree block (CTB), coding unit (CU), prediction unit (PU), transform unit (TU), or pixel / sample block may be referred to as a picture portion.

[0073] It should also be understood that when a component or tool is described as "active," the component / tool ​​is "enabled," "available for use," or "used," and when described as "inactive," the component / tool ​​is "disabled," "unavailable," or "not used," and that "inferable" means that the relevant value or parameter can be determined / obtained from other information without being explicitly signaled in the bitstream. Furthermore, it should also be understood that when a flag is described as "active," it means that the flag indicates that the relevant component / tool ​​is "active" (i.e., "valid").

[0074] In this specification, unless otherwise specified, the following terms are used with the same or functionally equivalent definitions as those in VVC7. The definitions used in VVC7 are as follows.

[0075] Slice: An integer number of complete blocks of a picture, or an integer number of consecutive complete CTU rows within a block, contained exclusively in a single NAL unit.

[0076] Slice header: A portion of a coded slice that contains data elements related to all blocks or CTU rows within blocks represented in the slice.

[0077] Block: A rectangular area of ​​a CTU within a specific block column and a specific block row in a picture.

[0078] Sub-image: A rectangular area of ​​one or more strips within an image.

[0079] Picture (or image): An array of luma samples in monochrome format or an array of luma samples and two corresponding arrays of chroma samples in 4:2:0, 4:2:2, and 4:4:4 color formats.

[0080] Coded picture: A coded representation of a picture that includes VCL NAL units with a specific value of nuh_layer_id within an AU and includes all CTUs of the picture.

[0081] Encoded representation: A data element represented in an encoded form.

[0082] Raster Scan: The mapping of a rectangular two-dimensional pattern to a one-dimensional pattern such that the first entry in the one-dimensional pattern is from the top row of the two-dimensional pattern scanned from left to right, followed similarly by the second, third, etc. rows of the pattern (downwards), each scanned from left to right.

[0083] Block: An M×N (M columns×N rows) array of samples, or an M×N array of transform coefficients.

[0084] Coding block: A block of M×N samples for some values ​​of M and N such that the partitioning of the CTB into the coding block is a partition.

[0085] Coding Tree Block (CTB): An NxN block of samples for some value of N such that the partitioning of components into CTBs is a partition.

[0086] Coding Tree Unit (CTU): A CTB for luma samples, two corresponding CTBs for chroma samples of a picture with three sample arrays, or a CTB for samples of a monochrome picture or a picture coded using three separate color planes and a syntax structure for coding the samples.

[0087] Coding Unit (CU): A coding block of luma samples, two corresponding coding blocks of chroma samples for a picture with three sample arrays, or a coding block of samples for a monochrome picture or a picture coded using three separate color planes and a syntax structure for coding the samples.

[0088] Component: An array or a single sample from one of the three arrays (luminance and two chrominance) that make up a picture in 4:2:0, 4:2:2, or 4:4:4 color format, or an array or a single sample of an array that makes up a picture in monochrome format.

[0089] Picture Parameter Set (PPS): A syntax structure containing syntax elements that apply to zero or more entire coded pictures as determined by the syntax elements found in each slice header.

[0090] Sequence Parameter Set (SPS): A syntax structure containing syntax elements that apply to zero or more complete CVSs as determined by the contents of the syntax elements found in the PPS referenced by the syntax elements found in each slice header.

[0091] In this specification, unless otherwise specified, the following terms are also used with the same or functionally equivalent definitions as they are defined below.

[0092] Block group: An integer number of complete (ie, whole) blocks of a picture that are exclusively contained in a single NAL unit.

[0093] “Block slice,” “partial block,” “portion of a block,” or “slice of a block”: an integer number of consecutive complete CTU rows within a block of a picture that do not form a complete (ie, entire) block.

[0094] Slice segment: A picture that contains an integer number of complete blocks or an integer number of consecutive complete CTU rows within a block exclusively in a single NAL unit.

[0095] Slice segment header: A portion of an encoded slice segment that contains data elements related to all blocks or CTU rows within blocks represented in the slice segment.

[0096] Slice when slice segments are present: a collection of one or more slice segments that together represent a block or an integer number of complete blocks of a picture

[0097] Embodiments of the Invention

[0098] Partitioning and bitstreaming of pictures / images

[0099] 3.1 Partitioning the image into blocks and strips

[0100] In most coding systems (such as HEVC or the emerging VVC standard), the compression of video relies on block-based video coding. In these coding systems, a video consists of a sequence of frames or pictures or images or samples that can be displayed at different points in time (e.g., at different temporal positions within the video). In the case of multi-layer video (e.g., scalable, stereoscopic or 3D video), it may be necessary to decode several pictures to be able to form the final / resulting image to be displayed at a specific point in time. A picture may also consist of more than one image component (i.e., the image data of a picture includes more than one image component). Examples of such image components would be components for encoding luminance, chrominance or depth information.

[0101] Compression of video sequences uses several different partitioning techniques (ie, different schemes / frameworks / arrangements / mechanisms for partitioning / split pictures) for individual pictures and how these partitioning techniques are implemented during the compression process.

[0102] Figure 1 1 shows the partitioning of a picture into blocks and slices according to an embodiment of the present invention, which is compatible with VVC7. Pictures 101 and 102 are divided into coding tree units (CTUs) represented by dotted lines. CTUs are the basic unit of encoding and decoding in VVC7. For example, in VVC7, a CTU can encode an area of ​​128×128 pixels.

[0103] A coding tree unit (CTU) may also be referred to as a block (of pixels or component samples (values)), a macroblock, or even a coding block. A coding tree unit can be used to simultaneously encode / decode different image components of a picture, or can be limited to only one image component so that different image components of a picture can be encoded / decoded separately / independently. When the data of an image includes separate data for each component, a CTU groups multiple coding tree blocks (CTBs), one CTB for each component.

[0104] like Figure 1 As shown, the picture can also be partitioned according to a block grid (i.e., divided into one or more block grids) represented by thin solid lines. A block is a picture portion (a part / portion of a picture) that is a rectangular area (of pixels / component samples) that can be defined independently of the CTU partitioning. For example, in VVC7, a block can also correspond to a CTU sequence to Figure 1 In the example represented in , the partitioning technique may constrain the boundaries of the blocks to be consistent / aligned with the boundaries of the CTU.

[0105] Blocks are defined such that block boundaries break spatial dependencies of the encoding / decoding process (i.e., in a given picture, a block is defined / specified such that it can be encoded / decoded independently of other spatially "neighboring" blocks in the same picture). This means that encoding / decoding of CTUs within a block is not based on pixels / samples or reference data from other blocks in the same picture.

[0106] Some encoding / decoding systems (e.g., embodiments of the present invention or embodiments for VVC7) provide the concept of stripes (i.e., also use partitioning techniques based on one or more stripes). This mechanism enables a picture to be partitioned into one or several groups of blocks, which are collectively referred to as stripes. Each stripe consists of one or several blocks or partial blocks. As shown in Figures 101 and 102, two different types of stripes are provided. The first type of stripes is limited to strips that form rectangular areas / regions in the picture as indicated by the thick solid lines in Figure 101. Figure 101 shows the partitioning of the picture into six different rectangular strips (0) to (5). The second type of stripes is limited to continuous blocks in raster scan order as indicated by the thick solid lines in Figure 102 (so that they form a sequence of blocks). Figure 102 shows the partitioning of the picture into three different stripes (0) to (2) consisting of continuous blocks in raster scan order. Generally, rectangular strips are a structure / arrangement / configuration for dealing with the selection of regions of interest (ROIs) in videos. A slice can be encoded in a bitstream (or decoded from a bitstream) as one or several Network Abstraction Layer (NAL) units. A NAL unit is a logical unit of data used to encapsulate data in an encoded / decoded bitstream (e.g., a packet containing an integer number of bytes, where multiple packets together form the encoded video data). In the encoding / decoding system of VVC7, a slice is typically encoded as a single NAL unit. When a slice is encoded as several NAL units in a bitstream, each NAL unit of a slice is called a slice segment. A slice segment includes a slice segment header containing coding parameters for the slice segment. According to a variant, the header of the first slice segment NAL unit of a slice contains all coding parameters for the slice. The slice segment headers of subsequent NAL units of a slice may contain fewer parameters than the first NAL unit. In this case, the first slice segment is an independent slice segment, and the subsequent segments are dependent slice segments (because they depend on the coding parameters of the NAL unit from the first slice segment).

[0107] 3.2 Partitioning into Sub-images

[0108] Figure 2The present invention illustrates sub-picture partitioning of a picture, i.e., partitioning a picture into one or more sub-pictures, according to an embodiment of the present invention. A sub-picture represents a portion of a picture (a part or portion of a picture) that covers a rectangular area of ​​the picture. Each sub-picture can have different sizes and coding parameters than another sub-picture. The sub-picture layout, i.e., the geometric shape of the sub-pictures within a picture (e.g., as defined using the position and size / width / height of the sub-pictures), allows grouping of slice sets of the picture and can constrain (i.e., impose restrictions on) temporal motion prediction between two pictures.

[0109] exist Figure 2 , the block partition of picture 201 is a 4×5 block grid. Strip partitioning defines 24 strips, which include one stripe for each block (except for the last column of blocks on the right hand side, where for the last column of blocks, each block is partitioned into two stripes (i.e., a stripe includes partial blocks)). Picture 201 is also partitioned into two sub-pictures 202 and 203. A sub-picture is defined as one or more stripes forming a rectangular area. Sub-picture 202 (shown as a dotted area) includes the stripes in the first three block columns (starting from the left), and sub-picture 203 (shown as a shaded area with a diagonal line across it) includes the remaining stripes (in the last two block columns on the right hand side). As Figure 2 As shown in , VVC7 and embodiments of the present invention provide for allowing single slice partitions and single block partitions to be defined at the picture level (e.g., for each picture using syntax elements provided in the PPS). Sub-picture partitioning is applied on top of block and slice partitions. Another aspect of sub-pictures is that each sub-picture is associated with a flag set. This makes it possible to indicate (using one or more of the flag sets) that the temporal prediction is constrained to use data from a reference frame that is part of the same sub-picture (e.g., the temporal prediction is restricted so that the predictor of a sub-picture cannot use reference data from another sub-picture). For example, a reference Figure 2 , CTB 204 belongs to sub-picture 202. When the temporal prediction is indicated to be constrained for sub-picture 202, the temporal prediction of sub-picture 202 cannot use reference blocks (or reference data) from sub-picture 203. As a result, slices of sub-picture 202 can be encoded / decoded independently of slices of sub-picture 203. This feature / attribute / property / capability is useful in viewport-dependent streaming that involves segmenting an omnidirectional video sequence into spatial portions, where each spatial portion represents a specific viewing direction of 360 (degree) content. A viewer can then select a segment corresponding to a desired / relevant viewing direction, and using this property of the sub-picture, the segment can be encoded / decoded without accessing data from the remaining portion of the 360 ​​content.

[0110] Another use for sub-pictures is generating streams with regions of interest. Sub-pictures provide spatial representations of these regions of interest that can be independently encoded and decoded. Sub-pictures are designed to enable easy access to the coded data corresponding to these regions. Consequently, the coded data corresponding to a sub-picture can be extracted and a new bitstream generated containing data for a single sub-picture or a combination of a sub-picture and one or more other sub-pictures. This means that sub-picture-based bitstream generation can be used to increase flexibility and scalability.

[0111] 3.3 Bitstream

[0112] Figure 3 The organization (i.e., structure, configuration, or arrangement) of a bitstream according to an embodiment of the present invention that complies with the requirements of a coding system of VVC7 is shown. The bitstream 300 consists of data representing / indicating an ordered sequence of syntax elements and coded (image) data. The syntax elements and the coded (image) data are placed (i.e., packaged / packetized) into NAL units 301 to 308. There are different NAL unit types. The network abstraction layer (NAL) provides the ability / performance to encapsulate the bitstream into packets for different protocols, such as Real Time Protocol / Internet Protocol (RTP / IP), ISO base media file format, etc. The network abstraction layer also provides a framework for packet loss resilience.

[0113] NAL units are divided into VCL NAL units and non-VCL NAL units. VCL stands for Video Coding Layer. VCL NAL units contain the actual coded video data. Non-VCL NAL units contain additional information. This additional information can be parameters required to decode the coded video data or supplementary data that can enhance the usability of the decoded video data. Figure 3 The NAL units 306 in VCL_VCL_NAL_UNIT_Slices correspond to slices (ie, they include the actual coded video data for the slices) and constitute the VCL NAL units of the bitstream.

[0114] Different NAL units 301 to 305 correspond to different parameter sets, and these NAL units are non-VCL NAL units. The DPS NAL unit 301 represents a decoding parameter set NAL unit and contains parameters that are constant for a given decoding process. The VPS NAL unit 302 (VPS stands for video parameter set NAL unit) contains parameters defined for the entire video (e.g., the entire video includes one or more sequences of pictures / images) and is therefore applicable when decoding the encoded video data of the entire bitstream. The DPS NAL unit can define parameters that are more static than those in the VPS NAL unit (in the sense that the parameters are stable and do not change as much during the decoding process). In other words, the parameters of the DPS NAL unit change less frequently than the parameters of the VPS NAL unit. The SPS NAL unit 303 (SPS stands for sequence parameter set) contains parameters defined for a video sequence (i.e., a sequence of pictures or images). Specifically, the SPS NAL unit can define the sub-picture layout and associated parameters of the video sequence. The parameters associated with each sub-picture specify the coding constraints that apply to the sub-picture. According to a variant, the parameters include a flag indicating that temporal prediction between sub-pictures is restricted so that data from the same sub-picture can be used during the temporal prediction process.Another flag may enable or disable loop filters (ie post-filtering) across sub-picture boundaries.

[0115] The PPS NAL unit 304 (PPS stands for Picture Parameter Set) contains the parameters defined for a picture or group of pictures. The APSNAL unit 305 (APS stands for Adaptive Parameter Set) contains the parameters of the loop filter, which is typically an adaptive loop filter (ALF) or a shaper model (or luma mapping with chroma scaling model) or a scaling matrix used at the slice level. The bitstream may also contain SEINAL units ( Figure 3 (not shown), which represents the supplemental enhancement information NAL unit. The periodicity (or frequency) of occurrence of these parameter sets (or NAL units) in the bitstream is variable. A VPS defined for the entire bitstream may appear only once in the bitstream. In contrast, an APS defined for a slice may appear once for each slice in each picture. In practice, different slices may rely on (e.g., refer to) the same APS, so there are typically fewer APS NAL units in a picture bitstream than there are slices.

[0116] The AUD NAL unit 307 is an access unit delimiter NAL unit that separates two access units. An access unit is a set of NAL units that may include one or more coded pictures with the same decoding timestamp (i.e., a group of NAL units associated with one or more coded pictures with the same timestamp).

[0117] The PH NAL unit 308 is a picture header NAL unit that groups parameters common to a set of slices of a single coded picture.A picture may refer to one or more APSs to indicate the AFL parameters, shaper models, and scaling matrices used by the slices of the picture.

[0118] Each VCL NAL unit 306 contains video / image data for a slice. A slice may correspond to an entire picture or a sub-picture, a single block or multiple blocks or a fragment (partial block) of a block. For example, Figure 3 A slice comprises a number of blocks 320 . A slice consists of a slice header 310 and a raw byte sequence payload (RBSP) 311 containing coded pixel / component sample data coded as coded blocks 340 .

[0119] The syntax of the PPS in VVC7 includes a syntax element for specifying the size of a picture in units of luma samples, and also includes a syntax element for specifying partitioning of each picture in units of blocks and slices.

[0120] The PPS contains syntax elements that allow the determination (i.e., ability to determine) of the slice positions in a picture / frame. Since a sub-picture forms a rectangular area in a picture / frame, the set of slices, portion of a block, or block belonging to a sub-picture can be determined from parameter set NAL units (i.e., one or more of the DPS, VPS, SPS, PPS, and APSNAL units).

[0121] Encoding and decoding processing

[0122] 3.4 Encoding Processing

[0123] Figure 4 A coding method for encoding a picture of a video in a bit stream according to an embodiment of the present invention is shown.

[0124] In a first step 401, an image is segmented into sub-pictures. For each sub-picture, the sub-picture size is determined based on the spatial access granularity required by the application (e.g., the sub-picture size can be expressed based on the size, scale, or level of granularity of the regions / spatial portions / regions within the picture required by the application / usage scenario, and the sub-picture size can be small enough to encompass a single region / spatial portion / region). Typically, for viewport-dependent streaming, the sub-picture size is set to cover a predetermined range of the field of view (e.g., a 60° horizontal field of view). For region-of-interest adaptive streaming, the width and height of each sub-picture are determined to depend on the region of interest present in the input video sequence. Typically, the size of each sub-picture is set to encompass a region of interest. The sub-picture size is determined in units of luma samples or as multiples of the size of a CTB. Additionally, the position of each sub-picture within the coded picture is determined in step 401. The sub-picture position and size form sub-picture layout information, which is typically signaled in non-VCL NAL units (such as parameter set NAL units). For example, in step 402, the sub-picture layout information is encoded in an SPS NAL unit.

[0125] The SPS syntax for such an SPS NAL unit typically contains the following syntax elements:

[0126]

[0127] The descriptor column gives the encoding method used to encode the syntax element, for example, u(n) (where n is an integer value) means that the syntax element is encoded using n bits, and ue(v) means that the syntax element is encoded using unsigned integer 0-order exponential Golomb coding (where the left leading bit is a variable length code). Except for signed integers, se(v) is equivalent to ue(v). u(v) means that the syntax element is encoded using a fixed-length code (with a specific length in bits determined from other parameters).

[0128] The presence of sub-pictures signaled in the bitstream of a picture depends on the value of the flag subpics_present_flag. When this flag is equal to 0, it indicates that the bitstream does not contain information related to the partitioning of the picture into one or more sub-pictures. In this case, it is inferred that a single sub-picture covering the entire picture exists. When this flag is equal to 1, a set of syntax elements specifies the layout of sub-pictures in a frame (i.e., a picture): signaling includes the use of a loop (i.e., a programming structure for repeating a sequence of instructions until a certain condition is met) to determine / define / specify the sub-pictures of the picture (the number of sub-pictures in a picture is encoded using the sps_num_subpics_minus1 syntax element), which includes defining the position and size of each sub-picture. The index of this "for loop" is the sub-picture index. The syntax elements subpic_ctu_top_left_x[i] and subpic_ctu_top_left_y[i] correspond to the column index and row index of the first CTU of the i-th sub-picture, respectively. The subpic_width_minus1[i] and subpic_height_minus1[i] syntax elements signal the width and height of the i-th sub-picture in units of CTU.

[0129] In addition to the sub-picture layout, the SPS also specifies constraints on sub-picture boundaries: for example, subpic_treated_as_pic_flag[i] equal to 1 indicates that the boundaries of the i-th sub-picture are treated as picture boundaries for temporal prediction. This ensures that the coding blocks of the i-th sub-picture are predicted from data of reference pictures belonging to the same sub-picture. When equal to 0, the flag indicates that temporal prediction may be constrained or unconstrained. The second flag (loop_filter_across_subpic_enabled_flag[i]) specifies whether the loop filtering process is allowed to use data (usually pixel values) from another sub-picture. These two flags make it possible to indicate whether a sub-picture is encoded independently of other sub-pictures. This information is useful when determining whether a sub-picture can be extracted from, derived from, or merged with other sub-pictures.

[0130] In step 403, the encoder determines the partitions in the blocks and slices of the video sequence and describes these partitions in a non-VCL NAL unit such as PPS. Figure 6 This step is further described.The signaling of slice and block partitioning is subject to sub-picture constraints, such that each sub-picture includes at least one slice, and a portion of a block (ie, a partial block) or one or more than one block.

[0131] In step 404, at least one slice forming a sub-picture is encoded in a bitstream.

[0132] 3.5 Decoding Processing

[0133] Figure 5 The following illustrates a general decoding process for slices according to an embodiment of the present invention. For each VCL NAL unit, the decoder determines the PPS and SPS that apply to the current slice. Typically, identifiers for the PPS and SPS used for the current picture are determined. For example, the picture header of the slice signals the identifier of the PPS in use. The PPS associated with this PPS identifier then also references an SPS using another identifier (the SPS identifier).

[0134] In step 501, the decoder determines the sub-picture partitioning, for example by parsing a parameter set describing / indicating the sub-picture layout to determine the size of the sub-pictures of the picture / frame, typically their width and height. For VVC7 and embodiments compliant with this part of VVC7, the parameter set comprising information for determining the sub-picture partitioning is an SPS. In a second step 502, the decoder parses the syntax elements of a parameter set NAL unit (or non-VCL NAL unit) related to the partitioning of the picture into blocks. For example, for a VVC7 compliant stream, the signaling of the block partitioning is in a PPS NAL unit. During this determination step, the decoder initializes a set of variables describing / defining the characteristics of the blocks present in each sub-picture. For example, the following information for the i-th sub-picture may be determined (see Figure 6 Step 601):

[0135] ● A flag indicating whether the sub-picture contains a fragment of a tile (ie, a partial tile) (see Figure 6 Step 603)

[0136] ● An integer value indicating the number of tiles in the sub-picture ( Figure 6 Step 602)

[0137] ●Integer value specifying the width of the sub-image in blocks ( Figure 6 Step 604)

[0138] ●Integer value specifying the height of the sub-image in blocks ( Figure 6 Step 604)

[0139] ● A list of tile indices present in the sub-picture in raster scan order ( Figure 6 Step 605)

[0140] Figure 6 Signaling of stripe partitioning according to an embodiment of the present invention is shown, involving determination steps that may be used in both encoding and decoding processes.

[0141] In step 503, the decoder relies on the signaling of the slice partition (in a non-VCL NAL unit, for example, typically in the PPS of VVC7) and previously determined information to infer (i.e., derive or determine) the slice partition of each sub-picture. In particular, the decoder can infer (i.e., derive or determine) the number of slices, the width and height of one or more slices. The decoder can also obtain information present in the slice header to determine the decoding position of the CTB present in the slice data.

[0142] In a final step 504 , the decoder decodes the slices forming the sub-pictures at the positions determined in step 503 of the picture.

[0143] Signaling a partition

[0144] 3.6 Signaling Stripe Partitions

[0145] According to an embodiment of the present invention, a slice may consist of an integer number of complete blocks of a picture or an integer number of complete and continuous CTU rows or columns within a block (the latter possibility means that the slice can include partial blocks as long as the partial blocks include continuous CTU rows or columns).

[0146] Two slice modes may also be supported / provided for use, namely raster scan slice mode and rectangular slice mode. In raster scan slice mode, a slice contains a sequence of complete blocks of a picture in a block raster scan order. In rectangular slice mode, a slice contains multiple complete blocks that together form a rectangular area of ​​a picture, or multiple consecutive complete CTU rows (or columns) that together form a block of a rectangular area of ​​a picture. Blocks within the slice are scanned in a block raster scan order within the rectangular area corresponding to the rectangular slice.

[0147] The syntax of VVC7 for specifying a slice structure (layout and / or partitioning) is independent of the syntax of VVC7 for sub-pictures. For example, slice partitioning (i.e., partitioning a picture into slices) is performed on top of (i.e., based on or with reference to) a block structure / partition without reference to a sub-picture (i.e., without reference to syntax elements for forming a sub-picture). On the other hand, VVC7 imposes some restrictions (i.e., constraints) on sub-pictures, for example, a sub-picture must contain one or more slices, and a slice header includes a slice_address syntax element, which is an index of the slice relative to its sub-picture (i.e., an index defined for the associated sub-picture, such as an index of a slice within a slice in a sub-picture). VVC7 also only allows strips that include partial blocks in rectangular strip mode, and raster scan strip mode does not define such strips that include partial blocks. The current syntax system used in VVC7 is not designed to enforce all of these constraints, so implementations of the syntax system result in systems that are prone to generating bitstreams that do not conform to the VVC7 specifications / requirements.

[0148] Therefore, embodiments of the present invention use information from the sub-picture layout definition to specify slice partitioning in an attempt to provide better coding efficiency for the signaling of sub-pictures, blocks, and slices.

[0149] The requirement to have the ability to define multiple slices within a block (also called "block fragment" slices or slices that include partial blocks) comes from omni-directional streaming requirements. It has been identified that slices need to be defined within blocks for BEAMER (Bitstream Extraction and Merging) operations for OMAF streams. This means that "block fragment" slices can exist in different sub-pictures to allow BEAMER operations, which then means that having a sub-picture containing one complete block with multiple slices makes little sense.

[0150] The following first three embodiments of the present invention (Example 1, Example 2, and Example 3) define the determination and signaling of slice partitioning based on sub-pictures. The first embodiment 1 includes a syntax system that does not allow / block / prohibit a sub-picture from containing multiple "block fragment" slices, while the second embodiment 2 includes a syntax system that allows / permits only when a sub-picture contains at most one block (i.e., if a sub-picture contains more than one block, all of its slices contain an integer number of complete blocks). Example 3 provides explicit signaling of the use of "block fragment" slices. When "block fragment" slices are not used, signaling syntax elements related to such slices is avoided, which can improve the coding efficiency of signaling slice partitioning. Embodiments 1 to 3 allow / permit the use of block fragment slices only in rectangular slice mode.

[0151] A fourth embodiment 4 is an alternative to the first three embodiments, in which a unified syntax system is used to allow / permit the use of tile segment slices in both raster scan and rectangular slice modes. A fifth embodiment 5 is an alternative to the other embodiments, in which the sub-picture layout and slice partitioning are not signaled in the bitstream, but are inferred (i.e., determined or derived) from the signaling of the block.

[0152] Example 1

[0153] In the first embodiment (Embodiment 1), the syntax of the slice partition of VVC7 is modified to avoid specifying many constraints that are difficult to implement correctly, and the parameters for the slice partition, such as the size of the slice, are also inferred / derived based on the sub-picture layout. Since a sub-picture is represented by a set / group of complete slices (i.e., it is composed of a set / group of complete slices), the size of the slice can be inferred when the sub-picture contains a single slice. Similarly, based on the number of slices in the sub-picture and the size of the slices processed / encountered earlier in the sub-picture, the size of the last slice can be inferred / derived / determined.

[0154] In embodiment 1, a slice is allowed to include a fragment / part of a block (i.e., a partial block) only when the slice covers the entire / whole sub-picture (in other words, a single slice exists in the sub-picture). When more than one slice exists in a sub-picture, it is necessary to signal the slice size. On the other hand, when a single slice exists in a sub-picture, the slice size is the same as the sub-picture size. Therefore, when a slice includes a fragment of a block, the slice size is not signaled in the parameter set NAL unit (because it is the same as the sub-picture size). Therefore, the slice width and height can be constrained to be in units of blocks only for scenarios where the slice size is signaled.

[0155] The PPS syntax of this embodiment includes specifying the number of slices included in each sub-picture. When the number of slices is greater than one, the size (width and height) of the slice is expressed in units of blocks. As described above, the size of the final slice is not signaled and is inferred / derived / determined from the sub-picture layout.

[0156] According to a variation of this embodiment, the PPS syntax contains the following syntax elements with the following semantics (i.e., definitions or functions):

[0157] PPS Syntax

[0158]

[0159]

[0160] PPS semantics

[0161] Slices are defined for each sub-picture. A "for loop" is used together with the num_slices_in_subpic_minus1 syntax element to form / process the correct number of slices in that particular sub-picture. The syntax element num_slices_in_subpic_minus1[i] indicates the number of slices (in the sub-picture with sub-picture index equal to i) minus 1, i.e. the syntax element indicates a value one less than the number of slices in the sub-picture. When equal to 0, it indicates that the sub-picture contains a single slice of size equal to the sub-picture size. When the number of slices is greater than 1, the size of the slice is expressed in units of an integer number of blocks. The size of the final slice is inferred from the sub-picture size (and the sizes of the other blocks in the sub-picture). With this approach, a "block segment" slice can be defined when it covers a complete sub-picture through the following semantics of the syntax element:

[0162] pps_num_subpics_minus1 plus 1 specifies the number of sub-pictures in the coded picture that references the PPS. A requirement for bitstream conformance is that the value of pps_num_subpic_minus1 shall be equal to sps_num_subpics_minus1 (which is the number of sub-pictures defined at the SPS level).

[0163] single_slice_per_subpic_flag equal to 1 specifies that each sub-picture consists of one and only one rectangular slice. single_slice_per_subpic_flag equal to 0 specifies that each sub-picture can consist of one or more rectangular slices. When subpics_present_flag is equal to 0, single_slice_per_subpic_flag shall be equal to 0. When single_slice_per_subpic_flag is equal to 1, num_slices_in_pic_minus1 is inferred to be equal to sps_num_subpics_minus1 (which is the number of sub-pictures defined at the SPS level).

[0164] num_slices_in_subpic_minus1[i] plus 1 specifies the number of rectangular slices in the i-th sub-picture. The value of num_slices_in_subpic_minus1 shall be in the range of 0 to MaxSlicesPerPicture-1 (inclusive), where MaxSlicesPerPicture is specified in Appendix A. When no_pic_partition_flag is equal to 1, the value of num_slices_in_subpic_minus1[0] is inferred to be equal to 0.

[0165] This syntax element determines the length of slice_address of the slice header, which is Ceil(log2(num_slices_in_subpic_minus1[SubPicIdx]+1)) bits (where SubPicIdx is the index of the sub-picture of the slice). The value of slice_address can be in the range of 0 to num_slices_in_subpic_minus1[SubPicIdx] (inclusive).

[0166] tile_idx_delta_present_flag equal to 0 specifies that tile_idx_delta values ​​may be present in the PPS, and that all rectangular slices in all sub-pictures of the picture referencing the PPS are specified in raster order. tile_idx_delta_present_flag equal to 1 specifies that tile_idx_delta values ​​may be present in the PPS, and that all rectangular slices in all sub-pictures of the picture referencing the PPS are specified in the order indicated by the tile_idx_delta values.

[0167] slice_width_in_tiles_minus1[i][j] plus 1 specifies the width of the j-th rectangular strip in tile columns in the i-th sub-picture. The value of slice_width_in_tiles_minus1[i][j] should be in the range of 0 to NumTileColumns-1 (inclusive) (where NumTileColumns is the number of tile columns in the tile grid). When not present, the value of slice_width_in_tiles_minus1[i][j] is inferred from the sub-picture size.

[0168] slice_height_in_tiles_minus1[i][j] plus 1 specifies the height of the j-th rectangular slice in tile rows in the i-th sub-picture. The value of slice_height_in_tiles_minus1[i][j] should be in the range of 0 to NumTileRows-1 (inclusive) (where NumTileRows is the number of tile rows in the tile grid). When not present, the value of slice_height_in_tiles_minus1[i][j] is inferred from the size of the i-th sub-picture.

[0169] tile_idx_delta[i][j] specifies the difference in tile index between the j-th rectangular slice and the (j+1)-th rectangular slice of the i-th sub-picture. The value of tile_idx_delta[i][j] shall be in the range of -NumTilesInPic[i]+1 to NumTilesInPic[i]-1 (inclusive) (where NumTilesInPic[i] is the number of tiles in the picture). When not present, the value of tile_idx_delta[i][j] is inferred to be equal to 0. In all other cases, the value of tile_idx_delta[i][j] shall not be equal to 0.

[0170] Thus, according to this variant:

[0171] A sub-picture is a rectangular area of ​​one or more stripes within a picture. The addresses of stripes are defined relative to the sub-picture. This association / relationship between a sub-picture and its stripes is reflected in the syntax system that defines the stripes in a "for loop" applied to each sub-picture.

[0172] • By design, unwanted partitioning that could result in having more than one block fragment slice from two different blocks in the same sub-picture is avoided.

[0173] • Slice partitioning can be derived from both block and sub-picture partitioning, which improves the signaling coding efficiency.

[0174] According to another variation of this variation, the following process is used to perform this derivation / derivation of slice partitions. For rectangular slices, when single_slice_per_subpic_flag is equal to 0, the list NumCtuInSlice[i] (where i ranges from 0 to num_slices_in_pic_minus1 (inclusive)) (specifying the number of CTUs in the i-th slice) and the matrix CtbAddrInSlice[i][j] (where i ranges from 0 to num_slices_in_pic_minus1 (inclusive) and j ranges from 0 to NumCtuInSlice[i]-1 (inclusive)) (specifying the picture raster scan address of the j-th CTB in the i-th slice) are derived as follows:

[0175]

[0176]

[0177] The function AddCtbsToSlice(sliceIdx, startX, stopX, startY, stopY) fills the CtbAddrInSlice array of the slice with an index equal to SliceIdx. The array is filled with CTB addresses for each CTB in raster scan order, where the vertical address of a CTB row is between startY and stopY, and the horizontal address of a CTB column is between startX and stopX.

[0178] The process includes applying a processing loop to each sub-picture. For each sub-picture, the tile index of the first tile in the sub-picture is determined based on the horizontal and vertical addresses of the first tile of the sub-picture (i.e., the subpicTileTopLeftX[i] and SubpicTileTopLeftX[i] variables) and the number of tile columns specified by the tile partition information. This value infers / indicates / represents the index of the first tile in the first slice of the sub-picture. For each sub-picture, a second processing loop is applied to each slice of the sub-picture. The number of slices is equal to one plus a variable num_slices_in_subpics_minus1[i][j], which is encoded in the PPS or inferred / derived / determined from other information included in the bitstream. When the slice is the last in the sub-picture, the width of the slice in units of tiles is inferred / derived / determined to be equal to the sub-picture width in units of tiles minus the horizontal address of the column of the first tile of the slice plus the horizontal address of the column of the first tile of the sub-picture. Similarly, the height of a slice in a block is inferred / derived / derived / determined to be equal to the sub-picture height in units of blocks minus the vertical address of the row of the first block of the slice plus the vertical address of the row of the first block of the sub-picture. The index of the first block of the previous slice is encoded in the block partition information (e.g., as the difference in block index from the first block of the previous slice), or it is inferred / derived / derived / determined to be equal to the next block in raster scan order of the block in the sub-picture.

[0179] When a sub-picture contains a fragment of a block (i.e., a partial block), the width and height of the slice in CTU units are inferred / derived / derived / determined to be equal to the width and height in CTU units of the sub-picture. The CtbAddrInSlice[sliceIdx] array is filled with the CTUs of the sub-picture in raster scan order. Otherwise, the sub-picture contains one or more blocks, and the CtbAddrInSlice[sliceIdx] array is filled with the CTUs of the blocks included in the slice. The tiles included in the stripe of the sub-picture are tiles having a vertical address of a tile column and a horizontal address of the tile column, the vertical address being defined from the range of [tileX, tileX+slice_width_in_tiles_minus1[i][j]], and the horizontal address being defined from the range of [tileY, tileY+slice_height_in_tiles_minus1[i][j]], where tileX is the vertical address of the tile column of the first tile of the stripe; tiley is the horizontal address of the tile row of the first tile of the stripe; i is the sub-picture index, and j is the index of the stripe in the sub-picture.

[0180] Finally, the processing loop for the slice includes a step for determining the first tile in the next slice of the sub-picture. When the tile index offset (tile_idx_delat[i][j]) is encoded (tile_idx_delta_present_flag[i] is equal to 1), the tile index of the next slice is set to the value equal to the index of the first tile of the current slice plus the tile index offset value. Otherwise (i.e., when the tile index offset is not encoded), tileIdx is set to the tile index of the first tile of the sub-picture plus the product of the number of tile columns in the picture and the height of the current slice in tiles minus 1.

[0181] According to another variant, when a sub-picture includes a partial block, rather than signaling the number of slices included in the sub-picture in the PPS, it is inferred / derived / derived / determined that the sub-picture consists of the partial block. For example, the following alternative PPS syntax can be used to do this.

[0182] Alternative PPS syntax

[0183] In another variation, when a sub-picture represents a fragment of a block (i.e., the sub-picture includes a partial block), the number of slices in the sub-picture is inferred to be equal to one. As the number of sub-pictures representing fragments of blocks (i.e., the number of sub-pictures including partial blocks) increases, the coded data size of the syntax elements is further reduced, which improves the compression of the stream. The syntax of the PPS is, for example, as follows:

[0184]

[0185] The new semantics / definition of num_slices_in_subpic_minus1[i] is as follows:

[0186] num_slices_in_subpic_minus1[i] plus 1 specifies the number of rectangular slices in the i-th sub-picture. The value of num_slices_in_subpic_minus1 shall be in the range of 0 to MaxSlicesPerPicture-1 (inclusive), where MaxSlicesPerPicture is specified in Appendix A. When not present, the value of num_slices_in_subpic_minus1[i] is inferred to be equal to 0 (i in the range of 0 to pps_num_subpic_minus1 (inclusive).

[0187] The tileFractionSubPicture[i] variable specifies whether the i-th sub-picture covers a fragment (partial block) of a block, ie the size of the sub-picture is strictly lower (ie smaller) than the block to which the first CTU of the sub-picture belongs.

[0188] The decoder determines this variable based on the sub-picture layout and tile grid information as follows: When both the top and bottom horizontal boundaries of the sub-picture are tile boundaries, tileFractionSubpicture[i] is set equal to 0. In contrast, when at least one of the top or bottom horizontal sub-picture boundaries is not a tile boundary, tileFractionSubpicture[i] is set equal to 1.

[0189] In yet another variation, the presence or absence of slice_width_in_tiles_minus1[i][j] and slice_height_in_tiles_minus1[i][j] are inferred from the width and height of the sub-picture in units of tiles. For example, the variables subPictureWidthInTiles[i] and subPictureHeightInTiles[i] define the width and height of the i-th sub-picture in units of tiles, respectively. When the sub-picture is a fragment of a tile, the width of the sub-picture is set to 1 (because a fragment of a tile has a width equal to the width of the tile), and the height is conventionally set to 0 to indicate that the height of the sub-picture is less than a full tile. It should be understood that any other preset / predetermined values ​​can be used, the main constraint being that the two values ​​are set / determined so that they do not represent possible sub-picture sizes in units of tiles. For example, the values ​​can be set equal to the maximum number of tiles in the picture plus 1. In this case, it can be inferred that the sub-picture is a fragment of a tile because it is impossible for a sub-picture to have a width or height greater than the maximum number of tiles.

[0190] These variables are initialized once the tile partitioning is determined (typically based on the num_exp_tile_columns_minus1, num_exp_tile_rows_minus1, tile_column_width_minus1[i], and tile_row_height_minus1[i] syntax elements). This processing may include looping through each sub-picture. That is, for each sub-picture, if the sub-picture covers a fragment of a tile, the width and height of the sub-picture in tile units are set to 1 and 0, respectively. Otherwise, the width of the sub-picture in tile units is determined as follows: the horizontal address of the CTU column of the first CTU of the sub-picture (determined from the sub-picture layout syntax element subpic_ctu_top_left_x[i] of the i-th sub-picture) is used to determine the horizontal address of the tile column of the first tile in the sub-picture.

[0191] Then, for each tile column of the tile grid, the horizontal addresses of the leftmost and rightmost CTU columns of the tile column are determined. When the horizontal address of the CTU column of the first CTU of the sub-picture is between these two addresses, the horizontal address indicates the horizontal address of the tile column of the first tile in the sub-picture. The same process applies to determining the horizontal address of the tile column containing the rightmost CTU column of the sub-picture. The horizontal address of the CTU column of the rightmost CTU column is equal to the sum of the horizontal address of the CTU column of the first CTU of the sub-picture and the width of the sub-picture in CTU units (subpic_width_minus1[i] plus 1). The width of the sub-picture in tiles is equal to the difference between the horizontal address of the tile column of the last CTU column and the horizontal address of the first CTU of the sub-picture. The same principle applies when determining the height of the sub-picture in tiles. The process determines the height of the sub-picture in tiles as the difference between the vertical addresses of the tile rows of the first CTU and the last CTU row of the sub-picture.

[0192] In another variation, when the number of tiles in a sub-picture is equal to the number of slices in the sub-picture, slice_width_in_tiles_minus1[i][j] and slice_height_in_tiles_minus1[i][j] do not exist and are inferred to be equal to 0. In effect, the sub-picture and slice constraints enforce that each slice contains exactly one tile, when equal. When the sub-picture is a tile fragment (tileFractionSubpicture[i] is equal to 1), the number of tiles in the sub-picture is equal to 1. Otherwise, it is equal to the product of subPictureHeightInTiles[i] and SubPictureWidthInTiles[i].

[0193] In another variation, when the width of the sub-picture in tiles is equal to 1, slice_width_in_tiles_minus1[i][j] does not exist and is inferred to be equal to 0.

[0194] In another variation, when the height of the sub-picture in tiles is equal to 1, slice_height_in_tiles_minus1[i][j] does not exist and is inferred to be equal to 0.

[0195] In yet another variation, any combination of three of the foregoing another variations is used.

[0196] In some previous variants, the presence of syntax elements was inferred from sub-picture partition information. When sub-picture partitions are defined in different parameter set NAL units, parsing of slice partitions depends on information from the other parameter set NAL units. This dependency can limit the use of the variant in certain applications, because parsing the parameter set that includes the slice partition cannot be performed without storing information from the other parameter set. For decoding of the parameters, i.e., determining the values ​​encoded by the syntax elements, this dependency is not a limitation, because the decoder needs to use all parameter sets in order to decode pixel samples in any way (there may be some delay due to having to wait for decoding of all relevant parameter sets). Therefore, in another variant, the inference of the presence of syntax elements is enabled only when sub-picture, block, and slice partitions are signaled in the same parameter set NAL unit. For example, the variable subpictureWidthInTiles[i] specifying the width of the i-th sub-picture in units of tiles, the variable subpictureHeightInTiles[i] specifying the height of the i-th sub-picture in units of tiles, the variable subpictileTopLeftX[i] specifying the horizontal address of the column of the first tile in the i-th sub-picture, and subpicTileTopLefty[i] specifying the vertical address of the row of the first tile in the i-th sub-picture (where i is in the range of 0 to pps_num_subpicture_minus1 (inclusive)) are determined as follows:

[0197]

[0198]

[0199] The tileFractionSubpicture[i] variable that specifies whether a sub-picture contains a tile fragment is exported as follows:

[0200]

[0201] The list SliceSubpicToPicIdx[i][k] that specifies the number of rectangular slices in the i-th sub-picture and the picture-level slice index of the k-th slice of the i-th sub-picture is derived as follows:

[0202]

[0203] in:

[0204] ●CtbToTileRowBd[ctbAddrY] converts the vertical CTB address (ctbAddrY) to the upper tile row boundary in units of CTB.

[0205] ●CtbTotileColBd[ctbAddrX] converts the horizontal CTB address (ctbAddrX) to the left tile column boundary in units of CTB.

[0206] ●ColWidth[i] is the width of the i-th block column in CTB units

[0207] ●RowHeight[i] is the height of the i-th block row in CTB units

[0208] tileColBd[i] is the position of the i-th tile column boundary in CTB units.

[0209] tileRowBd[i] is the position of the i-th tile row boundary in CTB units.

[0210] NumTileColumns is the number of tile columns

[0211] NumTileRows is the number of tile rows

[0212] Figure 7 An example of signaling sub-picture and slice partitioning using the above embodiment / variant / further variant is shown. In this example, the picture 700 is partitioned into 9 sub-pictures labeled (1) to (9) and a 4×5 block grid (block boundaries are shown with thick solid lines). For each sub-picture, the slice partitioning (the area included in each slice is shown with thin solid lines, which is just within the slice boundary) is as follows:

[0213] ● Sub-picture (1): 3 stripes, which contain 1 block, 2 blocks, and 3 blocks respectively. The height of the stripe is equal to 1 block, and the width of the stripe is 1, 2, and 3 blocks respectively (i.e., 3 stripes consisting of a row of blocks arranged in the horizontal direction)

[0214] Sub-image (2): 2 equal-sized stripes, 1 block wide and 1 block high (i.e., 2 stripes each consisting of a single block)

[0215] ● Sub-pictures (3) to (6): 1 "block fragment" stripe, i.e. a stripe consisting of a single partial block

[0216] Sub-picture (7): 2 stripes of column size 2 blocks (i.e., 2 stripes each consisting of columns of 2 blocks arranged in the vertical direction)

[0217] ●Sub-image (8): 1 strip of 3 blocks

[0218] ● Sub-image (9): 2 stripes with a row size of 1 block and a row size of 2 blocks

[0219] For sub-picture (1), the width and height of the first two slices are encoded, while the size of the last slice is inferred.

[0220] For sub-picture (2), the width and height of the first two slices are inferred since there are two slices for two blocks in the sub-picture.

[0221] For sub-pictures (3) to (6), the number of slices in each sub-picture is equal to 1, and the width and height of the slices are inferred to be equal to the sub-picture size, because each sub-picture is a fragment of a block.

[0222] For sub-pictures (7), the width and height of the first slice and the width and height of the last slice are inferred from the sub-picture size.

[0223] For sub-picture (8), the width and height of the stripe are inferred to be equal to the sub-picture size, since there is a single stripe in the sub-picture.

[0224] For sub-picture (9), the height of the slice is inferred to be equal to 1 (because the sub-picture height in blocks is equal to 1), and the width of the first slice is encoded, while the width of the last slice is equal to the width of the sub-picture minus the width of the first slice.

[0225] Example 2

[0226] In a second embodiment (Example 2), the constraint that a "tile segment" slice (i.e., a slice that is a portion of a tile) should cover the entire sub-picture is relaxed / removed. As a result, a sub-picture can contain one or more slices, each of which includes one or more tiles, but can also contain one or more "tile segment" slices.

[0227] In this embodiment, sub-picture partitioning allows / enables prediction / derivation / determination of slice positions and sizes.

[0228] According to a variation of this embodiment, this operation may be performed using the following PPS syntax.

[0229] PPS Syntax

[0230] For example, the PPS syntax is as follows:

[0231]

[0232]

[0233] The semantics of the syntax elements single_slice_per_subpic_flag, tile_idx_delta_present_flag, num_slices_in_subpic_minus1[i] and tile_idx_delta[i][j] are the same as in the previous embodiment.

[0234] The slice_width_minus1 and slice_height_minus1 syntax elements (parameters) specify the slice size in units of blocks or CTUs, depending on sub-picture partitioning.

[0235] When the last CTU of the slice is the last CTU of the block, the variable newTileIdxDeltaRequired is set to 1. When the slice is not a "block fragment" slice, newTileIdxDeltaRequired is equal to 1. When the slice is a "block fragment" slice, if the slice is not the last in the block in the slice, newTileIdxDeltaRequired is set to 0. Otherwise, it is the last in the block and newTileIdxDeltaRequired is set to 1.

[0236] In a first alternative variant, a sub-picture is constrained / restricted to contain a slice of fragment blocks of a single block. In this case, if the sub-picture contains more than one block, the size is in units of blocks. Otherwise, the sub-picture contains a single block or a portion of a block (partial block), and the slice height is defined in units of CTUs. The width of the slice must be equal to the sub-picture width and can therefore be inferred and does not need to be encoded in the PPS.

[0237] slice_width_minus1[i][j] plus 1 specifies the width of the j-th rectangular strip. The value of slice_width_in_tiles_minus1[i][j] should be in the range of 0 to NumTileColumns-1 (inclusive) (where NumTileColumns is the number of tile columns in the tile grid). When not present (i.e., when SubPictureWidthInTiles[i]*SubPictureHeightInTiles[i] == 1 or SubPictureWidthinTiles[i] is equal to 1), the value of slice_width_in_tiles_minus1[i][j] is inferred to be equal to 0.

[0238] slice_height_in_tiles_minus1[i][j] plus 1 specifies the height of the j-th rectangular slice in the i-th sub-picture. The value of slice_height_in_tiles_minus1[i][j] should be in the range of 0 to NumTileRows-1 (inclusive) (where NumTileRows is the number of tile rows in the tile grid). When not present (i.e., when subPictureHeightInTiles[i] is 1 and num_slices_in_subpic_minus1[i] == 0), the value of slice_height_in_tiles_minus1[i][j] is inferred to be 0.

[0239] The variable SliceWidthInTiles[i][j] that specifies the width of the j-th rectangular strip of the i-th sub-picture in units of blocks, the variable SliceHeightInTiles[i][j] that specifies the height of the j-th rectangular strip of the i-th sub-picture in units of blocks, and the variable SliceHeightInCTUs[i][j] that specifies the height of the j-th rectangular strip of the i-th sub-picture in units of CTB are derived as follows (i is in the range of 0 to pps_num_subpic_minus1, and j is in the range of 0 to num_slices_in_subpic_minus1[i]).

[0240]

[0241]

[0242] With this algorithm, SliceHeightInCTUs[i][j] is only valid when sliceHeightInTiles[i][j] is equal to 0.

[0243] In an alternative variant, a sub-picture is allowed to contain block segment slices from several blocks (i.e., slices that include partial blocks from more than one block), with the limitation / restriction / condition that all slices of the sub-picture must be block segment slices. Thus, a sub-picture cannot be defined that contains a first block segment slice that includes partial blocks of a first block and a second block segment slice that includes partial blocks of a second block that is different from the first block, and another slice that entirely covers a third (different) block (i.e., the third block is a complete / whole block). In this case, if the sub-picture contains more slices than blocks in the sub-picture, the slice_height_minus1[i][j] syntax element is in units of CTUs, and slice_width_minus1[i][j] is inferred to be equal to the sub-picture width in units of CTUs. Otherwise, the size of the slice is in units of blocks.

[0244] In another variation, the slice PPS syntax is as follows:

[0245]

[0246]

[0247] In this other variant, the same principles apply, except that separate syntax elements define the slice width and height in units of tiles or the slice height in units of CTUs, depending on whether the slice is defined in units of CTUs. When the slice is signaled in units of tiles, the slice width and height are defined by slice_width_in_tiles_minus1[i][j] and slice_height_in_tiles_minus1[i][j], while when the width is expressed in units of CTUs, slice_width_in_ctu_minus1[i][j] is used. The variable sliceInCtuFlag[i] is equal to 1 to indicate that the i-th sub-picture contains only tile fragment slices (i.e., there are no entire / complete tile slices in the sub-picture). When equal to 0, it indicates that the i-th sub-picture contains a slice including one or more tiles.

[0248] For i in the range 0 to pps_num_subpic_minus1, the variable sliceInCtuFlag[i] is derived as follows:

[0249]

[0250] The determination of the sliceInCtuFlag[i] variable introduces a parsing dependency between slice and sub-picture partition information. Therefore, in a variant, when slice, block, and sub-picture partitions are signaled in different parameter set NAL units, sliceInCtuFlag[i] is signaled and not inferred.

[0251] In another variant, a sub-picture is allowed to contain block segment slices from several blocks without any specific constraints / restrictions. Thus, a sub-picture can be defined that contains a first block segment slice (which includes a partial block of the first block) and a second block segment slice (which includes a partial block of a second block different from the first block) and another slice (which entirely covers a third (different) block (i.e., the third block is a complete / whole block)). In this case, when the number of blocks in the sub-picture is greater than 1, a flag indicates whether the slice size is specified in CTU or block units. For example, the following syntax of the PPS signals the slice_in_ctu_flag[i] syntax element, which indicates whether the slice size of the i-th sub-picture is expressed in CTU or block units. slice_in_ctu_flag[i] equal to 0 indicates the presence of the slice_width_in_tiles_minus1[i][j] and slice_height_in_tiles_minus1[i][j] syntax elements and the absence of slice_height_in_ctu_minus1[i][j], that is, the slice size is expressed in block units. slice_in_ctu_flag[i] equal to 1 indicates the presence of the slice_height_in_ctu_minus1[i][j] and the absence of the slice_width_in_tiles_minus1[i][j] and slice_height_in_tiles_minus1[i][j] syntax elements, that is, the slice size is expressed in CTU units.

[0252]

[0253]

[0254] Figure 8 An example of signaled sub-picture and slice partitioning using the embodiments / variants / described further above is shown. In this example, the picture 800 is partitioned into six sub-pictures labeled (1) to (6) and a 4×5 grid of blocks (block boundaries are shown with thick solid lines). For each sub-picture, the slice partitioning (regions included in each slice are shown with thin solid lines just inside the slice boundaries) is as follows:

[0255] ● Sub-picture (1): 3 stripes with row sizes of 1 block, 2 blocks, and 3 blocks (i.e., 3 stripes consisting of a row of blocks arranged in the horizontal direction)

[0256] Sub-image (2): 2 equal-sized stripes, the size of 1 block (i.e., 2 stripes each consisting of a single block)

[0257] ● Sub-image (3): 4 "block fragment" strips, i.e. 4 strips each consisting of a single partial block

[0258] Sub-image (4): 2 strips of 2 blocks in a column (i.e., 2 strips each consisting of a column of 2 blocks arranged in a vertical direction)

[0259] ●Sub-image (5): 1 strip of 3 blocks

[0260] ● Sub-image (6): 2 stripes with a row size of 1 block and a row size of 2 blocks

[0261] For sub-picture (3), the number of slices in the sub-picture is equal to 4, and the sub-picture contains only two blocks. The width of the inferred slice is equal to the sub-picture width, and the height of the slice is specified in CTU units.

[0262] For sub-pictures (1), (2), (4) and (5), the number of slices is lower than the blocks in the sub-picture, so the width and height are specified in units of blocks when necessary (i.e., when they cannot be inferred / derived / determined from other information).

[0263] For sub-picture (1), the width and height of the first two slices are encoded, while the size of the last slice is inferred.

[0264] For sub-image (2), the width and height of the first two stripes are inferred since there are 2 stripes of 2 blocks in the sub-image.

[0265] For sub-picture (4), the width and height of the first slice and the size of the last slice are inferred from the sub-picture size.

[0266] For sub-picture (5), the width and height are inferred to be equal to the sub-picture size since there is a single stripe in the sub-picture.

[0267] For sub-picture (5), the slice height is inferred to be 1 (because the sub-picture height in blocks is 1), the width of the first slice is encoded, and the width of the last slice is inferred to be the width of the sub-picture minus the size of the first slice.

[0268] Example 3

[0269] In this third embodiment (Embodiment 3), in the bitstream, it is specified whether tile segment slices are enabled or disabled. The principle is to include a syntax element in one of the parameter set NAL units (or non-VCL NAL units) indicating whether the use of "tile segment" slices is allowed.

[0270] According to a variant, the SPS includes a flag indicating whether "tile segment" slices are allowed, so when the flag indicates that "tile segment" slices are not allowed, the signaling of "tile segment" can be skipped. For example, when the flag is equal to 0, it indicates that "tile segment" slices are not allowed. When the flag is equal to 1, "tile segment" slices are allowed. The NAL unit can include a syntax element to indicate the location of the "tile segment" slice. For example, the following SPS syntax elements and their semantics can be used to do this.

[0271] SPS syntax: used to enable / disable block segment striping

[0272]

[0273] SPS semantics

[0274] sps_tile_fraction_slices_enabled_flag specifies whether "tile slice" slices are enabled in the coded video sequence. sps_tile_fraction_slices_enabled_flag equal to 0 indicates that the slice should contain an integer number of blocks. sps_tile_fraction_slices_enabled_flag equal to 1 indicates that the slice may contain an integer number of blocks or an integer number of CTU rows from a block.

[0275] In another variation, the sps_tile_fraction_slices_enabled_flag is specified at the PPS level to provide more granularity for adaptively applying / defining the presence of "tile fraction" slices. In yet another variation, the flag can be located in the picture header NAL unit to allow / enable the presence of "tile fraction" slices to be adapted on a per-picture basis. The flag can be present in multiple NAL units with different values ​​to allow overriding configurations defined in higher-level parameter sets. For example, the value of the flag in the picture header overrides the value in the PPS, which overrides the value in the SPS.

[0276] In alternative variations, the value of sps_tile_fraction_slices_enabled_flag may be constrained or inferred from other syntax elements. For example, sps_tile_fraction_slices_enabled_flag is inferred to be equal to 0 when sub-pictures are not used in the video sequence (ie, subpics_present_flag is equal to 0).

[0277] Variations of Embodiments 1 and 2 may consider the value of sps_tile_fraction_slices_enabled_flag in a similar manner to infer whether there is a tile slice slice being signaled. For example, the PPS presented above may be modified as follows:

[0278]

[0279]

[0280] When sps_tile_fraction_slices_enabled_flag is equal to 0, the signaling of the slice height is inferred in units of tiles.

[0281] Example 4

[0282] In VVC7, tile segment striping is enabled only for rectangular striping mode. Embodiment 4 described below has the advantage of enabling tile segment striping to be used also in raster scan striping mode. This provides the possibility of adjusting the length of the coded slice in bits more precisely, since the slice boundaries are not constrained to align with the tile boundaries as in VVC7.

[0283] This principle involves defining (or providing information about) slice partitioning in two places. A parameter set defines a slice in terms of tiles. Tile fraction slices are signaled in the slice header. In a variant, the sps_tile_fraction_slices_enabled_flag is pre-set to 1, and tile fractions are always present, signaled in the slice header.

[0284] To achieve this, in practice, the semantics of slices are modified from those of VVC7 and the aforementioned embodiments / variants / further variations: a slice is a collection of one or more slice segments that collectively represent a block or an integer number of complete blocks of a picture. A slice segment represents an integer number of complete blocks within a picture, or an integer number of consecutive complete CTU rows within a block (i.e., a "block fragment"), that are exclusively contained in a single NAL unit, i.e., one or more blocks or "block fragments." A "block fragment" slice is a collection of consecutive CTU rows of a block. A slice segment is allowed to contain all CTU rows of a block. In this case, a slice segment contains a single slice segment.

[0285] According to a variation, the PPS syntax of any previous embodiment is modified to include tile segment specific signaling. An example of such a PPS syntax modification is shown below.

[0286] PPS Syntax

[0287] The PPS syntax of any previous embodiment is modified to remove the block segment specific signaling. For example, the PPS syntax is as follows:

[0288]

[0289]

[0290] For syntactic elements, the same semantics apply.

[0291] Stripe Segment Syntax

[0292]

[0293] The slice segment NAL unit consists of a slice segment header and slice segment data, which is similar to the VVC7 NAL unit structure of a slice. The slice header from the previous embodiment becomes a slice segment header with the same syntax elements as in the slice header, but as a slice segment header, it includes additional syntax elements for locating / identifying slice segments in a slice (for example, described / defined in the PPS).

[0294]

[0295] The slice segment header includes a signal to specify which CTU row the slice segment starts with in the slice. slice_ctu_row_offset specifies the CTU row offset of the first CTU in the slice.

[0296] When rect_slice_flag is equal to 0 (i.e., the slice mode is in raster scan slice mode), the CTU row offset is relative to the first row of the block with an index equal to slice_address. When rect_slice_flag is equal to 1 (i.e., in rectangular slice mode), the CTU row offset is relative to the first CTU of the slice with an index equal to slice_address in the sub-picture identified by slice_subpic_id. The CTU row offset is encoded using either variable or fixed length coding. For fixed length, the number of CTU rows in the slice is determined from the PPS, and the length in bits of the syntax element is equal to log2(the number of CTU rows minus 1).

[0297] There are two ways to indicate the end of a slice segment.

[0298] In the first approach, a slice segment indicates the number of CTU rows in the slice segment (minus 1). The number of CTU rows is encoded using either variable-length coding or fixed-length coding. For fixed-length coding, the number of CTU rows in the slice is determined from the PPS. The length in bits of the syntax element is equal to log2(the difference between the number of CTU rows in the slice and the CTU row offset minus 1).

[0299] The syntax of a stripe header is as follows:

[0300]

[0301]

[0302] num_ctu_rows_in_slice_minus1 specifies the number of CTU rows in the slice segment NAL unit plus 1. The range of num_ctu_rows_in_slice_minus1 is 0 to the number of CTU rows in the blocks contained in the slice minus 2.

[0303] When sps_tile_fraction_slices_enabled_flag is equal to 1 and num_tiles_in_slice_segment_minus1 is equal to 0, the variable NumCtuInCurrSlice, which specifies the number of CTUs in the current slice, is equal to the number of CTU rows multiplied by the width in CTUs of the blocks present in the slice.

[0304] In the second approach, the slice segment data includes a signal to specify whether the slice segment ends at the end of each CTU row. The advantage of this second approach is that the encoder does not have to predetermine the number of CTUs in a given slice segment. This reduces encoder latency, and the encoder can output the slice header in real time, whereas with the first approach, the slice header must be buffered to indicate the number of CTU rows in the slice segment at the end of encoding the slice segment.

[0305] Example 5

[0306] Embodiment 5 is a signaled modification of the sub-picture layout, which can result in improvements in certain situations. In fact, increasing the number of sub-pictures or slices or blocks in a video sequence limits / restricts the time and effectiveness / efficiency of the intra-frame prediction mechanism. As a result, the compression efficiency of the video sequence can be reduced. For this reason, there is a high probability that the sub-picture layout can be predetermined / determined / predicted / estimated based on the application requirements (e.g. the size of the ROI). The encoding process then generates a block partition that will best match the sub-picture layout. In the best case scenario, each sub-picture contains exactly one block. In order to limit the impact on compression efficiency, the encoder attempts to minimize the number of slices for each sub-picture by using a single slice for each sub-picture. Therefore, the best option for the encoder is to define one slice and one block for each sub-picture.

[0307] In this case, the sub-picture layout and the slice layout are the same. This embodiment adds a flag to the SPS to indicate this specific case / scenario / circumstance. When the flag is equal to 1, the sub-picture layout does not exist and can be inferred / derived / determined to be the same as the slice partition. Otherwise, when the flag is equal to 0, the sub-picture layout is explicitly signaled in the bitstream according to the above description of the previous embodiment / variant / additional variant.

[0308] According to a variant, the SPS includes the sps_single_slice_per_subpicture flag for this purpose:

[0309]

[0310] sps_single_slice_per_subpicture equal to 1 indicates that each sub-picture includes a single slice and subpic_ctu_top_left_x[i], subpic_ctu_top_left_y[i], subpic_width_minus1[i], and subpic_height_minus1[i] are not present (i is in the range of 0 to sps_num_subpics_minus1 (inclusive)). sps_single_slice_per_subpicture equal to 0 indicates that the sub-picture may or may not include a single slice and subpic_ctu_top_left_x[i], subpic_ctu_top_left_y[i], subpic_width_minus1[i], and subpic_height_minus1[i] are present (i is in the range of 0 to sps_num_subpics_minus1 (inclusive)).

[0311] According to yet another variant, the PPS syntax includes the following syntax element to indicate that the sub-picture layout can be inferred from the slice layout:

[0312]

[0313]

[0314] If pps_single_slice_per_subpic_flag or sps_single_slice_per_subpic_flag is equal to 1, a single slice exists for each sub-picture. When sps_single_slice_per_subpic_flag is equal to 1, no slice layout exists from the SPS, and pps_single_slice_per_subpic_flag must be equal to 0. The PPS then specifies the slice partitioning. The i-th sub-picture has a size and position corresponding to the i-th slice (i.e., the i-th sub-picture and the i-th slice have the same size and position).

[0315] When sps_single_slice_per_subpic_flag is equal to 0, the slice layout is present in the SPS, and pps_single_slice_per_subpic_flag can be equal to 1 or 0. When sps_single_slice_per_subpic_flag is equal to 1, then the SPS specifies sub-picture partitioning. The i-th slice has a size and position corresponding to the i-th sub-picture (i.e., the i-th slice and the i-th sub-picture have the same size and position).

[0316] To maintain the same sub-picture layout of the encoded video sequence, the encoder may constrain all PPSs that reference an SPS with sps_single_slice_per_subpic_flag equal to 1 to describe / define / enforce the same slice partitioning.

[0317] In a variant, another flag (pps_single_slice_per_tile) is provided in the PPS to indicate that there is a single slice for each tile. When this flag is equal to 1, the slice partition is inferred to be equal to (i.e., the same as) the tile partition. In this case, when sps_single_slice_per_subpic_flag is equal to 1, the sub-picture and slice partition is inferred to be the same as the tile partition.

[0318] Implementation of the embodiment of the present invention

[0319] One or more of the aforementioned embodiments / variants may be implemented in the form of an encoder or decoder that performs one or more of the method steps of the aforementioned embodiments / variants. The following embodiments illustrate such implementations.

[0320] Figure 9a is a flowchart showing the steps of an encoding method according to an embodiment / variant of the present invention, and Figure 9b is a flowchart illustrating the steps of a decoding method according to an embodiment / variant of the present invention.

[0321] according to Figure 9a In the encoding method, sub-picture partition information is obtained at 9911, and slice partition information is obtained at 9912. At 9915, the obtained information is used to determine information for determining one or more of the following: the number of slices in the sub-picture; whether the sub-picture includes only a single slice; and / or whether a slice can include tile segments. Then, at 9919, data for obtaining the determined information is encoded, for example, by providing the data in a bitstream.

[0322] according to Figure 9b In a decoding method, at 9961, data is decoded (e.g., from a bitstream) to obtain information for determining the number of slices in a sub-picture, whether the sub-picture includes only a single slice, and / or whether a slice can include tile segments. At 9964, the obtained information is used to determine one or more of the following: the number of slices in the sub-picture, whether the sub-picture includes only a single slice, and / or whether a slice can include tile segments. Then, at 9967, based on the determination and the result thereof, sub-picture partition information and / or slice partition information are determined.

[0323] It should be understood that any of the aforementioned embodiments / variations may be Figure 10 9402, entropy coding 9409 and / or bitstream generation 9410) or Figure 11 The decoder in (e.g., when performing bitstream processing 9561, entropy decoding 9562 and / or video signal generation 9569) is used.

[0324] Figure 10 A block diagram of an encoder according to an embodiment of the present invention is shown. The encoder is represented by connected modules, each module being adapted to implement at least one corresponding step of a method for encoding an image of a sequence of images according to one or more embodiments / variants of the present invention, for example in the form of programmed instructions executed by a central processing unit (CPU) of the device.

[0325] Encoder 9400 receives as input a raw sequence of digital images i0 through in 9401. Each digital image is represented by a set of samples, sometimes also referred to as pixels (hereinafter referred to as pixels). After the encoding process is completed, encoder 9400 outputs a bitstream 9410. Bitstream 9410 includes data for multiple coding units or image portions, such as slices. Each slice includes a slice header that transmits the encoding values ​​of the encoding parameters used to encode the slice, and a slice body that includes the encoded video data. Module 9402 divides the input digital images i0 through in 9401 into pixel blocks. A block corresponds to an image portion (hereinafter, an image portion refers to any type of part of an image, such as a block, slice, slice segment, or sub-picture) and can be of variable size (e.g., 4×4, 8×8, 16×16, 32×32, 64×64, 128×128 pixels, and several rectangular block sizes are also considered). A coding mode is selected for each input block.

[0326] Two types of coding modes are provided: coding modes based on spatial prediction coding (intra-frame prediction) and coding modes based on temporal prediction (eg inter-frame coding, merge, skip). Possible coding modes are tested.

[0327] Module 9403 implements an intra-frame prediction process in which a block to be coded is predicted by a predictor calculated from the neighboring pixels of the block to be coded. If intra-frame coding is selected, an indication of the selected intra-frame predictor and the difference between the given block and its predictor are encoded to provide a residual.

[0328] Temporal prediction is implemented by the motion estimation module 9404 and the motion compensation module 9405. First, a reference image is selected from the reference image set 9416, and a portion of the reference image (also referred to as a reference region or image portion, which is the region closest to the given block to be encoded (closest in terms of pixel value similarity)) is selected by the motion estimation module 9404. The motion compensation module 9405 then uses the selected region to predict the block to be encoded. The motion compensation module 9405 calculates the difference between the selected reference region and the given block (also referred to as a residual block / data). The selected reference region is indicated using motion information (e.g., a motion vector).

[0329] Therefore, in both cases (spatial and temporal prediction), when not in SKIP mode, the residual is calculated by subtracting the predictor from the original block.

[0330] In the intra prediction implemented by module 9403, the prediction direction is encoded. In the inter prediction implemented by modules 9404, 9405, 9416, 9418, 9417, at least one motion vector or information (data) for identifying such a motion vector is encoded for temporal prediction.

[0331] If inter prediction is selected, information related to the motion vector and the residual block is encoded. To further reduce the bit rate, the motion vector is encoded as a difference relative to the motion vector predictor, assuming uniform motion. The motion vector predictor from the motion information predictor candidate set is obtained from the motion vector field 9418 by the motion vector prediction and encoding module 9417.

[0332] The encoder 9400 further includes a selection module 9406 for selecting a coding mode by applying a coding cost criterion such as a rate-distortion criterion. To further reduce redundancy, a transform module 9407 applies a transform (such as a DCT) to the residual block, and the resulting transformed data is then quantized by a quantization module 9408 and entropy encoded by an entropy coding module 9409. Finally, when not in SKIP mode and the selected coding mode requires encoding of a residual block, the encoded residual block of the current block being encoded is inserted into a bitstream 9410.

[0333] The encoder 9400 also decodes the encoded image to generate a reference image for motion estimation of subsequent images (e.g., a reference image in a reference image / picture 9416). This allows the encoder and decoder receiving the bitstream to have the same reference frame (e.g., using a reconstructed image or a reconstructed image portion). The inverse quantization ("dequantization") module 9411 performs the inverse quantization ("dequantization") of the quantized data, which is then inversely transformed by the inverse transform module 9412. The intra-frame prediction module 9413 uses the prediction information to determine which predictor to use for a given block, and the motion compensation module 9414 essentially adds the residual obtained by module 9412 to a reference region obtained from a reference image set 9416. The reconstructed frame of pixels (image or image portion) is then filtered by module 9415 using post-filtering to obtain another reference image from the reference image set 9416.

[0334] Figure 11 A block diagram of a decoder 9560 that can be used to receive data from an encoder according to an embodiment of the present invention is shown. The decoder is represented by connected modules, each module being adapted to implement a corresponding step of the method implemented by the decoder 9560, for example in the form of programming instructions executed by a CPU of the device.

[0335] The decoder 9560 receives a bitstream 9561 comprising coded units (e.g., data corresponding to image parts, blocks, or coding units), each coding unit consisting of a header containing information about coding parameters and a body containing coded video data. Figure 10 As illustrated, the coded video data is entropy encoded on a predetermined number of bits for a given image portion (e.g., a block or CU), and motion information (e.g., an index to a motion vector predictor) is encoded. The received coded video data is entropy decoded by module 9562. The residual data is then dequantized by module 9563, and an inverse transform is then applied by module 9564 to obtain pixel values.

[0336] The mode data indicating the coding mode is also entropy decoded, and based on the mode, the coding block (unit / set / group) of the image data is intra-type decoded or inter-type decoded. In the case of intra-frame mode, the intra-frame prediction module 9565 determines the intra-frame predictor based on the intra-frame prediction mode specified in the bitstream (for example, the intra-frame prediction mode can be determined using the data provided in the bitstream). If the mode is inter-frame mode, motion prediction information is extracted / obtained from the bitstream to find (identify) the reference area used by the encoder. For example, motion prediction information includes a reference frame index and a motion vector residual. The motion vector predictor is added to the motion vector residual to obtain a motion vector by the motion vector decoding module 9570.

[0337] The motion vector decoding module 9570 applies motion vector decoding to each image portion (e.g., the current block or CU) encoded by motion prediction. Once the index of the motion vector predictor for the current block is obtained, the actual value of the motion vector associated with the image portion (e.g., the current block or CU) can be decoded and used to apply motion compensation by module 9566. The reference image portion indicated by the decoded motion vector is extracted / obtained from the reference image set 9568 so that module 9566 can perform motion compensation. The decoded motion vector is used to update the motion vector field data 9571 for use in predicting subsequently decoded motion vectors.

[0338] Finally, a decoded block is obtained. Where appropriate, post-filtering is applied by the post-filtering module 9567. Finally, a decoded video signal 9569 is obtained and provided by the decoder 9560.

[0339] Figure 12 The data communication system in which one or more embodiments of the present invention may be implemented is illustrated. The data communication system includes a transmitting device (in this case, a server 9201) operable to transmit data packets of a data stream to a receiving device (in this case, a client terminal 9202) via a data communication network 9200. The data communication network 9200 may be a wide area network (WAN) or a local area network (LAN). Such a network may be, for example, a wireless network (Wifi / 802.11a, b, or g), an Ethernet network, an Internet network, or a hybrid network consisting of several different networks. In a specific embodiment of the present invention, the data communication system may be a digital television broadcast system in which the server 9201 transmits the same data content to multiple clients.

[0340] The data stream 9204 provided by the server 9201 can be composed of multimedia data representing video and audio data. In some embodiments of the present invention, the audio and video data streams can be captured by the server 9201 using a microphone and a camera, respectively. In some embodiments, the data stream can be stored on the server 9201 or received by the server 9201 from other data providers, or generated at the server 9201. The server 9201 is provided with an encoder for encoding the video and audio streams, in particular to provide a compressed bit stream for transmission, which is a more compact representation of the data presented as the input of the encoder. In order to obtain a better ratio of the quality of the transmitted data to the amount of the transmitted data, the video data can be compressed according to the high efficiency video coding (HEVC) format or the H.264 / advanced video coding (AVC) format or the versatile video coding (VVC) format. The client 9202 receives the transmitted bit stream and decodes the reconstructed bit stream to reproduce the video image on the display device and reproduce the audio data using the speaker.

[0341] Although a streaming scenario is considered in this embodiment, it will be appreciated that in some embodiments of the present invention, data communication between the encoder and decoder may be performed using, for example, a media storage device such as an optical disc, etc. In one or more embodiments of the present invention, video images may be transmitted along with data representing compensation offsets to be applied to reconstructed pixels of the image to provide filtered pixels in the final image.

[0342] Figure 13 The processing device 9300 configured to implement at least one embodiment / variant of the present invention is schematically illustrated. The processing device 9300 may be a device such as a microcomputer, a workstation, a user terminal, or a lightweight portable device. The device / apparatus 9300 includes a communication bus 9313 connected to:

[0343] - a central processing unit 9311 denoted as CPU, such as a microprocessor;

[0344] a read-only memory 9307, denoted ROM, for storing computer programs / instructions for operating the device 9300 and / or implementing the present invention;

[0345] - a random access memory 9312 represented as a RAM for storing executable codes of the method of the embodiment / variant of the present invention, and registers suitable for recording variables and parameters required for implementing the method for encoding a digital image sequence and / or the method for decoding a bit stream according to the embodiment / variant of the present invention; and

[0346] - a communication interface 9302 connected to a communication network 9303, through which digital data to be processed is transmitted or received.

[0347] Optionally, the device 9300 may further include the following components:

[0348] - a data storage component 9304 such as a hard disk, which is used to store a computer program for implementing the method of one or more embodiments / variants of the present invention and data used or generated during the implementation of one or more embodiments / variants of the present invention;

[0349] a disk drive 9305 for a disk 9306 (e.g. a storage medium), the disk drive 9305 being adapted to read data from or write data to the disk 9306, or;

[0350] - A screen 9309 for displaying data and / or serving as a graphical interface for interaction with the user by means of a keyboard 9310, a touch screen or any other indication / input means.

[0351] The device 9300 may be connected to various peripheral devices such as a digital camera 9320 or a microphone 9308 , each of which is connected to an input / output card (not shown) to provide multimedia data to the device 9300 .

[0352] The communication bus provides communication and interoperability between the various elements included in or connected to the device 9300. The representation of the bus is not limiting, and in particular, the central processing unit 9311 is operable to communicate instructions to any element of the device 9300 directly or with the aid of other elements of the device 9300.

[0353] The disk 9306 may be replaced by any information medium such as a rewritable or non-rewritable compact disk (CD-ROM), a ZIP disk or a memory card, and in general by an information storage component that can be read by a microcomputer or a processor, the disk 9306 being integrated into the device or not, possibly removable and suitable for storing one or more programs whose execution enables the implementation of the method for encoding a digital image sequence and / or the method for decoding a bit stream according to the present invention.

[0354] The executable code may be stored in a read-only memory 9306, on a hard disk 9304 or on a removable digital medium such as, for example, the disk 9306 as previously described. According to a variant, the executable code of the program may be received via the interface 9302 by means of the communication network 9303 to be stored in one of the storage means of the device 9300 (for example, the hard disk 9304) before execution.

[0355] The central processing unit 9311 is adapted to control and direct the execution of instructions or portions of software code for executing one or more programs according to the present invention, instructions stored in one of the aforementioned storage means. Upon power-up, one or more programs stored in non-volatile memory (e.g., on hard disk 9304, disk 9306, or in read-only memory 9307) are transferred to random access memory 9312 (which then contains the executable code of the one or more programs) and registers for storing variables and parameters necessary for implementing the present invention.

[0356] In this embodiment, the device is a programmable device that implements the invention using software. Alternatively, however, the invention may be implemented in hardware (for example in the form of an application specific integrated circuit or ASIC).

[0357] Implementation of the embodiment of the present invention

[0358] Implementation of the embodiment of the present invention

[0359] It should also be understood that according to other embodiments of the present invention, a decoder according to the above embodiments / variants is provided in a user terminal such as a computer, a mobile phone (cellular phone), a tablet or any other type of device (e.g., a display device) capable of providing / displaying content to a user. According to yet another embodiment, an encoder according to the above embodiments / variants is provided in an image capture device, which also includes a camera, a video camera or a web camera (e.g., a closed-circuit television or video surveillance camera) for capturing and providing content for encoding by the encoder. See below Figure 14 and 15 Two such examples are provided.

[0360] Figure 14 9450 is a diagram illustrating a network camera system 9450 including a network camera 9452 and a client device 9454 .

[0361] The network camera 9452 includes a camera unit 9456, an encoding unit 9458, a communication unit 9460, and a control unit 9462. The network camera 9452 and the client device 9454 are connected to each other via the network 9200 so as to be able to communicate with each other. The camera unit 9456 includes a lens and an image sensor (e.g., a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS)), and captures an image of an object and generates image data based on the image. The image can be a still image or a video image. The camera unit may also include a zoom component and / or a pan component adapted for zooming or panning (optically or digitally), respectively. The encoding unit 9458 encodes the image data using the encoding method described in the aforementioned embodiments / variants. The encoding unit 9458 uses at least one of the encoding methods described in the aforementioned embodiments / variants. For other examples, the encoding unit 9458 may use a combination of the encoding methods described in the aforementioned embodiments / variants.

[0362] The communication unit 9460 of the network camera 9452 transmits the encoded image data encoded by the encoding unit 9458 to the client device 9454. In addition, the communication unit 9460 can also receive commands from the client device 9454. The commands include commands for setting parameters for encoding by the encoding unit 9458. The control unit 9462 controls other units in the network camera 9452 according to the commands received by the communication unit 9460.

[0363] The client device 9454 includes a communication unit 2114, a decoding unit 2116, and a control unit 9468. The communication unit 2118 of the client device 9454 can transmit commands to the network camera 9452. Furthermore, the communication unit 2118 of the client device 9454 receives encoded image data from the network camera 9452. The decoding unit 9466 decodes the encoded image data using one or more of the decoding methods described in the aforementioned embodiments / variants. In other examples, the decoding unit 9466 may use a combination of the decoding methods described in the aforementioned embodiments / variants. The control unit 9468 of the client device 9454 controls other units in the client device 9454 based on user operations or commands received by the communication unit 2114. The control unit 9468 of the client device 9454 can also control the display device 9470 to display the image decoded by the decoding unit 9466.

[0364] The control unit 9468 of the client device 9454 further controls the display device 9470 to display a GUI (Graphical User Interface) for specifying the values ​​of parameters of the network camera 9452 (e.g., parameters used for encoding by the encoding unit 9458). The control unit 9468 of the client device 9454 can also control other units in the client device 9454 based on user operation input to the GUI displayed on the display device 9470. The control unit 9468 of the client device 9454 can also control the communication unit 9464 of the client device 9454 based on user operation input to the GUI displayed on the display device 9470 to transmit a command for specifying the values ​​of the parameters of the network camera 9452 to the network camera 9452.

[0365] Figure 15 9500 is a diagram illustrating a smartphone 9500. The smartphone 9500 includes a communication unit 9502, a decoding / encoding unit 9504, a control unit 9506, and a display unit 9508.

[0366] The communication unit 9502 receives the encoded image data via the network. The decoding / encoding unit 9504 decodes the encoded image data received by the communication unit 9502. The decoding unit 9504 decodes the encoded image data using the decoding method described in one or more of the aforementioned embodiments / variants. The decoding / encoding unit 9504 may also use at least one of the encoding or decoding methods described in the aforementioned embodiments / variants. In other examples, the decoding / encoding unit 9504 may use a combination of the decoding or encoding methods described in the aforementioned embodiments / variants.

[0367] The control unit 9506 controls other units in the smartphone 9500 according to a user operation or command received by the communication unit 9502. For example, the control unit 9506 controls the display unit 9508 to display an image decoded by the decoding / encoding section 9504.

[0368] The smartphone may also include an image recording device 9510 (e.g., a digital camera and associated circuitry) for recording images or video. Such recorded images or video may be encoded by the decoding / encoding section 9504 under the instruction of the control unit 9506. The smartphone may also include a sensor 9512 suitable for sensing the orientation of the mobile device. Such a sensor may include an accelerometer, a gyroscope, a compass, a global positioning (GPS) unit, or a similar position sensor. Such a sensor 2212 can determine whether the smartphone has changed orientation and can use such information when encoding the video stream.

[0369] Although the present invention has been described with reference to embodiments and modifications thereof, it will be understood that the present invention is not limited to the disclosed embodiments / modifications. It will be understood by those skilled in the art that various changes and modifications may be made without departing from the scope of the invention as defined by the appended claims. All features disclosed in this specification (including any appended claims, abstracts and drawings), and / or all steps of any method or process disclosed, may be combined in any combination, except for at least some mutually exclusive combinations of such features and / or steps. Unless expressly stated otherwise, each feature disclosed in this specification (including any appended claims, abstracts and drawings) may be replaced by alternative features for the same, equivalent or similar purposes. Therefore, unless expressly stated otherwise, each feature disclosed is merely an example of a general series of equivalent or similar features.

[0370] It should also be understood that any result of the above-mentioned comparison, determination, inference, evaluation, selection, execution, conduct, or consideration (e.g., a selection made during encoding, processing, or partitioning) can be indicated in data in the bitstream (e.g., a flag or information indicating the result) or can be determined / inferred from data in the bitstream, so that the indicated or determined / inferred result can be used in the processing instead of actually performing the comparison, determination, evaluation, selection, execution, conduct, or consideration during, for example, decoding or partitioning. It should be understood that when a "table" or "lookup table" is used, other data types such as arrays can also be used to perform the same function, as long as the data type is capable of performing the same function (e.g., representing a relationship / mapping between different elements).

[0371] In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be used to advantage. Reference signs appearing in the claims are by way of illustration only and shall not have a limiting effect on the scope of the claims.

[0372] In the foregoing embodiments / variations, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or codes, and executed by a hardware-based processing unit.

[0373] Computer-readable media may include computer-readable storage media, which corresponds to tangible media such as data storage media or communication media including, for example, any medium that facilitates transfer of a computer program from one place to another according to a communication protocol. In this manner, computer-readable media may generally correspond to (1) a non-transitory, tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described herein. A computer program product may include computer-readable media.

[0374] As an example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, flash memory or any other medium that can be used for storing desired program code in the form of instruction or data structure and can be accessed by a computer. In addition, any connection may be appropriately referred to as a computer-readable medium. For example, if instructions are sent from a website, server or other remote source using a coaxial cable, optical fiber cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave, etc.), then the coaxial cable, optical fiber cable, twisted pair, DSL or wireless technology (such as infrared, radio and microwave, etc.) are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connection, carrier wave, signal or other transient media, but are for non-transient tangible storage media. Disk and disc used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and blue disc, wherein the disc usually copies data magnetically, and the disc reproduces data optically by laser. Combinations of the above should also be included within the scope of computer-readable media.

[0375] Instructions may be executed by one or more processors such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate / logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor," as used herein, may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Furthermore, the techniques may be fully implemented in one or more circuits or logic elements.

Claims

1. A method for decoding image data of one or more images, wherein the image can include one or more slices and the image can include one or more sub-pictures, the method comprising: Obtaining, from a sequence parameter set (SPS), first information indicating a width of a sub-picture and second information indicating a height of the sub-picture; determining parameters associated with a slice included in the sub-picture using the first information indicating a width of the sub-picture and the second information indicating a height of the sub-picture, wherein the slice can correspond to an integer number of consecutive complete coding tree unit rows (CTU rows) within a block; and decoding the image using at least the determined parameters, In the decoding of the image, at least inter-frame prediction is used.

2. A method for encoding image data of one or more images, wherein an image can include one or more slices and the image can include one or more sub-pictures, the method comprising: Encoding first information indicating a width of a sub-picture and second information indicating a height of the sub-picture in a sequence parameter set (SPS); determining parameters associated with a slice included in the sub-picture using the first information indicating a width of the sub-picture and the second information indicating a height of the sub-picture, wherein the slice can correspond to an integer number of consecutive complete coding tree unit rows (CTU rows) within a block; and encoding the image using at least the determined parameters, In which, in encoding of the image, at least inter-frame prediction is used.

3. An apparatus for decoding image data of one or more images, wherein the image can include one or more slices and the image can include one or more sub-pictures, the apparatus comprising: An information decoder, configured to obtain, from a sequence parameter set (SPS), first information indicating a width of a sub-picture and second information indicating a height of the sub-picture; a parameter determiner configured to determine parameters associated with a slice included in the sub-picture using the first information indicating a width of the sub-picture and the second information indicating a height of the sub-picture, wherein the slice can correspond to an integer number of consecutive complete coding tree unit rows (CTU rows) within a block; and an image decoder for decoding the image using at least the determined parameters, Wherein, when the image decoder decodes the image, at least inter-frame prediction is used.

4. An apparatus for encoding image data of one or more images, wherein an image can include one or more slices and the image can include one or more sub-pictures, the apparatus comprising: an information encoder for encoding first information indicating a width of a sub-picture and second information indicating a height of the sub-picture in a sequence parameter set (SPS); a parameter determiner configured to determine parameters associated with a slice included in the sub-picture using the first information indicating a width of the sub-picture and the second information indicating a height of the sub-picture, wherein the slice can correspond to an integer number of consecutive complete coding tree unit rows (CTU rows) within a block; and an image encoder for encoding the image using at least the determined parameters, Wherein, when the image encoder encodes the image, at least inter-frame prediction is used.

5. A non-transitory computer-readable medium carrying a computer program, the computer program comprising program instructions adapted to, when executed by one or more processors, perform a method for decoding image data of one or more images, wherein an image can include one or more slices and the image can include one or more sub-pictures, the method comprising: Obtaining, from a sequence parameter set (SPS), first information indicating a width of a sub-picture and second information indicating a height of the sub-picture; determining parameters associated with a slice included in the sub-picture using the first information indicating a width of the sub-picture and the second information indicating a height of the sub-picture, wherein the slice can correspond to an integer number of consecutive complete coding tree unit rows (CTU rows) within a block; and decoding the image using at least the determined parameters, In the decoding of the image, at least inter-frame prediction is used.

6. A non-transitory computer-readable medium carrying a computer program, the computer program comprising program instructions adapted to, when executed by one or more processors, perform a method for encoding image data of one or more images, wherein an image can include one or more slices and the image can include one or more sub-pictures, the method comprising: Encoding first information indicating a width of a sub-picture and second information indicating a height of the sub-picture in a sequence parameter set (SPS); determining parameters associated with a slice included in the sub-picture using the first information indicating a width of the sub-picture and the second information indicating a height of the sub-picture, wherein the slice can correspond to an integer number of consecutive complete coding tree unit rows (CTU rows) within a block; and encoding the image using at least the determined parameters, In which, in encoding of the image, at least inter-frame prediction is used.

7. A computer program product comprising a computer program, the computer program comprising program instructions adapted to, when executed by one or more processors, perform a method for decoding image data of one or more images, wherein an image can comprise one or more slices and the image can comprise one or more sub-pictures, the method comprising: Obtaining, from a sequence parameter set (SPS), first information indicating a width of a sub-picture and second information indicating a height of the sub-picture; determining parameters associated with a slice included in the sub-picture using the first information indicating a width of the sub-picture and the second information indicating a height of the sub-picture, wherein the slice can correspond to an integer number of consecutive complete coding tree unit rows (CTU rows) within a block; and decoding the image using at least the determined parameters, In the decoding of the image, at least inter-frame prediction is used.

8. A computer program product comprising a computer program, the computer program comprising program instructions adapted to, when executed by one or more processors, perform a method for encoding image data of one or more images, wherein an image can comprise one or more slices and the image can comprise one or more sub-pictures, the method comprising: Encoding first information indicating a width of a sub-picture and second information indicating a height of the sub-picture in a sequence parameter set (SPS); determining parameters associated with a slice included in the sub-picture using the first information indicating a width of the sub-picture and the second information indicating a height of the sub-picture, wherein the slice can correspond to an integer number of consecutive complete coding tree unit rows (CTU rows) within a block; and encoding the image using at least the determined parameters, In which, in encoding of the image, at least inter-frame prediction is used.

Images