Encode and decode images based on tile group ID

By introducing tile group ID and predetermined sequential scanning in video encoding technology, the problems of limited tile flexibility and complex scene grouping are solved, efficient tile decoding and rendering are achieved, and video compression efficiency and user experience are improved.

CN112823525BActive Publication Date: 2024-12-06TELEFONAKTIEBOLAGET LM ERICSSON (PUBL)
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN201980065682.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-10-02
Filing Date
2019-10-02
Publication Date
2024-12-06
Estimated Expiration
2039-10-02

AI Technical Summary

Technical Problem

In the existing video encoding technology, the flexibility of tiles is limited and the non-adjacent tiles cannot be effectively supported, resulting in reduced compression efficiency and unnecessary tiled artifacts. In complex scenarios such as 3DoF, 3DoF+ and 6DoF videos. In this case, it is difficult to achieve efficient decoding and rendering of tile grouping.

Method used

Introduce tile group ID, assign each tile to the tile group, and scan and decode the tile group in a predetermined order in the bitstream, allowing tile combinations in non-adjacent areas to improve tile flexibility and decoding efficiency.

Benefits of technology

Through the use of tile group ID, the logical grouping of tiles is realized and the extraction speed of tiles in the bitstream is optimized, which reduces the delay and supports efficient decoding and rendering of complex scenarios, improving the compression efficiency and user experience of videos .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112823525B_ABST
    Figure CN112823525B_ABST
Patent Text Reader

Abstract

An encoder (80) and a decoder (60) are provided for encoding and decoding a picture, respectively. The encoder defines the structure of a picture segment by dividing the picture into a plurality of picture segment groups (T1, T2) and assigning a unique group ID to each group. Each picture segment group is further divided into one or more picture segments or "tiles" (T), wherein each picture segment is assigned the ID of its picture segment group. The encoder then encodes the picture segments of the picture segment group into a bitstream (B) and sends the bitstream to the decoder. Upon receiving the bitstream, the decoder extracts each picture segment group as a single entity and decodes the picture segments in each picture segment group in a predetermined order.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Application No. 62 / 740,360, filed on October 2, 2018, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates generally to video encoding and decoding techniques, and more particularly, to encoding and decoding of slices of pictures. Background Art

[0004] High Efficiency Video Coding (HEVC) is a block-based video codec standardized by both ITU-T and Moving Picture Experts Group (MPEG), utilizing both spatial prediction techniques and temporal prediction techniques. Spatial prediction reduces spatial redundancy and is implemented using intra (I) prediction within the same frame from the current picture. Temporal prediction reduces temporal redundancy and is implemented using inter (P) or bidirectional inter (B) prediction at the block level using previously decoded reference pictures. However, regardless of the specific type of prediction technique, the difference between the resulting original pixel data and the predicted pixel data, known as the "residual," is transformed into the frequency domain and quantized. The transformed residual is quantized, with the level determined by the quantization parameter (QP), helping to control the tradeoff between bitrate and video quality.

[0005] The transformed and quantized residual is then entropy encoded along with the necessary prediction parameters before being sent to the decoder. The prediction parameters that are also entropy encoded include the prediction mode and motion vectors. Upon receipt, the decoder performs entropy decoding, inverse quantization, and inverse transformation to obtain the residual. The decoder then reconstructs the image from the residual using intra-frame prediction or inter-frame prediction techniques.

[0006] Both MPEG and ITU-T are working on a successor to HEVC within the Joint Video Exploration Team (JVET). The name of the video codec is Versatile Video Coding (VVC). Summary of the invention

[0007] Embodiments herein are useful for encoding and decoding pictures that are divided into picture segments (referred to herein as "tiles"). For example, in one or more embodiments, a picture has a flexible partitioning structure in which an edge of a first tile abuts corresponding edges of two or more adjacent tiles in the picture. In these embodiments, a tile group ID can be used to identify which region of the picture each tile belongs to. With embodiments of the present disclosure, not all portions of a region need to be spatially adjacent.

[0008] Therefore, one or more embodiments of the present disclosure provide a method for decoding a picture. In one embodiment, the method includes: receiving a bitstream including one or more coded picture slice groups, wherein each picture slice group includes a group ID (e.g., a tile group ID as described above) and one or more coded picture slices; extracting a first picture slice group from the bitstream as a single entity; and decoding the one or more picture slices in the first picture slice group in a predetermined order.

[0009] It should be noted that a "picture slice group" is generally known as a sub-portion of a picture. In addition, as known to those of ordinary skill in the art, a "tile group" is also generally known as a sub-portion of a picture. Therefore, as described in more detail later, a picture slice group and a tile group refer to the same thing.

[0010] An embodiment of the present disclosure also provides a decoder. In one embodiment, the decoder includes a communication interface circuit and a processing circuit. The communication interface circuit is configured to receive a bitstream including one or more encoded picture slice groups from an encoder, wherein each picture slice group includes a group ID and one or more encoded picture slices. The processing circuit is configured to extract a first picture slice group from the bitstream as a single entity, and decode the one or more picture slices in the first picture slice group in a predetermined order.

[0011] An embodiment of the present disclosure also provides a method for encoding a picture. In one embodiment, the method includes: defining a structure of a picture segment, wherein defining the structure includes: dividing the picture into a plurality of picture segment groups, and assigning a unique group ID to each picture segment group. The method also includes: for each picture segment group, dividing the picture segment group into one or more tiles, and assigning a group ID of the picture segment group to each picture segment in the picture segment group. The method also includes: encoding the picture segments of each picture segment group into a bitstream, and sending the encoded bitstream to a decoder.

[0012] An embodiment of the present disclosure also provides an encoder, which includes a communication interface circuit and a processing circuit. In one embodiment, the communication interface circuit is configured to send a bitstream including one or more encoded picture fragment groups to a decoder, wherein each picture fragment group includes a group ID and one or more encoded picture fragments. The processing circuit is configured to define the structure of the picture fragment by dividing the picture into a plurality of picture fragment groups and assigning a unique group ID to each picture fragment group. In addition, the processing circuit is configured to divide the picture fragment group into one or more picture fragments for each picture fragment group, and assign the group ID of the picture fragment group to each picture fragment in the picture fragment group. The processing circuit is also configured to encode the picture fragments of each picture fragment group into a bitstream, and send the encoded bitstream to the decoder via the communication interface circuit.

[0013] An embodiment of the present disclosure also provides a decoder circuit configured to receive a bit stream comprising one or more encoded picture segment groups, wherein each picture segment group comprises a group ID and one or more encoded picture segments; extract a first picture segment group from the bit stream as a single entity; and decode the one or more picture segments in the first picture segment group in a predetermined order.

[0014] An embodiment of the present disclosure also provides an encoder circuit configured to send a bitstream including one or more encoded picture segment groups to a decoder, wherein each picture segment group includes a group ID and one or more encoded picture segments. The encoder circuit is also configured to define the structure of the picture segment by dividing the picture into a plurality of picture segment groups and assigning a unique group ID to each picture segment group. The encoder circuit is also configured to divide the picture segment group into one or more picture segments for each picture segment group, and assign the group ID of the picture segment group to each picture segment in the picture segment group. The encoder circuit is also configured to encode the picture segments of each picture segment group into a bitstream, and send the encoded bitstream to the decoder via the communication interface circuit.

[0015] Those skilled in the art will appreciate that the embodiments herein also include corresponding devices, computer programs, and carriers containing these computer programs. For example, in one or more embodiments, the computer program includes instructions that, when executed by a processing circuit in a decoder, cause the decoder to perform any of the above-described processes.

[0016] In one or more embodiments, the computer program includes instructions that, when executed by processing circuitry in an encoder, cause the encoder to perform any of the above-described processes.

[0017] In this regard, the computer program may include one or more code units or modules corresponding to the above-mentioned means or units.

[0018] The embodiment also includes a carrier embodying such a computer program. The carrier may include one of an electric signal, an optical signal, a radio signal or a computer-readable storage medium.

[0019] Embodiments herein also provide a non-transitory computer-readable storage medium containing a computer program, which, when executed by processing circuitry in a decoder, causes the decoder to perform as described above.

[0020] Embodiments herein also provide a non-transitory computer-readable storage medium containing a computer program, which, when executed by processing circuitry in an encoder, causes the encoder to perform as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 An example of partitioning a CTU into CUs using QTBT is shown.

[0022] Figure 2 An example tile partition is shown.

[0023] FIG. 3A to FIG. 3B An example tiling structure with high-resolution tiles and low-resolution tiles is shown.

[0024] Figure 4 An example of a tile structure that is not supported in HEVC is shown.

[0025] Figure 5 An example tile structure that may be supported in HEVC is shown.

[0026] FIG. 6A to FIG. 6B shows side-by-side frame packing ( Fig. 6A ) and upper and lower frame encapsulation ( Figure 6B ) example.

[0027] FIG. 7A to FIG. 7C Use cases for 3DoF, 3DoF+ and 6DoF video are shown respectively.

[0028] Figure 8 A stereoscopic picture utilizing side-by-side frame packing is shown.

[0029] Fig.9A The order of tiles in the bitstream when tile groups are not used is shown. White indicates tiles belonging to the left view and grey indicates tiles belonging to the right view.

[0030] Fig. 9B The order of tiles in the bitstream when tile groups are used is shown. White indicates tiles belonging to the left view, and grey indicates tiles belonging to the right view.

[0031] Fig.10 is a flow chart illustrating a method implemented at a decoder according to one embodiment.

[0032] Fig.11 is a flowchart illustrating a method of decoding a picture or a portion of a picture (having tiles grouped with tile group IDs) from a bitstream according to one embodiment.

[0033] Fig.12 is a flowchart illustrating a method of extracting a portion of a picture (having tiles grouped with tile group IDs) in a picture from a bitstream according to one embodiment.

[0034] Fig.13 is a flowchart illustrating a method for encoding a picture (with tiles using tile group IDs) into a bitstream according to one embodiment.

[0035] FIG. 14A to FIG. 14D A cubemap and scan order are shown according to one embodiment.

[0036] Fig.15 An example of using tile groups when packing cubemaps with different resolutions into a frame is shown, according to one embodiment.

[0037] Fig.16 A rearrangement of a fragmented rectangular frame packing arrangement is shown according to one embodiment.

[0038] Fig.17 Frame packing for texture plus depth is shown according to one embodiment.

[0039] Fig.18 An example of frame packing for 3DoF+ video using tile groups according to one embodiment is shown. Each tile group may consist of several tiles for spatial random access.

[0040] Fig.19 An example of a video encoded using three tile groups according to one embodiment is shown.

[0041] Fig. 20 Examples of some aspect ratios supported by a bitstream according to one embodiment are shown.

[0042] Fig.21 is a schematic diagram of an encoder configured according to one embodiment.

[0043] Fig. 22 is a schematic diagram of a decoder configured according to one embodiment.

[0044] Fig.23 is a schematic block diagram illustrating some functional components of an encoder according to one embodiment.

[0045] Fig.24 is a schematic block diagram illustrating some functional components of a decoder according to one embodiment.

[0046] Fig.25 A system including an encoder and a decoder according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0047] As mentioned earlier, HEVC is a block-based video codec standardized by ITU-T and MPEG, using both temporal prediction and spatial prediction. HEVC uses a block structure in which each top-level coding block, i.e., the largest block in the coding block partition, is divided by a quadtree (QT) structure, referred to herein as a coding tree unit (CTU). This division produces coding block partitions, referred to herein as coding units (CUs). The CU can then be further recursively divided into smaller CUs of equal size using a quadtree structure, up to a block size of 8×8.

[0048] The block structure in the current version of VVC is different from that in HEVC. Specifically, the block structure in VVC is called a quadtree plus binary tree (QTBT) plus ternary tree (TT) block structure (QTBT+TT). The CU in QTBT+TT can have a square or rectangular shape. As in HEVC, the coding tree unit (CTU) is first divided by a quadtree structure. Then, the CTU is further divided into partitions of equal size in the vertical or horizontal direction with a binary tree structure to form a coding block (also referred to as a coding unit-CU). Therefore, the block can have a square or rectangular shape.

[0049] The encoder can set the depth of quadtrees and binary trees in the bitstream. Figure 1 An example of partitioning a CTU using QTBT+TT is shown. TT allows a CU to be divided into three partitions instead of two partitions of equal size. This increases the possibility of using a block structure that is more suitable for the content structure in the picture.

[0050] Context-Adaptive Binary Arithmetic Coding (CABAC) is an entropy coding tool used in HEVC and VVC. CABAC is configured to encode binary symbols, which keeps the complexity low and allows the probability of more frequently used bits of the symbol to be modeled. Since the coding mode is usually strongly correlated locally, the probability model is adaptively selected based on the local context.

[0051] The concept of slices in HEVC divides a picture into independently coded slices, where each slice is read in raster scan order in units of CTUs. Different coding types can be used for slices of the same picture. For example, a slice can be an I slice, a P slice, or a B slice. However, the main purpose of a slice is to achieve resynchronization in the event of lost data.

[0052] The HEVC video coding standard also includes a tool called "tiles" that divide a picture into rectangular spatially independent regions. Using tiles, a picture in HEVC can be divided into multiple rows and columns of samples, where any given tile is located at the intersection of a given row and a given column. Figure 2An example of tile partitioning using four (4) rows of tiles and five (5) columns of tiles is shown, resulting in a total of twenty (20) tiles for the picture. Figure 2 As shown, tiles in HEVC are always aligned to CTU boundaries.

[0053] The tile partitioning structure, also referred to herein as "tile structure," is signaled in a picture parameter set (PPS) by specifying the thickness of rows and the width of columns. Individual rows and columns can have different sizes, but the partitioning always spans the entire picture, from left to right and from top to bottom, respectively.

[0054] Table 1 lists the PPS syntax used to specify tile structure in HEVC. As shown in Table 1, the flag (i.e., tiles_enabled_flag) indicates whether tiles are used. If tiles_enabled_flag is set, the number of tile columns (i.e., num_tile_columns_minus1) and the number of tile rows (i.e., num_tile_rows_minus1) are specified. uniform_spacing_flag is a flag that specifies whether the column width and row height are explicitly signaled or a predetermined method of equally spacing the tile boundaries should be used. If explicit signaling is indicated, the column width is signaled one by one, followed by the row height. The column width and row height information is signaled in units of CTU. Finally, the flag loop_filter_across_tiles_enabled_flag specifies whether the loop filter across tile boundaries is turned on or off for all tile boundaries in the picture.

[0055]

[0056] Table 1 - Tile syntax in HEVC

[0057] Similar to slices, there are no decoding dependencies between tiles of the same picture. This includes intra prediction, context selection, and motion vector prediction. However, one exception is that loop filter dependencies are generally allowed between tiles. However, these dependencies can be disabled by setting loop_filter_across_tiles_enabled_flag appropriately.

[0058] In contrast to slices, tiles do not require as much header data. The header overhead for each tile includes the signaling of bitstream offsets, which are present in the slice header and indicate the starting points of all tiles in the picture. The decoder decodes the starting points to enable the division of the coded picture into coded tiles so that they can be distributed for parallel decoding. In HEVC, it is mandatory to include the bitstream offsets in the slice header when tiles are enabled. However, the combination of tiles and slices is restricted in HEVC. Specifically, all CTUs in a tile belong to the same slice, or all CTUs belong to the same tile.

[0059] The bitstream offset can also allow tiles to be extracted and stitched together to reconstruct the tiles into an output stream. This requires some encoder-side constraints to make the tiles independent in time. One constraint restricts the motion vectors so that motion compensation for the tiles uses only samples included in the spatially co-located tiles of the previous picture. Another constraint restricts the temporal motion vector prediction (TMVP) so that the process is performed independently in time. In order to be completely independent, deblocking of the boundaries between tiles must be disabled via the loop_filter_across_tiles_enabled_flag described previously. However, disabling deblocking introduces visible lines between tiles. Therefore, some implementations disable deblocking while others do not. Motion Constrained Tile Sets (MCTS) are a feature in HEVC for signaling encoding-side constraints on sets of temporally independent tiles. A set of tiles in MCTS covers one or more tiles of a picture.

[0060] Tiles are sometimes used for 360-degree videos that are intended to be consumed using a head-mounted display (HMD) device. The field of view when using today's HMD devices is limited to about 20% of the entire sphere. This means that only 20% of the entire 360-degree video is consumed by the user. Typically, the entire 360-degree video sphere is available to the HMD device, which then cuts out the portion to be rendered for the user. This portion (i.e., the portion of the sphere that the user sees) is called a viewport. Well-known resource optimization is to make the HMD device video system aware of the user's head movements and the direction they are looking in, so that fewer resources are spent processing video samples that are not rendered to the user. Resources can be, for example, bandwidth from a server to a client or the decoding capabilities of a device. For future HMD devices, whose fields of view will be larger than what is currently possible, non-uniform resource allocation will still be beneficial. Specifically, the human visual system requires higher image quality in the central visual area (approximately 18° horizontal view), while lower requirements for image quality in the surrounding area (approximately 120° or more for a comfortable horizontal view). Therefore, non-uniform resource allocation will help meet the needs of the human visual system by allocating more resources in the central visual area compared to the surrounding areas.

[0061] Optimizing resources for regions of interest (ROIs) is another use case for tiles. RoIs can be specified in the content or extracted through methods such as eye tracking. One way to use head movement to reduce the required resources is to use tiles. This method first encodes a video sequence multiple times using tiles. The tile partition structure is the same in all encodings; however, the video sequence is encoded at different video qualities. This produces at least one high-quality encoding for the video sequence and one low-quality encoding for the video sequence. This means that for each tile at a specific point in time, there is at least one high-quality tile representation and at least one low-quality tile representation. The difference between a high-quality tile and a low-quality tile can be that the high-quality tile is encoded at a higher bitrate than the low-quality tile, or the high-quality tile has a higher resolution than the low-quality tile.

[0062] FIG. 3A to FIG. 3B An example of a high-quality tile having a higher resolution than a low-quality tile is shown. Specifically, Figure 3A shows a picture that has just been decoded by the decoder. In this context, a picture that is displayed as output from a decoder (i.e., a decoded picture) is a picture that is displayed in the compressed domain. Figure 3A In the example of , all 8 tiles AH in the compressed domain have equal spatial size. Then, after the picture is decoded but before the picture is rendered, the tiles AH are scaled and spatially arranged. The picture that has been prepared for rendering (i.e., after scaling and rearrangement) is the picture to be displayed in the output domain. Figure 3B The output field seen in shows the picture when rendered or presented to the user.

[0063] like FIG. 3A to FIG. 3B As shown, tiles B, D, F, and H are high-quality tiles because they have a higher resolution in the output domain. However, tiles A, C, E, and G are low-resolution tiles because the scaling step reduces the actual resolution.

[0064] In addition to explaining how tiles can have different resolutions, FIG. 3A to FIG. 3B It also describes the compression domain ( Figure 3A ) do not need to be in the same order as they are in the output domain ( Figure 3B ). Given that tiles are independent in both space and time, the spatial placement of tiles in the compressed domain is not critical.

[0065] When tiles are enabled in HEVC, there are multiple factors that increase the bit cost. First, prediction across tiles is disabled, which means that motion vectors and intra modes are not predicted across tiles. The use of tiles also disables quantization parameter (QP) prediction and context selection. Second, CABAC is initialized for each tile, which means that CABAC adaptability is hampered. Third, the bitstream offset must be signaled for each tile. Fourth, the tile partitioning structure needs to be specified in the PPS. Finally, CABAC is refreshed after each tile and the encoded data must be byte-aligned.

[0066] Tiles are useful; however, some considerations need to be addressed. For example, in its current form, HEVC constrains tiles to ensure that they span the entire picture vertically and horizontally. However, this limits the flexibility of tiles. For example, Figure 4 An example of a tile structure that is not supported by current implementations of HEVC is shown. Specifically, Figure 4 The tile structure has tiles arranged in such a way that one side (ie, a "tile border") of a given tile touches sides or tile borders of multiple adjacent tiles.

[0067] In order to virtually support this tile structure, a partition structure can be specified in HEVC, such as Figure 5 However, Figure 5 The tile structure shown in contains more tile boundaries. This will hurt compression efficiency and cause unnecessary tiling artifacts. In addition, if the tiles are to be completely spatially independent, the current form of HEVC only allows loop filtering dependencies to be enabled or disabled for all tile boundaries of a picture. Therefore, traditional HEVC solutions will have unnecessary loop filtering restrictions because they do not support flexible tile partitioning.

[0068] The concept of "flexible tiles" addresses these issues by providing a flexible tile partitioning method. Specifically, each tile comprises a single rectangular area, and the width and height of each tile are signaled to the decoder in the bitstream. Upon receipt, the decoder is configured to decode the individual width and height values ​​for each tile from the bitstream. These methods provide the encoder with more freedom to partition the picture into tiles, as it means that the encoder is no longer limited to defining the number of tile rows and tile columns as in HEVC.

[0069] Table 2 shows a possible tile syntax for expressing flexible tiles.

[0070]

[0071]

[0072] Table 2 - Flexible Tile Examples

[0073] Semantics of the syntax elements in Table 2

[0074] number_of_tiles_in_picture_minus2: specifies the number of tiles in the picture;

[0075] subtile_width_minus1: specifies the width of the subtile unit in coding tree units;

[0076] subtile_height_minus1: specifies the height of the subtile unit in coding tree units;

[0077] use_previous_tile_size_flag: When this flag is equal to 1, it specifies that the size of the current tile is equal to the size of the previous tile. However, when equal to 0, it specifies that the size of the current tile is not equal to the size of the previous tile. When this flag is not present, the value of use_previous_tile_size_flag is inferred to be equal to 0;

[0078] tile_width_minus1[i] plus 1: specifies the width of the i-th tile in sub-tile units; and

[0079] tile_height_minus1[i] plus 1: specifies the height of the i-th tile in sub-tile units.

[0080] Stereoscopic video (which has been around for a long time) is usually packed into one frame using side-by-side packing or top-and-bottom packing, respectively. FIG. 6A to FIG. 6B With stereoscopic video, each eye receives a separate view of the image, with the viewpoint of the second view slightly offset from the first view (e.g., the viewpoint for the user's left eye is slightly offset when compared to the viewpoint for the user's right eye).

[0081] There are some disadvantages to using stereoscopic video. Specifically, when the user is moving, the immersive experience is reduced.

[0082] In order to be able to at least partially look around an object, i.e., shift the viewport slightly, a combination of texture and depth information can be used, where other views can be synthesized based on the texture and depth information. Texture plus depth is also usually frame packed into one picture, side by side, top and bottom, or in some other way (see, for example, Fig.17 ).

[0083] MPEG is currently working on several activities for immersive video, which will be published in the MPEG-I standard set. One activity involves 3 degrees of freedom (3DoF), also known as 360° video. In these activities, users can look in all directions along a sphere using a head-mounted display (HMD). With stereoscopic video, the viewpoint is fixed.

[0084] Another activity involves extended 3 degrees of freedom video (3DoF+). With 3DoF+, users can experience the entire sphere as in a 360° video, but have the freedom to look around objects to some extent by moving their heads slightly. To achieve this technically, a 3DoF+ scene is built from a large number of views that contain both texture and depth information. Intermediate views are synthesized using texture and depth from adjacent views.

[0085] MPEG also has activities involving six degrees of freedom (6DoF) video. In 6DoF video, the user has full freedom to look around the object with a larger volume than 3DoF+. In some cases, the user can not only stand, but also walk around. One approach is to implement 6DoF video using a combination of background video and point cloud objects. Point cloud objects are described using geometric information, textures, and attributes attached to each point in the point cloud. Attributes can include, but are not limited to, color values ​​(e.g., RGB textures), reflectivity, occupancy, and opacity, which can be represented as texture maps, reflectivity maps, occupancy maps, and opacity maps.

[0086] exist FIG. 7A to FIG. 7C Use cases for 3DoF, 3DoF+, and 6DoF are shown in FIG. 3DoF, 3DoF+, and 6DoF can all support stereoscopic viewing; however, for each of these technologies, some kind of frame packing may occur.

[0087] Although flexible tiles increase the flexibility of tiles compared to HEVC, it does not address how to efficiently group video data in a bitstream. For example, in many video services including immersive video, part of the video data can be extracted from the video. Grouping the video data into chunks instead of dispersing them in different locations in the bitstream speeds up extraction and decoding. Simple cases, such as dividing a picture into two parts (e.g., stereoscopic pictures with side-by-side frame packing), can be solved using tiles and slices in HEVC. However, for more complex cases related to 3DoF video, 3DoF+ video, and 6DoF video, this is not possible using the tiles and slices currently defined in HEVC.

[0088] However, embodiments of the present disclosure solve these problems by introducing "tile groups" into a video codec configured to process images divided into tiles. Specifically, each tile is assigned to a tile group using a tile group ID. The tile group ID can be used to identify which area of ​​the picture each tile belongs to. It is not necessary for all parts of the area to be spatially adjacent. For example, a tile group can be a cube face in a cube map, two views in a frame-packed stereo video, or a texture and depth field in a frame-packed texture plus depth video. In addition, the tiles within each tile group can be scanned in a predefined order (e.g., a raster scan order). This allows each tile group to be extracted separately as a group block, thereby eliminating the need to assemble several tiles to construct the required part of the video. For tile groups, they can also be scanned in any required or desired order (e.g., a raster scan order) or in the order of their tile group IDs. Other scanning orders are also possible.

[0089] Embodiments of the present disclosure provide benefits and advantages not provided by conventional encoding and decoding methods. For example, embodiments of the present disclosure provide a mapping between tiles of a picture and logical groupings of tiles in the picture. This is useful for frame-packed videos, such as stereoscopic videos, texture plus depth videos, 3DoF videos, 3DoF+ videos, and 6DoF videos.

[0090] Another advantage provided by this embodiment comes from the fact that tiles that are related to each other (e.g., in terms of content type or scene geometry) are grouped into tile groups in the bitstream. Therefore, these tile groups can be extracted as one chunk. This is in contrast to tiles in HEVC, where such related images can be scattered across the bitstream. Therefore, grouping tiles as in this embodiment will speed up the extraction of tile groups from the bitstream and improve latency.

[0091] Additionally, tile groups can be sorted in the bitstream in order of importance (e.g., in a cubemap, the front-facing cube faces first). The most important tiles can then be more easily decoded and rendered first, further reducing perceived latency.

[0092] In addition, as previously described, each tile group has its own tile group ID. In some embodiments, the tile group ID can indicate how the tiles of a given video should be interpreted at the system level. For example, consider a stereoscopic picture with side-by-side frame packing. In this case, tiles with a tile group ID of "0" can be associated with the right view, and tiles with a tile group ID of "1" are associated with the left view. After such assignment, tiles can be processed according to the user's right view and left view. The tile group ID can also be used to assign corresponding labels to tile groups.

[0093] Another advantage is that the present embodiment allows certain properties to be defined for a given tile group. Such properties include, but are not limited to, properties that define motion constraints and loop filter constraints for each tile group, and properties that allow motion and loop filtering between tiles within a tile group but constrain motion and loop filtering between tile groups.

[0094] In a first embodiment, tiles are grouped into tile groups using tile group IDs. The tile group ID can be signaled, for example, in a PPS, SPS, or in a slice or tile header. For example, tiles belonging to the same tile group are encoded and decoded in a predetermined order (e.g., a raster scan order) within a tile group. Tiles are scanned continuously so that all tiles in a tile group are scanned in sequence. When scanning tiles in a tile group, tiles outside the tile group are not scanned. By continuously scanning tiles in a tile group, all tiles in a tile group can be decoded or extracted from a bitstream in a single group block without first extracting tiles from different parts of the bitstream and then assembling the tiles.

[0095] Can be used Figure 8 This can be seen in the simple case shown in . As shown in the figure, stereoscopic video is frame packed using a side-by-side format. The tiles TL associated with the first viewpoint are on the left ("L") side of the frame F, while the tiles TR associated with the second viewpoint are on the right ("R") side of the frame F. For reasons of parallel processing, each view is divided into several tiles.

[0096] FIG. 9A to FIG. 9B shows the order of tiles in the bitstream B. Specifically, Fig.9A The order of tiles T when HEVC tile syntax is used is shown. In this conventional case, tiles T are scanned in raster scan order. Therefore, every second tile TR in the bitstream B belongs to the right view R, and every other tile TL in the bitstream B belongs to the left view L. However, Fig. 9B The order of tiles T when they are grouped according to the present embodiment is shown. In these cases, each tile group is scanned one by one consecutively. Therefore, the tiles T of each tile group (i.e., the tiles in the first group (T1) of unshaded tiles and the tiles in the second group (T2) of shaded tiles) are consecutive in the bitstream.

[0097] In one embodiment, tile groups are encoded and decoded in the bitstream in the order of tile group IDs, for example, in ascending order.

[0098] In another embodiment, tile groups T1 and T2 are encoded and decoded in a raster scan order. In the context of the present disclosure, the "raster scan order" for tiles T within tile groups T1 and T2 is defined as a raster scan order for the upper left corner of the tiles in the tile groups. However, the raster scan order for tile groups T1 and T2 is defined as the following order: wherein, when the picture is scanned pixel by pixel (or block by block or CTU by CTU) in a raster scan order, tile groups T1 and T2 appear first. Alternatively, the raster scan order for tile groups T1 and T2 can also be defined as a raster scan order for the upper left corner of the first tile in each tile group T1 and T2.

[0099] In one embodiment, an order indicator (eg, a flag) is signaled to indicate whether tile groups are scanned in tile group ID order or in raster scan order.

[0100] Fig.10 A method 10 for decoding a bitstream B according to the present disclosure is shown. In this embodiment, a decoder receives a bitstream B including one or more encoded picture slice groups (e.g., "tile groups T1, T2") (box 12). Each picture slice group includes a group ID and one or more encoded picture slices (e.g., "tiles" T). Once received, the decoder extracts the first picture slice group from the bitstream as a single entity (box 14). As described above, this is different from conventional methods (e.g., HEVC), which first extract the picture slices in a given group from different parts of the received bitstream B before assembling the picture slices in the proper order. Therefore, the present embodiment enhances the operation of the decoder by speeding up the extraction of tile groups from the bitstream B and improving latency. Once the tile groups are extracted, the method 10 requires decoding the one or more picture slices in the first picture slice group in a predetermined order (box 16).

[0101] As previously mentioned, tiles T may be assigned to tile groups T1, T2 by the encoder using tile group IDs. However, such assignment is not always required. For example, in one embodiment, tiles T are assigned to tile groups T1, T2 without explicitly using tile group IDs. For example, when defining a tile structure, the first step would be to define tile groups T1, T2. Once defined, tiles T will be assigned to given tile groups T1, T2 without explicitly assigning any tile group IDs to tiles T.

[0102] Fig.11 A method 20 according to the present disclosure is shown for decoding a picture or a portion of a picture (having tiles grouped with tile group IDs) received in a bitstream B. One of ordinary skill in the art will appreciate that the method 20 is implemented at a decoder.

[0103] like Fig.11 As shown, method 20 starts with: the decoder parses the tile structure syntax of the picture from the received bitstream B, and the tile structure syntax includes the group ID of each tile T (box 22). For each tile T, the decoder determines the tile group T1, T2 to which the tile belongs based on the tile group ID (box 24). Then, the decoder selects the tile groups T1, T2 to be decoded (box 26). For example, in a common case, the decoder will decide to decode all tile groups T1, T2. Once determined, the decoder decodes the tiles T of the selected tile group from the bitstream B (box 28). More specifically, the tiles T of each tile group T1, T2 are scanned and decoded in a predefined order, and then decoded continuously from the bitstream B. In one aspect of this embodiment, the tile groups T1, T2 are also scanned and decoded in an explicit order of the tile group IDs of the tile groups T1, T2.

[0104] Fig.12 A method 30 implemented at a decoder is shown for extracting a portion of a picture having tiles T grouped with a tile group ID from a received bitstream B. Fig.12 As shown, the method 30 starts with: the decoder parses the tile structure syntax of the picture to obtain the group ID of each tile T (box 32). Then, for each tile T, the decoder determines the tile group T1, T2 to which the tile belongs based on the tile group ID (box 34). Then, the decoder selects the tile groups T1, T2 to be extracted from the received bitstream (box 36), and extracts the tiles T of the selected tile groups T1, T2 from the bitstream (box 38). The tiles T of each tile group T1, T2 scanned in a predefined order are grouped and placed consecutively to each other in the bitstream B. In one aspect of this embodiment, the tile groups T1, T2 are also scanned and extracted in an explicit order of the tile IDs of the tile groups T1, T2.

[0105] Fig.13 A method 40 implemented at an encoder is shown for encoding a picture having tiles T using tile group IDs into a bitstream B. Fig.13As shown, method 40 begins with the encoder defining a tile structure (box 42). In this embodiment, the encoder divides the picture into tile groups T1, T2 and assigns a unique tile group ID to each tile group T1, T2. Then, for each tile group, the encoder divides the tile group into one or more tiles T and assigns the tile group ID to each tile in the tile group (box 44). Once the tiles T are assigned to tile groups T1, T2, the encoder encodes the tile structure of the picture into a bitstream B, the tile structure including the tile group ID of each tile T (box 46), and encodes the tiles into the bitstream (box 48). In this embodiment, the tiles of each tile group are scanned in a predefined order and continuously encoded into the bitstream. However, it will be readily understood by those skilled in the art that the present disclosure is not limited thereto. On the one hand, for example, the tile groups are scanned and encoded in an explicit order of the tile group IDs of the tile groups. After encoding in this way, the encoder sends the bitstream to the decoder for decoding (box 50).

[0106] FIG. 14A to FIG. 14D FIG. 1 shows a block grouping according to an embodiment. As shown in the figure, Fig.14A A typical cubemap for 360° video is shown. The cubemap is divided into six cube faces - left (L), front (F), right (R), bottom (BM), back (BK), and top (T). The cube faces are marked with solid lines. The dashed lines are tile boundaries. One way to save bandwidth for 360° video for a head mounted display (HMD) is to divide the picture into several tiles. This allows only those tiles in the current viewpoint of the HMD to be sent to the HMD. In this example, each cube face L, F, R, BM, BK, and T is divided into 16 tiles.

[0107] Fig. 14B The scan order of tiles is shown, which in this case is the raster scan order used for tiles in HEVC. To extract a complete cube face in the example, four "tile chunks" always need to be assembled. As defined herein, a "tile chunk" comprises a set of contiguous tiles in the bitstream.

[0108] Fig. 14C The scanning order using the tile group ID in the raster scan order is shown. Specifically, each cube face L, F, R, BM, BK and T corresponds to a respective tile group. The unique tile group ID of each tile group (i.e., cube face) is assigned to the tiles in the tile group. After such assignment, each cube face L, F, R, BM, BK and T can be decoded or extracted as only one tile group block.

[0109] Fig.14DThe scanning order using tile group ID is shown. Specifically, the scanning order is achieved by increasing the tile group ID, where each tile group is assigned a unique tile group ID (i.e., 0, 1, 2, 3, 4, 5) and scanned in a raster scan order. As shown in the figure, the tile group IDs have been assigned differently to the cube faces, and the cube faces are scanned in the order of the tile group IDs. Scanning the tile groups in the order of tile group IDs allows the tiles to be scanned in order of importance. This may be important for low latency applications (such as 360° video).

[0110] exist Fig.15 Another example of tile grouping is depicted in . As described above, tile group boundaries are marked with solid lines and tile boundaries are marked with dashed lines. As shown in this embodiment, the encoder packs the same cube map of two different resolutions into one frame. Each cube face L, F, R, BM, BK and T of the high-resolution cube map is a tile group, while the low-resolution cube map can be divided into two tile groups - the first tile group is the cube face (L, F and R) and the second tile group is the cube face (BM, BK and T).

[0111] exist Fig.15 In an embodiment of the present invention, the encoder packs frames at multiple resolutions and packs multiple cube faces into corresponding tile groups. However, it should be understood by those skilled in the art that the present disclosure is not limited to this. On the one hand, for each resolution, the encoder only uses a single tile group.

[0112] In another embodiment, the encoder divides the picture P into tile groups based on the fragmented rectangular frame packing arrangement SEI message defined in Appendix D of HEVC, which states: "Component frame 0 is obtained by cropping region R0 from the decoded frame, and component frame 1 is obtained by vertically stacking regions R2 and R3 and placing the resulting rectangle to the right of region R1. Region R4 is not part of either of the two component frames and is therefore discarded. Therefore, in this aspect, the encoder of the present disclosure is configured to rearrange the frame packing arrangement to ensure that the pictures can be continuous in the bitstream sent to the decoder.

[0113] Fig.16This embodiment is shown, in which region R0 becomes a picture frame F1, and regions R1, R2 and R3 become another picture frame F2. If each region R0, R1, R2, R3 and R4 in frames F1 and F2 is a corresponding tile group, the encoder can encode each region and place it in the bitstream so that the two constituent picture frames F1 and F2 will be continuous in the bitstream in HEVC. However, there is not always a one-to-one correspondence between regions and tile groups. For example, if each region R0, R1, R2, R3 and R4 includes more than one tile at the height H of the region, the parts of the constituent picture frames F1 and F2 will not be continuous in the bitstream. Therefore, by using tile grouping according to this embodiment, the encoder can place the two constituent picture frames F1 and F2 continuously in the bitstream for frame-encapsulated pictures.

[0114] An example of a tile group ID implemented on top of HEVC tiles is shown in Table 3.

[0115]

[0116] Table 3 - HEVC tile syntax using tile group ID

[0117] Semantics of the syntax elements in Table 3

[0118] tile_group_ids_enabled_flag: When equal to 1, this flag specifies that the tile group ID is signaled for each tile. When equal to 0, this flag specifies that each tile belongs to the same tile group and tile_group_id[i] is inferred to be 0.

[0119] tile_group_id[i]: This flag specifies the tile group ID of the tile with index i. tile_group_id[i] should be between 0 and maxTileGroupID.

[0120] Table 4 shows an example of tile group ID for the flexible tile solution.

[0121]

[0122] Table 4 - Flexible Tile Syntax Using Tile Group IDs

[0123] Semantics of the syntax elements in Table 4

[0124] tile_group_ids_enabled_flag: When this flag is equal to 1, it specifies that the tile group ID is signaled for each tile. When this flag is equal to 0, it specifies that each tile belongs to the same tile group, and tile_group_id[i] is inferred to be 0.

[0125] tile_group_id[i] specifies the tile group ID of the tile with index i. tile_group_id[i] should be between 0 and maxTileGroupID.

[0126] maxTileGroupID: This parameter is defined as a large constant value (e.g., 256) for flexibility reasons. In some embodiments, this parameter can be defined as the number of tiles; however, defining this parameter as the number of tiles introduces limitations when using tile group IDs. Specifically, when defining this parameter as the number of tiles, no gaps are allowed in the range of tile group IDs used. This limitation may be problematic if tile groups are extracted from the bitstream.

[0127] The syntax in Tables 3 and 4 above describes how to assign a tile group ID to each tile by signaling the tile ID of each tile. However, in another version of this embodiment, the spatial structure of each tile group is first defined and signaled in the bitstream along with a unique tile group ID. Each tile group is then divided into tiles, with a descriptive syntax for this being signaled in the bitstream.

[0128] In one embodiment of the present disclosure, a label is assigned to each tile group ID. Such a label can be, for example, the picture type of a picture that has been frame-packed (e.g., the left view and the right view in a stereoscopic video), or the texture and depth portion of a video that has been texture-plus-depth packed. The label assignment to the tile group ID can be signaled in an SPS, in a PPS, in an SEI message, or in a system layer such as an ISOBMFF or OMAF box. The label can include descriptive text, a reference to the descriptive text, an integer, or a dynamic expression.

[0129] The following use case examples apply to both Embodiment 1 and Embodiment 2 discussed above.

[0130] Previously in Figure 8 An example of a stereoscopic picture with side-by-side frame packing is shown in FIG. 1 . To support parallel processing, each view is divided into several tiles. The first tile group is assigned a “left view” label L and the second tile group is assigned a “right view” label R. This labeling not only enables the decoder to directly extract tiles T belonging to only one of the views (e.g., Figure 8 TL and TR in the left view), and by grouping all tiles in the left view L with a first tile group whose tile group ID is 0, and grouping tiles in the right view R with a second tile group whose tile group ID is 1, the encoder can group tiles belonging to the same view together in the bitstream.

[0131] Fig.16An example of a common frame packing of texture plus depth is depicted. As shown in the figure, this embodiment of the present disclosure divides the picture frame into multiple tiles. The tile associated with the texture of the picture is located in the middle and is encoded as a tile belonging to tile group 0. Tile group 0 is assigned the label "texture". Above and below the texture portion, tiles associated with the depth map that have been spatially compressed and horizontally split can be seen, and these tiles are encoded as two tiles belonging to tile group 1. Tile group 1 is assigned the label "depth".

[0132] In 3DoF+ video projects in MPEG, in order to accurately represent the scene without occlusion artifacts, a maximum of 25 texture and depth maps are used. One approach is to send a central view at full resolution (5760×2880 resolution for each of the textures and depths) and send sparse representations of the other 24 views with reduced resolution but encoded with a higher QP than the central view. After decoding, the sparse representation is combined with the central view to form each of the other 24 views. Intermediate views between any of the 25 views are constructed using adjacent texture and depth maps.

[0133] Packing of views for the compressed domain may be done in one or more (eg, several) pictures. Fig.18 An example of how packing into one frame is done in 3DoF+ views is shown. Fig.18 As shown, each texture (in white) and depth map (in grey) belongs to a unique tile group. For spatial random access reasons, each tile group may include several tiles.

[0134] Fig.18 The tile group IDs of the tile groups are shown. Tile group 0 may be assigned the label "centralized texture" and tile group 1 may be assigned the label "centralized depth". The sparse views are numbered 0 to 23. Tile group 2 may be assigned the label "sparse texture 0" and tile group 3 the label "sparse depth 0", while tile groups 4 and 5 may be assigned the labels "sparse texture 1" and "sparse depth 1", and so on. If, for example, a 3DoF+ player wishes to render sparse views Fig. 22 and 23 , it requests the decoder to decode tile groups with tags “centralized texture”, “centralized depth”, “sparse texture 22”, “sparse depth 22”, “sparse texture 23”, and “sparse depth 23” (corresponding to tile groups with tile group IDs 0, 1, 46, 47, 48, and 49). All tiles in each tile group are fetched in full chunks and decoded before creating and rendering a synthetic view from the decoded texture and depth maps.

[0135] In MPEG's 6DoF video project, a complete VR experience can be created by adding a background video with a point cloud object, where the point cloud object is created using geometric information and point attributes, which can include color values ​​(e.g., RGB), reflectivity (I), and opacity. Typically for a background video, the point cloud geometry information and point attributes can be described and compressed as a video. The use of a common frame encapsulation ensures temporal alignment of frames and simplifies the decoding process. The geometry information, color values, reflectivity, and transparency of each background image and each point cloud object can be assigned to a unique tile group ID, which can be mapped to a descriptive label. This makes it easy to access specific parts of the scene. This is particularly useful if parts of the scene are divided into several tiles because it provides random access in space.

[0136] Tags can also be related to the aspect ratio of the video. An example is when the video was recorded with an aspect ratio that is wider than the device decoding the video can display. Typically, the full bitstream is sent to the receiving device, which decodes the bitstream and crops the video.

[0137] However, consider instead that the video bitstream is encoded using three different tile groups, such as Fig.19 As shown. The portion of tile group 1 and the portion of tile group 2 are not spatially adjacent. Tile group 0 will represent an aspect ratio of 4:3, the combination of tile groups 0 and 1 will represent a widescreen aspect ratio of 16:9, and the combination of tile groups 0, 1, and 2 will represent an anamorphic projection aspect ratio of 2.35:1, as shown in FIG. Fig. 20 shown.

[0138] In one embodiment, labels may be assigned to tile groups as follows:

[0139] Assign tile group ID 0 to the label "4:3 aspect ratio and wider";

[0140] · assign tile group ID 1 to the label "16:9 aspect ratio and wider"; and

[0141] • Assign tile group ID 2 to the label "2.35:1 aspect ratio".

[0142] In one aspect of this embodiment, a label is mapped to one or more tile group IDs. Then, in these aspects, the tile group IDs may be mapped to labels as follows:

[0143] "4:3 aspect ratio" is mapped to tile group ID 0;

[0144] "16:9 aspect ratio" is mapped to tile group IDs 0 and 1;

[0145] · "2.35:1 aspect ratio" maps to tile group IDs 0, 1, and 2.

[0146] If the sender knows that the receiving device can only display a specific aspect ratio, such as a 4:3 aspect ratio, the sender may choose to send only the video of that aspect ratio, i.e., the tile group with tile group ID 0. However, if the sender does not know the display capabilities of the receiving device, it may send the full bitstream. In these later cases, the receiving device may then decide to decode only the tile group with tile group ID 0 to save some processing power.

[0147] The previous embodiments describe grouping tiles in a single layer. However, it will be readily appreciated by those skilled in the art that the present disclosure is not limited thereto. In one embodiment of the present disclosure, multi-layer tile grouping is supported. In more detail, a tile group may include one or more tile subgroups, which in turn may include one or more tiles. Therefore, according to this embodiment, a tile group may be referred to as a tile group of a certain level. For example, a tile group of level 1 may include one or more tile subgroups of level 2, which may include one or more tiles.

[0148] The scanning order for one or more tile subgroups within a given tile group may be any scanning order required or desired, as long as the tile subgroups are placed consecutively in the bitstream. However, one embodiment of the present disclosure sequentially scans the tile subgroups within a given tile group before placing the tile subgroups consecutively in the bitstream.

[0149] As an example, you can Fig.18 A practical use case for exporting tile subgroups within a tile group is shown in the 3DoF+ example. A first level of the hierarchy (e.g., "level 1") may include a tile group assigned the label "texture" and a tile group assigned the label "depth". A second level of the hierarchy (e.g., "level 2") may include tile groups assigned the label "central view" and labels "sparse view 0" through "sparse view 1". Fig. 22 ". As an example, then fetching all textures would be fairly straight forward, since the decoder would only need to know to fetch the so-called "level 1" tile group labeled "texture".

[0150] exist Fig.15 Another practical use case for tile subgroups within tile groups can be seen in the cubemap example shown. Specifically, in this example, the encoder assigns those tile groups to "Level 1" based on resolution, and assigns the cube faces of the cubemap to tile subgroups and "Level 2" tile groups.

[0151] In another embodiment, a set of attributes is defined for each tile group. The attributes may define how to decode the tiles in the tile group. The attributes may also provide information about the tile group, such as the relative video quality of the tile group, etc. Each attribute may be defined separately for each tile group, or the same attribute may be defined for all tile groups in a picture. Tile group attributes may also be referred to as tile group characteristics or tile group parameters.

[0152] An example of a tile group attribute to be defined for each tile group in accordance with the present disclosure is a "delta_QP" value. In one embodiment, the delta_QP value may be defined as the difference between a reference quantization parameter (QP) value and the QP of the tile group. The reference QP value may be a signaled QP value for a parameter set, a picture, a slice, etc. The QP value for the tile group may be an initial QP value for the tile group, e.g., a QP value for a first block in a first tile in a tile group, or a QP value for predicting a QP value for a first block in a first tile in a tile group.

[0153] In another embodiment, the delta_QP value may be defined as the difference between the QP value of the previous tile group and the QP value of the current tile group. In one example, if a tile group in a picture has texture content and another tile group in the picture has depth content, delta_QP may be beneficially defined per tile group (i.e., on a per tile group basis), thereby supporting situations where different tile groups may wish to use different QP values. In such an embodiment, for example, a first tile group may be encoded using a high QP value, while a second tile group may be encoded using a low QP value.

[0154] Other examples of tile group properties are so-called "deblocking parameters" (e.g., deblocking filter strength). In HEVC, the strength of the deblocking filter can be adjusted by the encoder on a per-picture and / or per-slice basis. Furthermore, according to the present disclosure, deblocking parameters may be provided on a per-tile group basis, thus adapting the deblocking parameters to the content of each tile group.

[0155] In HEVC, similar to slice boundaries, tile boundaries break parsing and spatial prediction dependencies so that tiles can be processed independently. However, loop filters (e.g., deblocking and SAO filters) can still cross tile boundaries to optionally prevent tile boundary artifacts. This functionality is controlled by the loop_filter_across tile_enabled_flag syntax element in the PPS. Setting this function for each tile group is another example of a tile group attribute according to the present disclosure. Specifically, in a case where some of the tile groups in a picture are independent of each other in content while other tile groups are correlated, loop_filter_across tile_groups_enabled_flag can be set for each tile group. Therefore, the flag can be disabled for tile groups with independent content and enabled for tile groups with correlated content.

[0156] Similar to what MCTS does for tiles, another example of a tile group attribute is to signal whether inter prediction is allowed between tile groups. If inter prediction is allowed, inter prediction is allowed over tile boundaries within a tile group, but not over tile group boundaries. The same tile group attribute may be signaled for all tile groups, or the tile group attribute may be signaled separately for each tile group.

[0157] An example of tile group properties is shown in Table 5 below, which has syntax and semantics for tile group inter prediction and loop filtering on HEVC.

[0158]

[0159]

[0160] Table 5 - HEVC tile syntax using tile group ID and tile group attributes

[0161] loop_filter_across_tiles_enabled_flag: When this flag is equal to 1, it specifies that loop filtering operations may be performed across tile boundaries in pictures that reference the PPS. When this flag is equal to 0, it specifies that loop filtering operations are not performed across tile boundaries in pictures that reference the PPS. Loop filtering operations include deblocking filter and sample adaptive offset filter operations. When not present, the value of loop_filter_across_tiles_enabled_flag is inferred to be equal to 1.

[0162] tile_group_ids_enabled_flag: When this flag is equal to 1, it specifies that the tile group ID is signaled for each tile. When this flag is equal to 0, it specifies that each tile belongs to the same tile group, and tile_group_id[i] is inferred to be 0.

[0163] tile_group_id[i]: It specifies the tile group ID of the tile with index i. The valid value of tile_group_id[i] shall be between 0 and maxTileGroupID.

[0164] loop_filter_across_tile_groups_enabled_flag: When this flag is equal to 1, it specifies that loop filtering operations can be performed across tile group boundaries in pictures that reference the PPS. When this flag is equal to 0, it specifies that loop filtering operations are not performed across tile group boundaries in pictures that reference the PPS. The loop filtering operations include deblocking filter and sample adaptive offset filter operations. If loop_filter_across_tiles enabled_flag is equal to 0, loop_filter_across_tile_groups_enabled_flag is inferred to be 0. Otherwise, if not present, the value of loop_filter_across_tile_groups_enabled_flag is inferred to be equal to 1.

[0165] prediction_across_tile_groups_enabled_flag: When this flag is equal to 1, it specifies that sample values ​​outside each tile group or outside the picture may be referenced for inter prediction. When this flag is equal to 0, it specifies that no sample values ​​outside each tile group or outside the picture are referenced for inter prediction. In addition, motion vector candidates derived from blocks outside the current tile group may not be used for prediction. When not present, prediction_across_tile_groups_enabled_flag is inferred to be equal to 1.

[0166] According to one embodiment, loop_filter_across_tile_groups_enabled_flag is signaled if loop_filter_across_tiles_enabled_flag is set to 1. If loop filtering is already disabled for tile boundaries, loop filtering is automatically disabled for tile group boundaries. In these cases, it does not make sense to signal the value of loop_filter_across_tile_groups_enabled_flag.

[0167] As previously mentioned, one attribute that can be defined on a per-tile group basis is a delta_QP value. In one embodiment, for example, the decoder is configured to determine a reference QP value (refQP) from a sequence parameter set or a picture parameter set. The delta_QP value is then signaled for each tile group according to any of the foregoing embodiments.

[0168] Focusing on one tile group T as an example, the corresponding delta_QP value is considered to be deltaQP1. Optionally, there may be a deltaQP2 signaled for the first block of the first tile group T. Then, the delta_QP value to be used for the first block becomes refQP+deltaQP1+deltaQP2. If the block does not have deltaQP2 (e.g., which may occur when the first block does not include any non-zero transform coefficients), the QP value of the first block becomes refQP+deltaQP1.

[0169] The QP value assigned to the first block is used to scale the decoded transform coefficients. However, according to this embodiment, the QP value can also be used in the deblocking process of sample values ​​near the boundary between the first block and the adjacent blocks. According to this embodiment, the possibility of signaling a delta_QP value on a per tile group basis is beneficial because it allows the decoder to "stitch" tile groups originating from different bitstreams into one output bitstream. As an example, the first tile group T1 may have been encoded into the bitstream B1 using the QP value Q1, while the second tile group T2 may have been encoded into the bitstream B2 using the QP value Q2. If T1 and T2 are spliced ​​into one output bitstream without the possibility of setting the tile group QP, the QP values ​​of T1 and T2 may not be set correctly in the output stream. By making delta_QP part of the tile group properties and using one of the aforementioned embodiments, the correct QP values ​​can be set for all output tile groups.

[0170] Note that in HEVC, block delta_QP is signaled only for blocks that contain at least one non-zero transform coefficient. This means that if the first block in the first tile in tile groups T1 and T2 does not contain any non-zero coefficients, then if tile groups T1 and T2 are spliced ​​together into one picture, it may not be possible to assign the correct QP value to these two blocks unless a slice header is inserted.

[0171] The embodiments herein refer to encoders and decoders (e.g., Fig.21 The encoder 60 shown in FIG. Fig. 22The encoder 60 and the decoder 80 are described in detail with reference to the decoder 80 shown in FIG. The encoder 60 and the decoder 80 may be implemented using a suitable combination of hardware and / or software. Therefore, an encoder including one or more processors and, for example, one or more memories is configured to perform the method according to the embodiments of the present invention.

[0172] Fig.21 6 is a schematic diagram of an example encoder 60, which includes a communication interface 72, input / output circuits 68, a processor 62, one or more memories 64, and a computer program 66 stored in the memory 64. The processor 62 can be any combination of one or more of a suitable central processing unit (CPU), a multiprocessor, a microcontroller, a digital signal processor (DSP), an application specific integrated circuit (ASIC), etc. capable of executing software instructions contained in the computer program 66 stored in the one or more memories 64. The one or more memories 64 are, for example, data buffers, read-write memories (RWM), read-only memories (ROM), random access memories (RAM), cache memories, flash memory devices, optical storage devices, etc. The communication interface 72 may include input and / or output circuits 68 configured to transmit a bit stream, for example, via a transmitter or transceiver antenna 70.

[0173] Fig.21 is a schematic diagram of some components of an example decoder 80, which includes one or more processors and, for example, one or more memories. Fig. 22 The decoder 80 shown in FIG. 8 is configured to perform a decoding process according to embodiments herein.

[0174] Specifically, Fig. 22 As shown, the decoder 80 includes a processor 82, a communication interface 88, an input / output circuit 90, one or more memories 84, and a computer program 86. The processor 82 can be any combination of one or more of a suitable central processing unit (CPU), a multiprocessor, a microcontroller, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), etc. that can execute software instructions contained in the computer program 86 stored in the one or more memories 84. The one or more memories 84 are, for example, data buffers, read-write (RWM) memories, or read-only (ROM) memories. The decoder 80 includes a communication interface 88 having an input and / or output circuit 90, which is configured to receive a bit stream, for example, via a receiver or a transceiver.

[0175] like Fig.23As shown, the encoder 60 may have units, modules or circuits configured to perform the methods described herein. For example, the encoder 60 includes a picture structure definition unit / module 100, which is configured to define the structure of the picture segment unit as described herein. Specifically, the picture structure definition unit / module 100 divides the picture into multiple picture segment groups and assigns a unique group ID to each picture segment group. For each picture segment group, the group ID assignment unit / module 102 divides the picture segment group into one or more tiles and assigns the group ID of the picture segment group to each picture segment in the picture segment group. The encoding unit / module 104 encodes the picture segments of each picture segment group into a bitstream, and the transmission unit / module 106 sends the encoded bitstream to a decoder (e.g., decoder 80).

[0176] like Fig.24 As shown, the decoder 80 may also have units, modules or circuits configured to perform the methods described herein. For example, in this embodiment, the decoder 80 includes a receiving unit / module 110, which is configured to receive a bitstream as described herein. Specifically, the bitstream includes one or more coded picture segment groups, each picture segment group including a group ID and one or more coded picture segments. The extraction unit / module 112 extracts the first picture segment group from the bitstream as a single entity, and the decoding unit / module 114 decodes the one or more picture segments in the first picture segment group in a predetermined order, as described above.

[0177] Fig.25 An example system or method 120 is shown in which an encoder 60 and / or decoder 60 configured according to the embodiments described herein would be useful. The system 120 includes a media source 122 (e.g., a capture, renderer, or file). The media source 122 is fed to a projection map 124 and then fed to a tile encoder or transcoder (e.g., encoder 60) configured as previously described herein. The encoded information (e.g., a bitstream) can be stored in a memory 126 and then fed to a server 128. It can be sent to a viewing device 132 via a network 130. For example, the image is one of a video, and the viewing device 132 has a decoder 80 configured as described herein for decoding the bitstream, rendering 134 the decoded picture, and displaying the image on a display 140 for viewing by a user of the viewing device 132. Such a viewing device can be, for example, a head mounted display (HMD) device or any display device.

[0178] Some embodiments contemplated herein are described more fully with reference to the accompanying drawings. However, other embodiments are included within the scope of the subject matter disclosed herein. The disclosed subject matter should not be construed as being limited to the embodiments set forth herein; rather, these embodiments are provided as examples to convey the scope of the subject matter to those skilled in the art.

Claims

1. A method (10) for decoding a picture, the method comprising: Receiving (12) a bitstream (B) comprising two or more coded tile groups, wherein each tile group comprises a group ID and one or more coded tiles (T); obtaining an indicator from the bitstream, the indicator indicating whether a loop filtering operation can be performed across a tile group boundary; Obtaining (14) a first tile group from the bitstream as a single entity; and In response to the indicator having a first value, decoding one or more tiles in the first tile group in a predetermined order, the decoding comprising performing a loop filtering operation across a tile group boundary; or In response to the indicator having a second value, decoding one or more tiles in the first tile group in a predetermined order without performing loop filtering operations across tile group boundaries, The method further comprises: Before obtaining the indicator, obtaining a second indicator from the bitstream, wherein the second indicator indicates whether a loop filtering operation can be performed across a tile boundary, In response to the second indicator indicating that loop filtering operations cannot be performed across tile boundaries, it is determined that loop filtering operations are also disabled across tile group boundaries.

2. The method according to claim 1, wherein: The indicator is a flag.

3. The method according to claim 1 or 2, wherein: The indicator is signaled for each tile group.

4. The method according to claim 3, wherein: One or more of the tile groups in the picture are related to content of other tile groups in the picture.

5. The method according to claim 1 or 2, wherein: The loop filtering operation includes at least one of a deblocking filtering operation and a sample adaptive offset filtering operation.

6. A decoder comprising: processor; as well as A memory storing executable instructions, which, when executed by the processor, cause the decoder to: Receiving (12) a bitstream (B) comprising two or more coded tile groups, wherein each tile group comprises a group ID and one or more coded tiles (T); obtaining an indicator from the bitstream, the indicator indicating whether a loop filtering operation can be performed across a tile group boundary; Obtaining (14) a first tile group from the bitstream as a single entity; and In response to the indicator having a first value, decoding one or more tiles in the first tile group in a predetermined order, the decoding comprising performing a loop filtering operation across a tile group boundary; or In response to the indicator having a second value, decoding one or more tiles in the first tile group in a predetermined order without performing loop filtering operations across tile group boundaries, Before obtaining the indicator, a second indicator is further obtained from the bitstream, wherein the second indicator indicates whether a loop filtering operation can be performed across a tile boundary. In response to the second indicator indicating that loop filtering operations cannot be performed across tile boundaries, it is determined that loop filtering operations are also disabled across tile group boundaries.

7. A non-transitory computer-readable storage medium (84) containing a computer program (86), the computer program (86) comprising executable instructions which, when executed by processing circuitry (82) in a decoder (80), cause the decoder to perform the method according to any one of claims 1 to 5.

Citation Information

Patent Citations

  • Motion-constrained tile set for region of interest coding

    CN105432082A

  • Level definitions for multi-layer video codecs

    CN106105209A

  • Video coding and decoding method, and corresponding signal

    CN1555654A

  • Loop filtering control over tile boundaries

    US20130107973A1