Video encoder, video decoder and corresponding method

Abstract

To provide a video coding mechanism.SOLUTION: A mechanism includes receiving a sub-bitstream including: a sub-picture of a picture partitioned into a plurality of slices including a first slice, a parameter set associated with the picture and the sub-picture, and a slice header associated with the first slice. The parameter set is parsed to obtain an identifier and a length of a slice address of the first slice. A slice address for the first slice is determined from the slice header based on the identifier and the length of the slice address. The sub-bitstream is decoded to create a video sequence of sub-pictures including the first slice. The video sequence of sub-pictures is forwarded for display.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] TECHNICAL FIELD This disclosure relates generally to video coding, and more particularly to address management when extracting sub-pictures from pictures in video coding. [Background technology]

[0002] The amount of video data required to represent even a relatively short video can be substantial, which can cause difficulties when the data is to be streamed or otherwise communicated across communication networks with limited bandwidth capacity. Thus, video data is typically compressed before being communicated across modern telecommunication networks. Since memory resources may be limited, the size of the video can also be an issue when the video is stored on a storage device. Video compression devices often use software and / or hardware at the source to code the video data before transmission or storage, thereby reducing the amount of data required to represent a digital video image. The compressed data is then received at the destination by a video decompression device that decodes the video data. Due to limited network resources and an ever-increasing demand for higher video quality, improved compression and decompression techniques that improve compression ratios with little or no sacrifice in picture quality are desirable. Summary of the Invention

[0003] In an embodiment, the disclosure includes a method implemented in a decoder, comprising: receiving, by a receiver of the decoder, a sub-bitstream including a sub-picture of a picture partitioned into a plurality of slices including a first slice, a parameter set associated with the picture and the sub-picture, and a slice header associated with the first slice; parsing, by a processor of the decoder, the parameter set to obtain an identifier and a length of a slice address of the first slice; determining, by the processor, a slice address of the first slice from the slice header based on the identifier and the length of the slice address; decoding, by the processor, the sub-bitstream to generate a video sequence of the sub-picture including the first slice; and forwarding, by the processor, the video sequence of the sub-picture for display. In some video coding systems, slices (also known as tile groups) may be addressed based on a set of indexes. Such indexes may increase in a raster scan order starting with index zero at the top left corner of the picture and ending with index N at the bottom right corner of the picture, where N is the number of indexes minus one. Such a system works well for most applications. However, certain applications, such as virtual reality (VR), only render sub-pictures of a picture. Some systems may improve coding efficiency when streaming VR content by transmitting only a sub-bitstream of a bitstream to a decoder if the sub-bitstream contains a sub-picture to be rendered. In such cases, an index-based addressing scheme may not work correctly because the top-left corner of the sub-picture received by the decoder is generally some index other than zero. To address such concerns, an encoder (or an associated slicer) may be required to rewrite each slice header to change the sub-picture index so that the top-left index starts at zero and the remaining sub-picture slices are adjusted accordingly.Dynamically rewriting slice headers (e.g., per user request) can be very processor intensive. The disclosed system employs an addressing scheme that allows extracting sub-bitstreams, including sub-pictures, without the slice headers needing to be rewritten. Each slice is addressed based on an identifier (ID) other than an index (e.g., a sub-picture ID). In this way, the decoder can consistently determine all relevant addresses regardless of which sub-picture is received and regardless of the location of the received sub-picture relative to the top-left corner of the complete picture. The ID is encoded in a variable length field, since the ID is defined arbitrarily (e.g., selected by the encoder). Thus, the length of the slice address is also signaled. The ID associated with the sub-picture is also signaled. The length is used to interpret the slice address, and the sub-picture ID is used to map the slice address from a picture-based location to a sub-picture-based location. By employing these mechanisms, the encoder, decoder, and / or associated slices may be improved. For example, a sub-bitstream may be extracted and transmitted instead of the entire bitstream, which reduces the utilization of network, memory, and / or processing resources. Moreover, such extraction of a sub-bitstream can be performed without rewriting each slice header per user request, which further reduces the utilization of network, memory, and / or processing resources.

[0004] Optionally, in any of the above aspects, other implementations of the aspect provide that an identifier is associated with the sub-picture.

[0005] Optionally, in any of the above aspects, another implementation of the aspect provides that the length of the slice address indicates the number of bits contained in the slice address.

[0006] Optionally, in any of the above aspects, another implementation of the aspect provides that determining a slice address of the first slice includes using, by the processor, a length from the parameter set to determine a bit boundary for interpreting the slice address from the slice header, and using, by the processor, the identifier and the slice address to map the slice address from a picture-based location to a sub-picture-based location.

[0007] Optionally, in any of the above aspects, another implementation of the aspect provides that the aspect further includes parsing, by the processor, the parameter set to obtain an identifier (ID) flag, the ID flag indicating that mapping is available for mapping slice addresses from picture-based locations to sub-picture-based locations.

[0008] Optionally, in any of the above aspects, another implementation of the aspect provides that the mapping between picture-based positions and sub-picture-based positions aligns the slice header to the sub-picture without the slice header having to be rewritten.

[0009] Optionally, in any of the above aspects, another implementation of the aspect provides that the slice address has a defined value and does not have an index.

[0010] In an embodiment, the disclosure includes a method implemented in an encoder, comprising: encoding, by a processor of the encoder, a picture into a bitstream, the picture having a plurality of slices including a first slice; encoding, by the processor, into the bitstream, a slice header including a slice address of the first slice; encoding, by the processor, into the bitstream, a parameter set including an identifier of the first slice and a length of the slice address; extracting, by the processor, a sub-bitstream of the bitstream by extracting the first slice based on the slice address, the length of the slice address, and the identifier of the first slice without rewriting the slice header; and storing, in a memory of the encoder, the sub-bitstream for communication to a decoder. In some video coding systems, slices (also known as tile groups) may be addressed based on a set of indexes. Such indexes may increase in a raster scan order starting with index zero at the top left corner of the picture and ending with index N at the bottom right corner of the picture, where N is the number of indexes minus one. Such a system works well for most applications. However, certain applications, such as virtual reality (VR), only render sub-pictures of a picture. Some systems may improve coding efficiency when streaming VR content by transmitting only a sub-bitstream of a bitstream to a decoder if the sub-bitstream contains a sub-picture to be rendered. In such cases, index-based addressing schemes may not work correctly because the top-left corner of the sub-picture received by the decoder is generally some index other than zero.To address such concerns, the encoder (or associated slicer) may be required to rewrite each slice header to change the sub-picture index so that the top-left index starts at zero and the remaining sub-picture slices are adjusted accordingly. Dynamically rewriting slice headers (e.g., per user request) may be very processor-intensive. The disclosed system employs an addressing scheme that allows for extracting sub-bitstreams that contain sub-pictures without the slice headers needing to be rewritten. Each slice is addressed based on an identifier (ID) other than an index (e.g., a sub-picture ID). In this way, the decoder can consistently determine all relevant addresses regardless of which sub-picture is received and regardless of the position of the received sub-picture relative to the top-left corner of the complete picture. The ID is encoded in a variable length field, since the ID is defined arbitrarily (e.g., selected by the encoder). Thus, the length of the slice address is also signaled. The ID associated with the sub-picture is also signaled. The length is used to interpret slice addresses, and the sub-picture ID is used to map slice addresses from picture-based locations to sub-picture-based locations. By employing these mechanisms, the encoder, decoder, and / or associated slices may be improved. For example, a sub-bitstream may be extracted and transmitted instead of the entire bitstream, which reduces the utilization of network, memory, and / or processing resources. Furthermore, such extraction of the sub-bitstream can be performed without rewriting each slice header for each user request, which further reduces the utilization of network, memory, and / or processing resources.

[0011] Optionally, in any of the above aspects, other implementations of the aspect provide that an identifier is associated with the sub-picture.

[0012] Optionally, in any of the above aspects, another implementation of the aspect provides that the length of the slice address indicates the number of bits contained in the slice address.

[0013] Optionally, in any of the above aspects, another implementation of the aspect provides that the length in the parameter set includes sufficient data to interpret a slice address from a slice header, and the identifier includes sufficient data to map the slice address from a picture-based location to a sub-picture-based location.

[0014] Optionally, in any of the above aspects, another implementation of the aspect provides that the aspect further comprises encoding, by the processor, an identifier (ID) flag in the parameter set indicating that mapping is available for mapping slice addresses from picture-based locations to sub-picture-based locations.

[0015] Optionally, in any of the above aspects, another implementation of the aspect provides that the slice address has a defined value and does not have an index.

[0016] Optionally, in any of the above aspects, another implementation of the aspect provides that extracting a sub-bitstream of the bitstream includes extracting a sub-picture of the picture, the sub-picture including the first slice, and the sub-bitstream having the sub-picture, a slice header, and a parameter set.

[0017] In an embodiment, the present disclosure includes a video coding device having a processor, a memory, a receiver coupled to the processor, and a transmitter coupled to the processor, wherein the processor, memory, receiver, and transmitter are configured to perform the method of any of the above aspects.

[0018] In an embodiment, the present disclosure includes a non-transitory computer-readable medium having a computer program product for use by a video coding device, the computer program product having computer-executable instructions stored on the non-transitory computer-readable medium such that, when executed by a processor, the computer program product causes the video coding device to perform a method of any of the above aspects.

[0019] In an embodiment, the present disclosure includes a decoder having a receiving means for receiving a sub-bitstream including a sub-picture of a picture partitioned into a plurality of slices including a first slice, a parameter set associated with the picture and the sub-picture, and a slice header associated with the first slice, a parsing means for parsing the parameter set to obtain an identifier and a length of a slice address of the first slice, a determining means for determining a slice address of the first slice from the slice header based on the identifier and the length of the slice address, a decoding means for decoding the sub-bitstream to generate a video sequence of the sub-picture including the first slice, and a transferring means for transferring the video sequence of the sub-picture for display.

[0020] Optionally, in any of the above aspects, a further implementation of the aspect provides that the decoder is further configured to perform the method of any of the above aspects.

[0021] In an embodiment, the present disclosure includes an encoder having an encoding means for encoding a picture into a bitstream, the picture having a plurality of slices including a first slice, encoding into the bitstream a slice header including a slice address of the first slice and encoding into the bitstream a parameter set including an identifier and a length of the slice address of the first slice, extracting means for extracting a sub-bitstream of the bitstream by extracting the first slice based on the slice address, the length of the slice address and the identifier of the first slice without rewriting the slice header, and storing means for storing the sub-bitstream for communication to a decoder.

[0022] Optionally, in any of the above aspects, a further implementation of the aspect provides that the encoder is further configured to perform the method of any of the above aspects.

[0023] For clarity, any one of the above embodiments may be combined with any one or more of the other above embodiments to produce new embodiments within the scope of the present disclosure.

[0024] These and other features will be more clearly understood from the following detailed description read in conjunction with the accompanying drawings and claims.

[0025] For a more complete understanding of the present disclosure, reference is now made to the following brief description, taken in conjunction with the accompanying drawings and detailed description, in which like reference numerals represent like parts. [Brief description of the drawings]

[0026] [Figure 1] 4 is a flow chart of an example of a method for coding a video signal. [Diagram 2] 1 is a schematic diagram of an example coding and decoding (codec) system for video coding. [Diagram 3]FIG. 1 is a schematic diagram illustrating an example of a video encoder. [Figure 4] FIG. 2 is a schematic diagram illustrating an example of a video decoder. [Diagram 5] FIG. 2 is a schematic diagram illustrating an example of a sub-bitstream extracted from a bitstream. [Figure 6] FIG. 2 is a schematic diagram illustrating an example of a picture partitioned for coding. [Figure 7] FIG. 2 is a schematic diagram illustrating an example of a sub-picture extracted from a picture. [Figure 8] 1 is a schematic diagram of an example video coding device. [Figure 9] 1 is a flow chart of an example method for encoding a picture bitstream to support extraction of a sub-picture sub-bitstream without rewriting slice headers by employing explicit address signaling. [Figure 10] 11 is a flow chart of an example method for decoding a sub-bitstream of a sub-picture by employing explicit address signaling. [Figure 11] 1 is a schematic diagram of an example system for transmitting sub-bitstreams of sub-pictures by employing explicit address signaling; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] Although exemplary implementations of one or more embodiments are provided below, it should first be understood that the disclosed systems and / or methods may be implemented using any number of technologies, whether currently known or in existence. The disclosure should in no way be limited to the exemplary implementations, drawings, and technologies described below, including the example designs and implementations shown and described herein, but may be modified within the full scope of the appended claims and their equivalents.

[0028] Various acronyms are used herein, including coding tree block (CTB), coding tree unit (CTU), coding unit (CU), coded video sequence (CVS), Joint Video Experts Team (JVET), motion constrained tile set (MCTS), maximum transfer unit (MTU), network abstraction layer (NAL), picture order count (POC), raw byte sequence payload (RBSP), sequence parameter set (SPS), versatile video coding (VVC), and working draft (WD).

[0029] Many video compression techniques may be used to reduce the size of a video file with minimal data loss. For example, video compression techniques may include performing spatial (e.g., intra-picture) prediction and / or temporal (e.g., inter-picture) prediction to reduce or eliminate data redundancy within a video sequence. For block-based video coding, video slices (e.g., video pictures or portions of video pictures) may be partitioned into video blocks, which may also be referred to as tree blocks, coding tree blocks (CTBs), coding tree units (CTUs), coding units (CUs), and / or coding nodes. Video blocks in intra-coded (I) slices of a picture are coded using spatial prediction with respect to reference samples in neighboring blocks in the same picture. Video blocks in inter-coded unidirectionally predictive (P) or bidirectionally predictive (B) slices of a picture may be coded by using spatial prediction with respect to reference pictures in neighboring blocks in the same picture or temporal prediction with respect to reference samples in other reference pictures. Pictures may be referred to as frames and / or images, and reference pictures may be referred to as reference frames and / or reference images. Spatial or temporal prediction results in a predictive block representing the image block. The residual data represents pixel differences between the original image block and the predictive block. Inter-coded blocks are therefore encoded according to a motion vector pointing to a block of reference samples forming the predictive block and the residual data indicating the differences between the coded block and the predictive block. Intra-coded blocks are encoded according to an intra-coding mode and the residual data. For further compression, the residual data may be transformed from the pixel domain to a transform domain. These result in residual transform coefficients, which may be quantized. The quantized transform coefficients may be initially arranged in a two-dimensional array. The quantized transform coefficients may be scanned to generate a one-dimensional vector of transform coefficients. Entropy coding may be applied to achieve even further compression.Such video compression techniques are described in further detail below.

[0030] To ensure that the encoded video can be accurately decoded, the video is encoded and decoded according to a corresponding video coding standard, including International Telecommunications Union (ITU) Standardization Sector (ITU-T) H.261, International Organization for Standardization / International Electrotechnical Commission (ISO / IEC) Moving Picture Experts Group (MPEG)-1 Part 2, Advanced Video Coding (AVC), also known as ITU-T H.262 or ISO / IEC MPEG-2 Part 2, ITU-T H.263, ISO / IEC MPEG-4 Part 2, ITU-T H.264 or ISO / IEC MPEG-4 Part 10, and High Efficiency Video Coding (HEVC), also known as ITU-T H.265 or MPEG-H Part 2. AVC includes extensions such as Scalable Video Coding (SVC), Multiview Video Coding (MVC) and Multiview Video Coding Plus Depth (MVC+D), as well as three-dimensional (3D) AVC (3D-AVC). HEVC includes extensions such as Scalable HEVC (SHVC), Multiview HEVC (MV-HEVC), and 3D HEVC (3D-HEVC). The Joint Video Experts Group (JVET) of ITU-T and ISO / IEC has begun developing a video coding standard called Versatile Video Coding (VVC). VVC is included in working drafts (WDs) including JVET-L1001.

[0031] To code a video image, the image is first partitioned and the partitions are coded into a bitstream. Various picture partitioning schemes are available. For example, an image may be partitioned into regular slices, dependent slices, tiles, and / or according to wavefront parallelism (WPP). For simplicity, HEVC restricts the encoder such that only regular slices, dependent slices, tiles, WPP, and combinations thereof can be used when partitioning slices into groups of CTBs for video coding. Such partitioning can be applied to support matching maximum transform unit (MTU) sizes, parallel processing, and reduced end-to-end delay. The MTU represents the maximum amount of data that can be transmitted in a single packet. If a packet payload exceeds the MTU, the payload is split into two packets through a process called fragmentation.

[0032] A regular slice, also simply referred to as a slice, is a partitioned portion of an image that can be reconstructed independently from other regular slices in the same picture, despite some interdependence due to loop filtering operations. Each regular slice is encapsulated in its own network abstraction layer (NAL) unit for transmission. Furthermore, intra-picture prediction (intra-sample prediction, motion information prediction, coding mode prediction) and entropy coding dependencies across slice boundaries may be disabled to support independent reconstruction. Such independent reconstruction supports parallelization. For example, parallelization based on regular slices uses minimal inter-processor or inter-core communication. However, because each regular slice is independent, each slice is associated with a separate slice header. The use of regular slices can incur significant coding overhead due to the bit cost of a slice header for each slice and due to the lack of prediction across slice boundaries. Furthermore, regular slices may be used to support MTU size matching requirements. Specifically, because regular slices are encapsulated in separate NAL units and may be coded independently, each regular slice must be smaller than the MTU in the MTU scheme to avoid splitting the slice into multiple packets. As such, the goals of parallelization and MTU size matching may place conflicting requirements on slice layout within a picture.

[0033] Dependent slices are similar to regular slices, but have shortened slice headers and allow partitioning on picture treeblock boundaries without interrupting intra-picture prediction. Dependent slices thus allow regular slices to be fragmented into multiple NAL units, which results in reduced end-to-end delay by allowing parts of a regular slice to be sent before encoding of the entire regular slice is completed.

[0034] A tile is a partitioned portion of an image made up of horizontal and vertical boundaries that make up the columns and rows of the tile. Tiles may be coded in raster scan order (right to left and top to bottom). The scan order of the CTBs is local within the tile. Thus, the CTBs in the first tile are coded in raster scan order before processing for the CTBs in the next tile. As with regular slices, tiles break the intra-picture prediction dependency in addition to the entropy decoding dependency. However, tiles may not be included in individual NAL units, and thus tiles may not be used for MTU size matching. Each tile can be processed by one processor / core, and inter-processor / inter-core communication used for intra-picture prediction between processing units that decode adjacent tiles may be limited to carrying a shared slice header (if the adjacent tiles are in the same slice) and performing sharing related to loop filtering of reconstructed samples and metadata. When more than one tile is included in a slice, the entry point byte offset for each tile other than the first entry point offset in the slice may be signaled in the slice header. For each slice and tile, at least one of the following conditions should be satisfied: 1) all coded tree blocks in a slice belong to the same tile, and 2) all coded tree blocks in a tile belong to the same slice.

[0035] In WPP, a picture is partitioned into a single row of CTBs. The entropy decoding and prediction mechanism may use data from CTBs in other rows. Parallel processing is enabled through parallel decoding of CTB rows. For example, the current row may be decoded in parallel with the previous row. However, the decoding of the current row is delayed from the decoding process of the previous row by two CTBs. This delay ensures that data for the CTB above the current CTB and the CTB to the right of the current CTB in the current row is available before the current CTB is coded. This approach appears as a wavefront when represented graphically. This staggered start allows parallelization with no more than the number of processors / cores as there are CTB rows in the picture. Since intra-picture prediction between adjacent treeblock rows in a picture is allowed, the amount of inter-processor / inter-core communication to enable intra-picture prediction can be significant. WPP partitioning does not take into account the NAL unit size. Therefore, WPP does not support MTU size matching. However, regular slicing may be used with WPP, with some coding overhead, to implement MTU size matching as desired.

[0036] A tile may also include a motion constrained tile set. A motion constrained tile set (MCTS) is a tile set designed such that associated motion vectors are restricted to point to full sample locations within the MCTS and fractional sample locations that require only full sample locations within the MCTS for interpolation. Furthermore, the use of motion vector candidates for temporal motion vector prediction derived from blocks outside the MCTS is not permitted. In this way, each MCTS can be decoded independently without the presence of tiles not included in the MCTS. A temporal MCTS supplemental enhancement information (SEI) message may be used to indicate the presence of an MCTS in a bitstream and to signal the MCTS. The MCTS SEI message provides supplemental information that can be used in MCTS sub-bitstream extraction (specified as part of the semantics of the SEI message) to generate a conforming bitstream for the MCTS set. The information includes multiple extraction information sets, each defining multiple MCTS sets, and including raw bytes sequence payload (RBSP) bytes of replacement video parameter sets (VPS), sequence parameter sets (SPS), and picture parameter sets (PPS) used during the MCTS sub-bitstream extraction process. When extracting a sub-bitstream according to the MCTS sub-bitstream extraction process, the parameter sets (VPS, SPS, and PPS) may be rewritten or replaced, and the slice header may be updated since one or all of the syntax elements related to slice addresses (including first_slice_segment_in_pic_flag and slice_segment_address) may use different values ​​in the extracted sub-bitstream.

[0037] The above scheme may include certain problems. In some systems, when there is more than one tile / slice in a picture, the address of the tile group may be signaled as an index in the tile group header by using a syntax element such as tile_group_address. The tile_group_address identifies the tile address of the first tile in the tile group. The length of the tile_group_address may be determined as Ceil(Log2(NumTilesInPic)) bits, where NumTilesInPic contains the number of tiles in the picture. The value of tile_group_address may be in the range of zero to NumTilesInPic-1, inclusive, and the value of tile_group_address may not be equal to the value of tile_group_address of any other coded tile group NAL unit of the same coded picture. The tile_group_address may be inferred to be zero when not present in the bitstream. The tile address described above contains a tile index. However, using the tile index as an address for each tile group may lead to some coding inefficiency.

[0038] For example, some use cases may require modification of AVC or HEVC slice segment headers between encoding and decoding, either at the client side just before passing the bitstream to the decoder, or at some network-based media processing entity. One example of such a use case is tiled streaming, where a panoramic video is encoded with HEVC tiles, but the decoder only decodes a portion of these tiles. By rewriting the HEVC slice segment header (SSH) along with the SPS / PPS, the bitstream can be manipulated to change the subset of tiles that are decoded and their spatial arrangement in the decoded video frame. One reason for this CPU processing cost is the fact that AVC and HEVC slice segment headers use variable length fields and have a byte alignment field at the end. This means that whenever a field in the SSH is modified, this may affect the byte alignment field at the end of the SSH, which is also rewritten in that case. And since all fields are encoded with variable length, the only way to know the location of the byte alignment field is to parse all the preceding fields. This incurs significant processing overhead, especially when tiles are used and one second's worth of video may contain hundreds of NALs. Some systems support explicit signaling of tile identifiers (IDs). However, some syntax elements may not be optimized and may contain unnecessary and / or redundant bits when signaled. Furthermore, some constraints associated with explicit tile ID signaling are not specified.

[0039] For example, the above mechanism allows a picture to be partitioned and compressed. For example, a picture may be partitioned into slices, tiles, and / or tile groups. In some examples, a tile group may be used synonymously with a slice. Such slices and / or tile groups may be addressed based on a set of indexes. Such indexes may increase in a raster scan order starting with index zero at the top left corner of the picture and ending with index N at the bottom right corner of the picture, where N is the number of indexes minus one. Such a system works well for most applications. However, certain applications, such as virtual reality (VR), only render sub-pictures of a picture. Such sub-pictures may be referred to as regions of interest in some circumstances. Some systems may improve coding efficiency when streaming VR content by transmitting only a sub-bitstream of a bitstream to a decoder if the sub-bitstream includes a sub-picture to be rendered. In such cases, an index-based addressing scheme may not work correctly because the top left corner of the sub-picture received by the decoder is generally some index other than zero. To address such concerns, the encoder (or associated slicer) may be required to rewrite each slice header to change the sub-picture index so that the top-left index starts at zero and the remaining sub-picture slices are adjusted accordingly. Dynamically rewriting slice headers (e.g., per user request) may be very processor intensive.

[0040] Various mechanisms are disclosed herein to improve coding efficiency and reduce processing overhead when extracting a sub-bitstream including a sub-picture from an encoded bitstream including a picture. The disclosed system employs an addressing scheme that allows extracting a sub-bitstream including a sub-picture without the slice header needing to be rewritten. Each slice / tile group is addressed based on an ID other than an index. For example, a slice may be addressed by a value that may be mapped to an index and stored in the slice header. This allows a decoder to read the slice address from the slice header and map the address from a picture-based location to a sub-picture-based location. In that the slice address is not a predefined index, the slice address is encoded in a variable length field. Thus, the length of the slice address is also signaled. The ID associated with the sub-picture is also signaled. The sub-picture ID and length may be signaled in the PPS. A flag may also be signaled in the PPS to indicate that an explicit addressing scheme is employed. By reading the flags, the decoder can obtain the length and the sub-picture ID. The length is used to interpret the slice address from the slice header. The sub-picture ID is used to map the slice address from a picture-based location to a sub-picture-based location. In this way, the decoder can consistently determine all relevant addresses regardless of which sub-picture is received and regardless of the location of the received sub-picture relative to the top-left corner of the complete picture. Furthermore, this mechanism allows such a determination to be made without rewriting the slice header to change the slice address value and / or without modifying the byte alignment field associated with the slice address. By employing the above mechanism, the encoder, decoder, and / or associated slices may be improved.For example, a sub-bitstream may be extracted and transmitted instead of the entire bitstream, which reduces the utilization of network, memory, and / or processing resources. Moreover, such extraction of a sub-bitstream can be performed without rewriting each slice header per user request, which further reduces the utilization of network, memory, and / or processing resources.

[0041] 1 is a flow chart of an exemplary operational method 100 of coding a video signal. Specifically, a video signal is encoded by an encoder. The encoding process compresses the video signal by using various mechanisms to reduce the video file size. The smaller file size allows the compressed video file to be transmitted towards a user while reducing the associated bandwidth overhead. A decoder then decodes the compressed video file to reconstruct the original video signal for display to an end user. The decoding process generally mirrors the encoding process to allow the decoder to consistently reconstruct the video signal.

[0042] At step 101, a video signal is input to the encoder. For example, the video signal may be an uncompressed video file stored in memory. As another example, the video file may be captured by a video capture device such as a video camera and encoded to support live streaming of video. The video file may include both audio and video components. The video component includes a series of image frames that, when viewed in succession, give the visual impression of movement. The frames include pixels represented in terms of light, referred to herein as luma components (or luma samples), and color, referred to herein as chroma components (or color samples). In some examples, the frames may also include depth values ​​to support three-dimensional displays.

[0043] At step 103, the video is partitioned into blocks. Partitioning involves subdividing pixels in each frame into square and / or rectangular blocks for compression. For example, in High Efficiency Video Coding (HEVC) (also known as H.265 and MPEG-H Part 2), a frame may first be divided into coding tree units (CTUs), which are blocks of a predefined size (e.g., 64x64 pixels). CTUs contain both luma and chroma samples. The coding tree may be used to divide the CTUs into blocks and then recursively subdivide the blocks until a configuration that supports further encoding is achieved. For example, the luma component of a frame may be subdivided until each block contains relatively uniform lightness and darkness values. Furthermore, the chroma component of a frame may be subdivided until each block contains relatively uniform color values. Thus, the partitioning mechanism varies depending on the content of the video frame.

[0044] At step 105, various compression mechanisms are used to compress the image blocks partitioned at step 103. For example, inter-prediction and / or intra-prediction may be used. Inter-prediction is designed to take advantage of the fact that objects in a common scene tend to appear in consecutive frames. Thus, blocks representing objects in a reference frame do not need to be repeatedly described in adjacent frames. In particular, objects such as tables may remain in a fixed position across multiple frames. Thus, the table may be described once and adjacent frames may refer back to the reference frame. Pattern matching mechanisms may be used to match objects across multiple frames. Furthermore, moving objects may be represented across multiple frames, for example, due to object motion or camera motion. As a specific example, a video may show a car moving across the screen across multiple frames. Motion vectors may be used to describe such motion. A motion vector is a two-dimensional vector that provides an offset from the coordinates of an object in one frame to the coordinates of the object in the reference frame. As such, inter prediction may encode an image block in a current frame as a set of motion vectors that indicate its offset from a corresponding block in a reference frame.

[0045] Intra prediction encodes blocks within a common frame. Intra prediction exploits the fact that luma and chroma components tend to cluster together within a frame. For example, a patch of green in a part of a tree tends to be located adjacent to a similar patch of green. Intra prediction uses multiple directional prediction modes (e.g., 33 in HEVC), planar mode, and direct current (DC) mode. The directional modes indicate that the current block is similar / same as the samples of neighboring blocks in the corresponding direction. The planar mode indicates that a series of blocks along a row / column (e.g., a plane) can be interpolated based on the neighboring blocks at the end of the row. The planar mode effectively indicates a smooth transition of light / color across the row / column by using a relatively constant slope of changing values. The DC mode is used for boundary smoothing and indicates that the block is similar / same as the average value associated with the samples of all neighboring blocks associated with the angular direction of the directional prediction mode. Thus, an intra prediction block can represent the image block as various associated prediction mode values ​​instead of the actual value. Furthermore, the inter-prediction block may represent the image block as a motion vector instead of an actual value. In either case, the prediction block may not represent the image block exactly in some cases. Any differences are stored in the residual block. A transform may be applied to the residual block to further compress the file.

[0046] In step 107, various filtering techniques may be applied. In HEVC, filters are applied according to an in-loop filtering scheme. The block-based prediction described above may cause the generation of blocky images at the decoder. Furthermore, the block-based prediction scheme may encode a block and then reconstruct the encoded image for later use as a reference block. The in-loop filtering scheme iteratively applies a noise suppression filter, a deblocking filter, an adaptive loop filter, and a sample adaptive offset (SAO) filter to the block / frame. These filters mitigate such blocking artifacts so that the encoded file can be accurately reconstructed. Furthermore, these filters mitigate artifacts in the reconstructed reference block so that the artifacts are less likely to cause further artifacts in subsequent blocks that are encoded based on the reconstructed reference block.

[0047] Once the video signal has been partitioned, compressed, and filtered, the resulting data is encoded into a bitstream at step 109. The bitstream includes the above data in addition to any signaling data desired to support proper video signal reconstruction at the decoder. For example, such data may include partition data, prediction data, residual blocks, and various flags that provide coding instructions to the decoder. The bitstream may be stored in memory for transmission to a decoder upon request. The bitstream may also be broadcast and / or multicast to multiple decoders. The generation of the bitstream is an iterative process. Thus, steps 101, 103, 105, 107, and 109 may be performed sequentially and / or simultaneously across multiple frames and blocks. The order shown in FIG. 1 is presented for clarity and ease of discussion and is not intended to limit the video coding process to any particular order.

[0048] The decoder receives the bitstream and begins the decoding process at step 111. Specifically, the decoder uses an entropy decoding scheme to convert the bitstream into corresponding syntax and video data. The decoder uses syntax data from the bitstream to determine the partitions of the frame at step 111. The partitioning should match the result of the block partitioning at step 103. The entropy encoding / decoding used at step 111 will now be described. The encoder makes many choices during the compression process, such as selecting a block partitioning scheme from several possible choices based on the spatial location of values ​​in the input image. Signaling the exact choice may use a number of bins. As used herein, a bin is a binary value that is treated as a variable (e.g., a bit value that can change depending on the situation). Entropy coding allows the encoder to discard any choices that are clearly not feasible for a particular case while retaining a set of acceptable choices. Each acceptable choice is then assigned a codeword. The length of the codeword depends on the number of allowable choices (e.g., one bin for two choices, two bins for three to four choices, etc.). The encoder then encodes a codeword for the selected choice. This scheme reduces the size of the codeword in that it is as large as desired to uniquely indicate a choice from a small subset of allowable choices, as opposed to uniquely indicating a choice from a potentially large set of all possible choices. The decoder then decodes the choice by determining the set of allowable choices in a similar manner as the encoder. By determining the set of allowable choices, the decoder can read the codeword and determine the choice made by the encoder.

[0049] In step 113, the decoder performs block decoding. Specifically, the decoder uses an inverse transform to generate a residual block. The decoder then uses the residual block and a corresponding prediction block to reconstruct an image block according to the partitioning. The prediction block may include both intra-prediction blocks and inter-prediction blocks generated in the encoder in step 105. The reconstructed image block is then positioned within a frame of the reconstructed video signal according to the partitioning data determined in step 111. The syntax for step 113 may also be signaled in the bitstream via entropy coding, as described above.

[0050] At step 115, filtering is performed on the frames of the reconstructed video signal in a manner similar to step 107 at the encoder. For example, a noise suppression filter, a deblocking filter, an adaptive loop filter, and an SAO filter may be applied to the frames to remove blocking artifacts. Once the frames have been filtered, the video signal may be output to a display at step 117 for viewing by an end user.

[0051] 2 is a schematic diagram of an exemplary coding and decoding (codec) system 200 for video coding. Specifically, the codec system 200 provides functionality to support the implementation of the operational method 100. The codec system 200 is generalized to represent components used in both encoders and decoders. The codec system 200 receives and partitions a video signal, as described with respect to steps 101 and 103 of the operational method 100, resulting in a partitioned video signal 201. The codec system 200 then compresses the partitioned video signal 201 into a coded bitstream when operating as an encoder, as described with respect to steps 105, 107, and 109 of the method 100. When operating as a decoder, the codec system 200 generates an output video signal from the bitstream, as described with respect to steps 111, 113, 115, and 117 of the operational method 100. The codec system 200 includes a general coder control component 211, a transform scaling and quantization component 213, an intra-picture estimation component 215, an intra-picture prediction component 217, a motion compensation component 219, a motion estimation component 221, a scaling and inverse transform component 229, a filter control analysis component 227, an in-loop filter component 225, a decoding picture buffer component 223, and a header formatting and context adaptive binary arithmetic coding (CABAC) component 231. Such components are coupled as shown. In FIG. 2, the black lines indicate the movement of data to be encoded / decoded, while the dashed lines indicate the movement of control data that controls the operation of the other components. In an encoder, all of the components of the codec system 200 may be present. A decoder may include a subset of the components of the codec system 200.For example, the decoder may include an intra-picture prediction component 217, a motion compensation component 219, a scaling and inverse transform component 229, an in-loop filter component 225, and a decoding picture buffer component 223. These components will now be described.

[0052] The partitioned video signal 201 is a captured video sequence that has been partitioned into blocks of pixels by a coding tree. The coding tree uses various partitioning modes to subdivide the blocks of pixels into smaller blocks of pixels. These blocks may then be further subdivided into smaller blocks. The blocks may be referred to as nodes on the coding tree. Larger parent nodes are partitioned into smaller child nodes. The number of times a node is subdivided is referred to as the depth of the node / coding tree. The partitioned blocks may be included in a coding unit (CU) in some cases. For example, a CU may be a subpart of a CTU that includes a luma block, a red differential chroma (Cr) block, and a blue differential chroma (Cb) block, along with the corresponding syntax instructions for the CU. The partitioning modes may include a binary tree (BT), a ternary tree (TT), and a quad tree (QT) that are used to partition a node into two, three, or four child nodes, respectively, whose shape changes depending on the partitioning mode being used. The partitioned video signal 201 is forwarded to a general coder control component 211, a transform scaling and quantization component 213, an intra-picture estimation component 215, a filter control analysis component 227, and a motion estimation component 221 for compression.

[0053] The generic coder control component 211 is configured to make decisions regarding the coding of images of a video sequence into a bitstream according to applicable constraints. For example, the generic coder control component 211 manages the optimization of bitrate / bitstream size versus reconstruction quality. Such decisions may be based on storage space / bandwidth availability and image resolution requirements. The generic coder control component 211 also manages buffer utilization in light of transmission rate to mitigate buffer under-run and over-run issues. To manage these issues, the generic coder control component 211 manages partitioning, prediction, and filtering by other components. For example, the generic coder control component 211 may dynamically increase compression complexity to increase resolution and increase bandwidth utilization, or decrease compression complexity to decrease resolution and bandwidth utilization. Thus, the generic coder control component 211 controls other components of the codec system 200 to balance video signal reconstruction quality and bitrate concerns. The generic coder control component 211 generates control data that controls the operation of other components. The control data is also forwarded to the header formatting and CABAC component 231 to be encoded into the bitstream for signaling parameters for decoding at the decoder.

[0054] The partitioned video signal 201 is also sent to a motion estimation component 221 and a motion compensation component 219 for inter prediction. A frame or slice of the partitioned video signal 201 may be divided into multiple video blocks. The motion estimation component 221 and the motion compensation component 219 perform inter predictive coding of the received video blocks relative to one or more blocks in one or more reference frames to provide temporal prediction. The codec system 200 may perform multiple coding passes, for example, to select an appropriate coding mode for each block of video data.

[0055] The motion estimation component 221 and the motion compensation component 219 may be highly integrated, but are depicted separately for conceptual purposes. Motion estimation performed by the motion estimation component 221 is the process of generating motion vectors, thereby estimating the motion of video blocks. A motion vector may indicate, for example, the displacement of a coded object relative to a predictive block. A predictive block is a block that is found to closely match a block to be coded in terms of pixel differences. A predictive block is sometimes referred to as a reference block. Such pixel differences may be determined by sum of absolute difference (SAD), sum of square difference (SSD), or other difference metrics. HEVC uses several coded objects, including CTUs, coding tree blocks (CTBs), and CUs. For example, a CTU may be divided into CTBs, which may then be divided into CBs for inclusion in a CU. A CU may be encoded as a prediction unit (PU), which includes predictive data, and / or a transform unit (TU), which includes transformed residual data, for the CU. The motion estimation component 221 generates the motion vectors, PUs, and TUs by using the rate-distortion analysis as part of a rate-distortion optimization process. For example, the motion estimation component 221 may determine multiple reference blocks, multiple motion vectors, etc. for a current block / frame, and may select the reference block, motion vector, etc. that has the best rate-distortion characteristics. The best rate-distortion characteristics balance both the quality of the video reconstruction (e.g., the amount of data lost due to compression) and the coding efficiency (e.g., the size of the final encoding).

[0056] In some examples, the codec system 200 may calculate values ​​for sub-integer pixel locations of reference pictures stored in the decoding picture buffer component 223. For example, the video codec system 200 may interpolate values ​​for quarter-pixel, eighth-pixel, or other fractional pixel locations of the reference pictures. Thus, the motion estimation component 221 may perform motion searches for full pixel and fractional pixel locations and output motion vectors with fractional pixel accuracy. The motion estimation component 221 calculates motion vectors for PUs of video blocks in inter-coded slices by comparing the location of the PUs with the location of the predictive blocks of the reference pixels. The motion estimation component 221 outputs the calculated motion vectors as motion data to the header formatting and CABAC component 231 for encoding and as motion data to the motion compensation component 219.

[0057] The motion compensation performed by the motion compensation component 219 may include fetching or generating a prediction block based on a motion vector determined by the motion estimation component 221. As before, the motion estimation component 221 and the motion compensation component 219 may be functionally integrated in some examples. Upon receiving a motion vector for a PU of a current video block, the motion compensation component 219 may find the location of the prediction block to which the motion vector points. A residual video block is then formed by subtracting pixel values ​​of the prediction block from pixel values ​​of the current video block being coded to form pixel difference values. In general, the motion estimation component 221 performs motion estimation on the luma component, and the motion compensation component 219 uses the motion vector calculated based on the luma component for both the chroma and luma components. The prediction block and the residual block are forwarded to the transform scaling and quantization component 213.

[0058] The partitioned video signal 201 is also sent to an intra picture estimation component 215 and an intra picture prediction component 217. As with the motion estimation component 221 and the motion compensation component 219, the intra picture estimation component 215 and the intra picture prediction component 217 may be highly integrated, but are depicted separately for conceptual purposes. The intra picture estimation component 215 and the intra picture prediction component 217 intra predict a current block relative to blocks in a current frame, as an alternative to the inter prediction performed by the motion estimation component 221 and the motion compensation component 219 between frames as described above. In particular, the intra picture estimation component 215 determines an intra prediction mode to be used to encode the current block. In some examples, the intra picture estimation component 215 selects an appropriate intra prediction mode for encoding the current block from a plurality of tested intra prediction modes. The selected intra prediction mode is then forwarded to the header formatting and CABAC component 231 for encoding.

[0059] For example, the intra picture estimation component 215 calculates rate-distortion values ​​for various tested intra prediction modes using a rate-distortion analysis and selects the intra prediction mode having the best rate-distortion characteristics among the tested modes. The rate-distortion analysis generally determines the bit rate (e.g., number of bits) used to generate the encoded block as well as the amount of distortion (or error) between the encoded block and the original unencoded block that was encoded to generate the encoded block. The intra picture estimation component 215 calculates a ratio from the distortion and rate for the various encoded blocks to determine which intra prediction mode exhibits the best rate-distortion value for that block. Moreover, the intra picture estimation component 215 may be configured to code the depth block of the depth map using a depth modeling mode (DMM) based on rate-distortion optimization (RDO).

[0060] The intra-picture prediction component 217 may generate a residual block from the prediction block based on a selected intra-prediction mode determined by the intra-picture estimation component 215 if implemented in an encoder, or may read the residual block from the bitstream if implemented in a decoder. The residual block includes value differences between the prediction block and the original block, represented as a matrix. The residual block is then forwarded to the transform scaling and quantization component 213. The intra-picture estimation component 215 and the intra-picture prediction component 217 may operate on both luma and chroma components.

[0061] The transform scaling and quantization component 213 is configured to further compress the residual block. The transform scaling and quantization component 213 applies a transform, such as a discrete cosine transform (DCT), a discrete sine transform (DST), or a conceptually similar transform, to the residual block to generate a video block having residual transform coefficient values. A wavelet transform, an integer transform, a subband transform, or other types of transforms may also be used. The transform may convert the residual information from a pixel value domain to a transform domain, such as a frequency domain. The transform scaling and quantization component 213 is also configured to scale the transformed residual information, for example, based on frequency. Such scaling may include applying a scale factor to the residual information such that different frequency information is quantized with different granularity, which may affect the final visual quality of the reconstructed video. The transform scaling and quantization component 213 is also configured to quantize the transform coefficients to further reduce the bit rate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be changed by adjusting a quantization parameter. In some examples, the transform scaling and quantization component 213 may then perform a scan of a matrix containing the quantized transform coefficients. The quantized transform coefficients are forwarded to the header formatting and CABAC component 231 for encoding into the bitstream.

[0062] The scaling and inverse transform component 229 applies the inverse operations of the transform scaling and quantization component 213 to support motion estimation. The scaling and inverse transform component 229 applies inverse scaling, transformation, and / or quantization to reconstruct residual blocks in the pixel domain, for example, for later use as reference blocks that can become predictive blocks for other current blocks. The motion estimation component 221 and / or motion compensation component 219 may calculate reference blocks by adding the residual blocks back to the corresponding predictive blocks for use in motion estimation of later blocks / frames. Filters are applied to the reconstructed reference blocks to mitigate artifacts introduced during scaling, quantization, and transformation. Such artifacts could otherwise cause erroneous predictions (and cause further artifacts) when subsequent blocks are predicted.

[0063] The filter control analysis component 227 and the in-loop filter component 225 apply filters to the residual block and / or to the reconstructed image block. For example, the transformed residual block from the scaling and inverse transform component 229 may be combined with the corresponding prediction block from the intra-picture prediction component 217 and / or the motion compensation component 219 to reconstruct the original image block. The filter may then be applied to the reconstructed image block. In some examples, the filter may be applied to the residual block instead. As with the other components of FIG. 2, the filter control analysis component 227 and the in-loop filter component 225 may be highly integrated and implemented together, but are depicted separately for conceptual purposes. The filter applied to the reconstructed reference block is applied to a particular spatial region and includes multiple parameters to adjust how such a filter is applied. The filter control analysis component 227 analyzes the reconstructed reference block to determine where such a filter should be applied and sets the corresponding parameters. Such data is forwarded to the header formatting and CABAC component 231 as filter control data for encoding. The in-loop filter component 225 applies such filters based on the filter control data. The filters may include deblocking filters, noise suppression filters, SAO filters, and adaptive loop filters. Such filters may be applied in the spatial / pixel domain (e.g., on reconstructed pixel blocks) or in the frequency domain, depending on the example.

[0064] When operating as an encoder, the filtered reconstructed image blocks, residual blocks, and / or predictive blocks are stored in the decoding picture buffer component 223 for later use in motion estimation, as described above. When operating as a decoder, the decoding picture buffer component 223 stores the reconstructed and filtered blocks and forwards them to a display as part of an output video signal. The decoding picture buffer component 223 may be any memory device capable of storing predictive blocks, residual blocks, and / or reconstructed image blocks.

[0065] The header formatting and CABAC component 231 receives data from various components of the codec system 200 and encodes such data into a coded bitstream for transmission to a decoder. Specifically, the header formatting and CABAC component 231 generates various headers for encoding control data, such as general control data and filter control data. Furthermore, residual data in the form of quantized transform coefficient data, as well as prediction data, including intra prediction and motion data, are all encoded into the bitstream. The final bitstream contains all information desired by the decoder to reconstruct the original partitioned video signal 201. Such information may also include an intra prediction mode index table (also called a codeword mapping table), definitions of the encoding contexts of various blocks, indications of the most probable intra prediction mode, indications of partition information, etc. Such data may be encoded by using entropy coding. For example, the information may be encoded using context adaptive variable length coding (CAVLC), CABAC, syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding techniques. Following entropy coding, the coded bitstream may be transmitted to another device (e.g., a video decoder) or archived for later transmission or retrieval.

[0066] 3 is a block diagram illustrating an example video encoder 300. The video encoder 300 may be used to implement the encoding functionality of the codec system 200 and / or to implement steps 101, 103, 105, 107, and / or 109 of the method of operation 100. The encoder 300 partitions an input video signal to result in a partitioned video signal 301 that is substantially similar to the partitioned video signal 201. The partitioned video signal 301 is then compressed and encoded into a bitstream by components of the encoder 300.

[0067] Specifically, the partitioned video signal 301 is forwarded to an intra-picture prediction component 317 for intra prediction. The intra-picture prediction component 317 may be substantially similar to the intra-picture estimation component 215 and the intra-picture prediction component 217. The partitioned video signal 301 is also forwarded to a motion compensation component 321 for inter prediction based on a reference block in a decoding picture buffer component 323. The motion compensation component 321 may be substantially similar to the motion estimation component 221 and the motion compensation component 219. The prediction block and the residual block from the intra-picture prediction component 317 and the motion compensation component 321 are forwarded to a transform and quantization component 313 for transforming and quantizing the residual block. The transform and quantization component 313 may be substantially similar to the transform scaling and quantization component 213. The transformed and quantized residual block and the corresponding prediction block (along with associated control data) are forwarded to an entropy coding component 331 for coding into a bitstream. The entropy coding component 331 may be substantially similar to the header formatting and CABAC component 231 .

[0068] The transformed and quantized residual block and / or the corresponding prediction block are also forwarded from the transform and quantization component 313 to the inverse transform and quantization component 329 for reconstruction into a reference block used by the motion compensation component 321. The inverse transform and quantization component 329 may be substantially similar to the scaling and inverse transform component 229. An in-loop filter in the in-loop filter component 325 is also applied to the residual block and / or the reconstructed reference block, depending on the example. The in-loop filter component 325 may be substantially similar to the filter control analysis component 227 and the in-loop filter component 225. The in-loop filter component 325 may include multiple filters as described with respect to the in-loop filter component 225. The filtered block is then stored in the decoding picture buffer component 323 for use as a reference block by the motion compensation component 321. The decoding picture buffer component 323 may be substantially similar to the decoding picture buffer component 223.

[0069] 4 is a block diagram illustrating an example video decoder 400. Video decoder 400 may be used to implement the decoding functionality of codec system 200 and / or implement steps 111, 113, 115, and / or 117 of method of operation 100. Decoder 400 receives a bitstream, for example, from encoder 300, and generates a reconstructed output video signal based on the bitstream for display to an end user.

[0070] The bitstream is received by the entropy decoding component 433. The entropy decoding component 433 is configured to implement an entropy decoding scheme, such as CAVLC, CABAC, SBAC, PIPE coding, or other entropy coding techniques. For example, the entropy decoding component 433 may use header information to provide context for interpreting additional data encoded as codewords in the bitstream. The decoded information includes any desired information for decoding a video signal from a residual block, such as general control data, filter control data, partition information, motion data, prediction data, and quantized transform coefficients. The quantized transform coefficients are forwarded to the inverse transform and quantization component 429 for reconstruction into a residual block. The inverse transform and quantization component 429 may be similar to the inverse transform and quantization component 329.

[0071] The reconstructed residual block and / or predictive block are forwarded to the intra picture prediction component 417 for reconstruction into an image block based on an intra prediction operation. The intra picture prediction component 417 may be similar to the intra picture estimation component 215 and the intra picture prediction component 217. Specifically, the intra picture prediction component 417 uses a prediction mode to locate a reference block in a frame and applies the residual block to the result to reconstruct an intra predicted image block. The reconstructed intra predicted image block and / or residual block and the corresponding inter prediction data are forwarded to the decoding picture buffer component 423 via the in-loop filter component 425. These may be substantially similar to the decoding picture buffer component 223 and the in-loop filter component 225, respectively. The in-loop filter component 425 filters the reconstructed image block, the residual block and / or the predictive block, and such information is stored in the decoding picture buffer component 423. The reconstructed image block from the decoding picture buffer component 423 is forwarded to the motion compensation component 421 for inter prediction. The motion compensation component 421 may be substantially similar to the motion estimation component 221 and / or the motion compensation component 219. Specifically, the motion compensation component 421 uses a motion vector from a reference block to generate a prediction block and applies a residual block to the result to reconstruct an image block. The resulting reconstructed block may also be forwarded to the decoding picture buffer component 423 via the in-loop filter component 425. The decoding picture buffer component 423 continues to store further reconstructed image blocks that may be reconstructed in a frame according to the partition information. Such frames may also be arranged in a sequence. The sequence is output to a display as a reconstructed output video signal.

[0072] 5 is a schematic diagram illustrating an example bitstream 500 including an encoded video sequence along with an HPS 513. For example, bitstream 500 may be generated by codec system 200 and / or encoder 300 for decoding by codec system 200 and / or decoder 400. As another example, bitstream 500 may be generated by an encoder at step 109 of method 100 for use by a decoder at step 111.

[0073] The bitstream 500 includes a sequence parameter set (SPS) 510, a number of picture parameter sets (PPS) 512, a number of slice headers 514, and image data 520. The SPS 510 includes sequence data common to all pictures in a video sequence included in the bitstream 500. Such data may include picture sizing, bit depth, coding tool parameters, bit rate limits, etc. The PPS 512 includes parameters that are specific to one or more corresponding pictures. Thus, each picture in the video sequence may reference one PPS 512. The PPS 512 may indicate coding tools, quantization parameters, offsets, picture-specific coding tool parameters (e.g., filter control), etc. available for tiles in the corresponding picture. The slice header 514 includes parameters that are specific to one or more corresponding slices in the picture. Thus, each slice in the video sequence may reference a slice header 514. The slice header 514 may include slice type information, a picture order count (POC), a reference picture list, prediction weights, tile entry points, deblocking parameters, etc. In some examples, a slice may be referred to as a tile group. In such cases, the slice header 514 may be referred to as a tile group header.

[0074] The image data 520 includes video data encoded according to inter-prediction and / or intra-prediction, together with corresponding transformed and quantized residual data. Such image data 520 is sorted according to a partitioning used to partition the image before encoding. For example, a video sequence is divided into pictures 521. The pictures 521 are divided into slices 523. The slices 523 may be further divided into tiles and / or CTUs. The CTUs are further divided into coding blocks based on a coding tree. The coding blocks may then be encoded / decoded according to a prediction mechanism. For example, the picture 521 may include one or more slices 523. The picture 521 refers to the PPS 512, and the slices 523 refer to the slice header 514. Each slice 523 may include one or more tiles. Each slice 523 and / or picture 521 may then include multiple CUUs.

[0075] Each picture 521 may contain the entire set of visual data associated with the video sequence at the corresponding time. The VR system may display a user-selected region of the picture 521, which creates the sense of being present in the scene represented by the picture 521. The regions that the user may want to see are not known at the time the bitstream 500 is encoded. Thus, the picture 521 may contain each possible region that the user may potentially see. However, in the VR context, the corresponding codec may be designed based on the presumption that the user will only see the selected region of the picture 521, and the remaining part of the picture 521 is discarded.

[0076] Each slice 523 may be a rectangle defined by a CTU in the top left corner and a CTU in the bottom right corner. In some examples, slices 523 include a succession of tiles and / or CTUs in a raster scan order proceeding from left to right and top to bottom. In other examples, slices 523 are rectangular slices. Rectangular slices may not traverse the entire width of the picture according to the raster scan order. Instead, rectangular slices may include rectangular and / or square regions of picture 521 defined in terms of CTUs and / or tile rows and CTUs and / or tile columns. Slices 523 are the smallest units that can be displayed separately by a decoder. Thus, slices 523 from picture 521 may be assigned to different sub-regions 522 to separately represent desired regions of picture 521. For example, in a VR context, picture 521 may include the entire visible sphere of data, but a user may only see sub-pictures 522 that include one or more slices 523 on a head mounted display.

[0077] As described above, the video codec may assume that non-selected regions of the picture 521 should be discarded at the decoder. Thus, a sub-bitstream 501 may be extracted from the bitstream 500. The extracted sub-bitstream 501 may include selected sub-pictures 522 and associated syntax. The non-selected regions of the picture 521 may be transmitted at a lower resolution or omitted to improve coding efficiency. The sub-pictures 522 are selected regions of the picture 521 and may include one or more associated slices 524. The slices 524 are subsets of the slices 523 that represent selected regions of the picture 521 that are associated with the sub-pictures 522. The sub-bitstream 501 also includes the SPS 510, PPS 512, slice headers 514, and / or sub-portions thereof that are associated with the sub-pictures 522 and slices 524.

[0078] Sub-bitstreams 501 may be extracted from bitstream 500. For example, a user using a decoder may view a segment of video. The user may select a corresponding region of picture 521. The decoder may request a subsequent sub-picture 522 associated with the region the user is currently viewing. The encoder may then transmit the sub-picture 522 associated with the selected region at a higher resolution and the remaining region of picture 521 at a lower resolution. To enable such functionality, the decoder may extract 529 one or more sub-bitstreams 501 from bitstream 500. Extraction 529 includes placing sub-picture 522 in sub-bitstream 501 while including slice 524 in sub-picture 522. Extraction 529 also includes placing associated SPS 510, PPS 512, and slice headers 514 in the sub-bitstream as desired to aid in decoding sub-picture 522 and slice 524.

[0079] One problem with extracting 529 the sub-bitstream 501 is that the addressing associated with the picture 521 may differ from the addressing associated with the sub-picture 522. Addressing issues are discussed in more detail below. In some systems, the slice headers 514 may be rewritten to adjust for such addressing inconsistencies. However, the sub-bitstream 501 may include many slice headers 514 (e.g., as many as one or two per picture 521), and such slice headers 514 are dynamically rewritten for each user. As such, rewriting the slice headers 514 in this manner may be very processor intensive. This disclosure includes mechanisms that allow slice headers 514 to be extracted 529 into the sub-bitstream 501 without rewriting the slice headers 514.

[0080] In a system that rewrites the slice header 514, slices 523 and 524 are addressed based on index values, such as slice index, tile index, CTU index, etc. Such indexes increase in value in raster scan order. To correct addressing inconsistencies, the disclosed embodiments use defined ID values ​​for each slice, tile, and / or CTU. Such defined IDs may be default values ​​and / or may be selected by the encoder. Although the defined IDs may increase in raster scan order in a consistent manner, such defined IDs may not be monotonically increasing. Thus, the defined IDs may leave gaps between values ​​to allow address management. For example, the indexes may increase monotonically (e.g., 0, 1, 2, 3, etc.), while the defined IDs may increase by some defined multiple (e.g., 0, 10, 20, 30, etc.). The encoder may include a mapping 535 in the bitstream 500 and the sub-bitstream 501, which allows a decoder to map from the defined IDs to an index that the decoder can interpret.

[0081] A parameter set, such as SPS 510 and / or PPS 512, may include an ID flag 531. The ID flag 531 may be set to indicate that a mapping 535 is available to map slice addresses from picture 521 based locations to sub-picture 522 based locations. Thus, the ID flag 531 may be set to indicate to a decoder that the disclosed mechanism is being used in the bitstream 500 and the sub-bitstream 501. For example, the ID flag 531 may be coded as an explicit tile ID flag, sps_subpic_id_present_flag, or other syntax element. The ID flag 531 may be encoded in the bitstream 500 and extracted 529 into the sub-bitstream 501.

[0082] A parameter set such as SPS 510 and / or PPS 512 may include a syntax element for ID 532. ID 532 may indicate a sub-picture 522 within picture 521. For example, an array of IDs 532 may be included in PPS 512 of bitstream 500. When sub-bitstream 501 is extracted 529, IDs 532 associated with sub-pictures 522 to be sent to a decoder may be included in PPS 512 of sub-bitstream 501. In another example, an indication to an associated ID 532 may be inserted in PPS 512 in sub-bitstream 501 to enable the decoder to determine the correct ID 532. For example, ID 532 may be coded as SubPicIdx, Tile_id_val[i], or other syntax element that indicates a boundary of sub-picture 522.

[0083] A parameter set such as SPS 510 and / or PPS 512 may include a syntax element for the length of slice address 533. Furthermore, slice header 514 may include a slice address 534 of slice 523. Slice address 534 is included as a defined ID value. Slice address 534 may be directly extracted 529 into slice header 514 in sub-bitstream 501 without modification to avoid rewriting slice header 514. For example, slice address 534 may be coded as slice_address, tile_group_address, or other syntax element indicating the boundary of slices 523 and 524. Slice address length 533 may then be used to interpret slice address 534. For example, slice address 534 may include an encoder-defined value and thus be coded as a variable length value followed by a byte alignment field. The slice address length 533 may indicate the number of bits contained in the corresponding slice address 534 and thus may indicate the boundaries of the slice address 534 to a decoder. As such, a decoder may use the slice address length 533 (e.g., from the PPS 512) to interpret the slice address 534. As such, the slice header 514 does not need to be rewritten to adjust the byte alignment field following the slice address 534. For example, the slice address length 533 may be coded as subpic_id_len_minus1, tile_id_len_minus1, or other syntax element indicating the slice address length 533. The slice address length 533 may be included in the PPS 512 of the bitstream 500 and then extracted 529 into the PPS 512 of the sub-bitstream 501.

[0084] The mapping 535 may also be transmitted in a parameter set, such as the SPS 510, the PPS 512, and / or the slice header 514. The mapping 535 indicates a mechanism for mapping slice addresses from picture 521 based locations to sub-picture 522 based locations. The mapping 535 may be encoded in the bitstream 500 and extracted 529 into a corresponding parameter set in the sub-bitstream 501. For example, the mapping 535 may be coded as a SliceSubpicToPicIdx[SubPicIdx][slice_address] syntax element, a tileIDToIdx[Tile_group_address] syntax element, or other syntax element indicating a mechanism for mapping slice addresses from picture 521 based locations to sub-picture 522 based locations.

[0085] Thus, a decoder can read the sub-bitstream 501 and obtain the ID flag 531 to determine that the slices 524 are addressed by a defined address instead of an index. The decoder can obtain the ID 532 to determine the sub-pictures 522 that are included in the sub-bitstream 501. The decoder can also obtain the slice address 534 and the slice address length 533 to interpret the slice address 534. The decoder can then obtain the mapping 535 to map the slice address 534 into a format that the decoder can interpret. The decoder can then use the slice address 534 when decoding and displaying the sub-picture 522 and the corresponding slice 524.

[0086] 6 is a schematic diagram illustrating an exemplary picture 600 partitioned for coding. For example, picture 600 may be encoded into and decoded from bitstream 500, e.g., by codec system 200, encoder 300, and / or decoder 400. Furthermore, picture 600 may be partitioned and / or included into sub-pictures within sub-bitstream 501 to support encoding and decoding according to method 100.

[0087] Picture 600 may be partitioned into slices 623, which may be substantially similar to slices 523. Slices 623 may be further partitioned into tiles 625 and CTUs 627. In FIG. 6, slices 623 are represented by thick lines with alternating white background and shading to graphically distinguish slices 623 from one another. Tiles 625 are shown by dashed lines. Boundaries of tiles 625 that lie on the boundaries of slices 623 are represented as thick dashed lines, and boundaries of tiles 625 that do not lie on the boundaries of slices 623 are represented as less thick dashed lines. Boundaries of CTUs 627 are represented as less thick solid lines, except where the boundaries of CTUs 627 are covered by the boundaries of tiles 625 or slices 623. In this example, picture 600 includes 9 slices 623, 24 tiles 625, and 216 CTUs 627.

[0088] As shown, slices 623 are rectangular with boundaries that may be defined by contained tiles 625 and / or CTUs 627. Slices 623 may not extend across the entire width of picture 600. Tiles 625 may be generated in slices 623 according to rows and columns. CTUs 627 may be partitioned from tiles 625 and / or slices 623 to generate partitions of picture 600 suitable to be subdivided into coding blocks for coding according to inter prediction and / or intra prediction. Picture 600 may be encoded in a bitstream, such as bitstream 500. Regions of picture 600 may be included in and extracted into sub-pictures, such as sub-picture 522 and sub-bitstream 501, respectively.

[0089] 7 is a schematic diagram illustrating an exemplary sub-picture 722 extracted from a picture 700. For example, the picture 700 may be substantially similar to the picture 600. Furthermore, the picture 700 may be encoded into the bitstream 500, for example, by the codec system 200 and / or the encoder 300. The sub-picture 722 may be extracted into and decoded from the sub-bitstream 501, for example, by the codec system 200, the encoder 300, and / or the decoder 400. Furthermore, the picture 700 may be used to support encoding and decoding according to the method 100.

[0090] As shown, picture 700 includes an upper left corner 702 and a lower right corner 704. Subpicture 722 includes one or more slices 723 from picture 700. When using indexes, the upper left corner 702 and the lower right corner 704 are associated with the first and last indexes, respectively. However, a decoder may only display subpicture 722 and not the entire picture 700. Furthermore, slice address 734 of first slice 723a may not align with upper left corner 702, and slice address 734 of third slice 723c may not align with lower right corner 704. As such, slice address 734 for subpicture 722 does not align with slice address 734 for picture 700. This disclosure uses defined IDs for slice addresses 734 instead of indexes. The decoder can use the mapping to map slice addresses 734 from locations based on the picture 700 to locations based on the subpicture 722. The decoder can then use the mapped slice addresses 734 to place the first slice 723a in the upper left corner 702 of the decoder display, the third slice 723c in the lower right corner 704 of the decoder display, and the second slice 723b between the first slice 723a and the third slice 723c.

[0091] As described herein, the present disclosure describes improvements for explicit tile ID signaling in video coding where tiles are used for picture partitioning. The description of the technique is based on VVC by JVET of ITU-T and ISO / IEC. However, the technique also applies to other video codec standards. The following are example embodiments described herein:

[0092] The concepts of tile index and tile ID may be distinguished. The tile ID of a tile may or may not be equal to the tile index of the tile. When the tile ID is different from the tile index, the mapping between the tile ID and the tile index may be signaled in the PPS. The tile ID may be used for signaling the tile group address in the tile group header instead of using the tile index. In this way, the value of the tile ID can remain the same when the tile group is extracted from the original bitstream. This can be achieved by updating the mapping between the tile ID and the tile index in the PPS referenced by the tile group. This approach addresses the case where the value of the tile index may change depending on the sub-picture to be extracted. It should be noted that other parameter sets (e.g., other than the slice header) may still be rewritten when performing sub-bitstream extraction based on MCTS.

[0093] This may be achieved by using a flag in the parameter set in which the tile information is signaled. For example, the PPS may be used as the parameter set. For example, the explicit_tile_id_flag may be used for this purpose. The explicit_tile_id_flag may be signaled regardless of the number of tiles in the picture and may indicate that explicit tile signaling is used. A syntax element may also be used to specify the number of bits for signaling the tile ID value (e.g., the mapping between the tile index and the tile ID). Such a syntax element may also be used to signal the tile ID / address in the tile group header. For example, the tile_id_len_minus1 syntax element may be used for this purpose. The tile_id_len_minus1 may not be present if the explicit_tile_id_flag is equal to zero (e.g., the tile ID is set equal to the tile index). If tile_id_len_minus1 is not present, the value of tile_id_len_minus1 may be inferred to be equal to the value of Ceil(Log2(NumTilesInPic)). A further constraint may require that a bitstream that is the result of an MCTS sub-bitstream extraction may include an explicit_tile_id_flag set equal to 1 for active PPSs unless the sub-bitstream contains the top-left tile in the original bitstream.

[0094] In an example embodiment, the video coding syntax may be modified as described below to achieve the functionality described herein. An example CTB raster and tile scan process may be described as follows: A list of ctbAddrTs, TileId[CtbAddrTs] ranging from 0 to PicSizeInCtbsY-1 inclusive, that specifies the conversion from CTB addresses in a tile scan to tile IDs, and a list of tileIdx, NumCtusInTile[tileIdx] ranging from 0 to PicSizeInCtbsY-1 inclusive, that specifies the conversion from tile indexes to numbers of CTUs in a tile, are derived as follows:

number

[0095] The list NumCtusInTile[tileIdx] of tileIdx ranging from 0 to PicSizeInCtbsY-1 inclusive, that specifies the conversion from tile index to the number of CTUs in the tile, may be derived as follows:

number

[0096] The set of NumTilesInPic tileId value pairs TileIdToIdx[tileId] that specifies the conversion from tile IDs to tile indexes may be derived as follows:

number

[0097] An example picture parameter set RBSP syntax may be written as follows: [Table 1]

[0098] An example tile group header syntax may be written as follows: [Table 2]

[0099] An example tile group data syntax may be written as follows: [Table 3]

[0100] Example picture parameter set RBSP semantics may be described as follows: explicit_tile_id_flag set equal to 1 specifies that the tile ID of each tile is explicitly signaled. explicit_tile_id_flag set equal to zero specifies that the tile ID is not explicitly signaled. For bitstreams that are the result of MTCS sub-bitstream extraction, the value of explicit_tile_id_flag may be set equal to 1 for active PPSs as long as the resulting bitstream does not contain the top-left tile in the original bitstream. tile_id_len_minus1 plus1 specifies the number of bits used to represent the syntax elements tile_id_val[i] and tile_group_address in the tile group header that references the PPS. The value of tile_id_len_minus1 may be in the range from Ceil(Log2(NumTilesInPic)) to 15, inclusive. If not present, the value of tile_id_len_minus1 may be inferred to be equal to Ceil(Log2(NumTilesInPic)). It should be noted that the value of tile_id_len_minus1 may be greater than Ceil(Log2(NumTilesInPic)) in some cases because the current bitstream may be the result of an MCTS sub-bitstream extraction. In that case, the tile ID, which may be a tile index in the original bitstream, may be represented by Ceil(Log2(OrgNumTilesInPic)) bits, where OrgNumTilesInPic is the NumTilesInPic of the original bitstream, greater than NumTilesInPic of the current bitstream. tile_id_val[i] identifies the tile ID of the i-th tile of the picture that refers to the PPS. The length of tile_id_val[i] is tile_id_len_minus1+1 bits.For any integers m and n in the range of 0 to NumTilesInPic-1, inclusive, tile_id_val[m] may not be equal to tile_id_val[n] if m is not equal to n, and tile_id_val[m] may be less than tile_id_val[n] if m is less than n.

[0101] The following variables: a list ColWidth[i] for i ranging from 0 to num_tile_columns_minus1 inclusive, specifying the width of the ith tile column in units of CTBs; a list RowHeight[j] for j ranging from 0 to num_tile_rows_minus1 inclusive, specifying the height of the jth tile row in units of CTBs; a list ColBd[i] for i ranging from 0 to num_tile_columns_minus1+1 inclusive, specifying the location of the ith tile column border in units of CTBs; a list RowBd[j] for j ranging from 0 to num_tile_rows_minus1+1 inclusive, specifying the location of the jth tile row border in units of CTBs; a list CtbAd of ctbAddrRs ranging from 0 to PicSizeInCtbsY-1 inclusive, specifying the conversion from CTB addresses in the CTB raster scan of the picture to CTB addresses in the tile scan drRsToTs[ctbAddrRs];A list of ctbAddrTs, ranging from 0 to PicSizeInCtbsY-1 inclusive, that specifies the conversion from CTB addresses in a tile scan to CTB addresses in a CTB raster scan of a picture;TileId[ctbAddrTs], a list of ctbAddrTs, ranging from 0 to PicSizeInCtbsY-1 inclusive, that specifies the conversion from CTB addresses in a tile scan to tile IDs;NumCtusInTile[tileIdx], a list of tileIdx, ranging from 0 to PicSizeInCtbsY-1 inclusive, that specifies the conversion from tile index to the number of CTUs in a tile;FirstCtbAddrTs[tileIdx], a list of tileIdx, ranging from 0 to NumTilesInPic-1 inclusive, that specifies the conversion from tile IDs to CTB addresses in a tile scan of the first CTB in a tile;A set of tuples of NumTilesInPic tileId values, TileIdToIdx[tileID], that specify a conversion from tile IDs to tile indexes, and a list of tileIdx, FirstCtbAddrTs[tileIdx], ranging from 0 to NumTilesInPic-1 inclusive, that specifies a conversion from tile IDs to CTB addresses in the tile scan of the first CTB in the tile; a list of ColumnWidthInLumaSamples[i], for i, ranging from 0 to num_tile_columns_minus1 inclusive, that specifies the width of the i-th tile column in units of luma samples; and a list of RowHeightInLumaSamples[j], for j, ranging from 0 to num_tile_rows_minus1 inclusive, that specifies the height of the j-th tile row in units of luma samples, may be derived by invoking the CTB raster and tile scan conversions.

[0102] tile_group_address specifies the tile ID of the first tile in the tile group. The length of tile_group_address is tile_id_len_minus1+1 bits. The value of tile_group_address may be in the range from 0 to 2tile_id_len_minus1+1-1, inclusive, and the value of tile_group_address may not be equal to the value of tile_group_address of any other coded tile group NAL unit of the same coded picture.

[0103] FIG. 8 is a schematic diagram of an example video coding device 800. The video coding device 800 is suitable for implementing the disclosed examples / embodiments described herein. The video coding device 800 has a downstream port 820, an upstream port 850, and / or a transceiver unit (Tx / Rx) 810 including a transmitter and / or a receiver for communicating data upstream and / or downstream over a network. The video coding device 800 also includes a processor 830 including a logic unit and / or a central processing unit (CPU) for processing data, and a memory 832 for storing data. The video coding device 800 may also have electrical, optical, electrical (OE), electrical, optical (EO), and / or wireless communication components coupled to the upstream port 850 and / or the downstream port 820 for communication of data over an electrical, optical, or wireless communication network. The video coding device 800 may also include input and / or output (I / O) devices 860 for communicating data to and from a user. The I / O devices 860 may include output devices such as a display for displaying video data, speakers for outputting audio data, etc. The I / O devices 860 may also include input devices such as a keyboard, a mouse, a trackball, etc., and / or corresponding interfaces for interacting with such output devices.

[0104] The processor 830 is implemented by hardware and software. The processor 830 may be implemented as one or more CPU chips, cores (e.g., as a multi-core processor), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and digital signal processors (DSPs). The processor 830 communicates with the downstream port 820, the Tx / Rx 810, the upstream port 850, and the memory 832. The processor 830 includes a coding module 814. The coding module 814 implements the disclosed embodiments described above, such as the methods 100, 900, and 1000 that may use the bitstream 500, the picture 600, and / or the picture 700. The coding module 814 may also implement any other method / mechanism described herein. Furthermore, the coding module 814 may implement the codec system 200, the encoder 300, and / or the decoder 400. For example, when acting as an encoder, coding module 814 can identify a flag, a sub-picture ID, and a length in the PPS. Coding module 814 can also encode a slice address in the slice header. Coding module 814 can then extract the sub-picture sub-bitstream from the picture bitstream without rewriting the slice header. When acting as a decoder, coding module 814 can call a flag to determine if an explicit slice address is used instead of an index. Coding module 814 can also read the length and sub-picture ID from the PPS and the slice address from the slice header. Coding module 814 can then use the length to interpret the slice address and the sub-picture ID to map the slice address from a picture-based address to a sub-picture-based address.As such, coding module 814 can determine the desired location of a slice regardless of the selected subpicture and without the slice header having to be rewritten to accommodate subpicture-based address changes. As such, coding module 814 allows video coding device 800 to provide additional functionality when partitioning and coding video data, avoid certain processing to reduce processing overhead, and / or improve coding efficiency. Thus, coding module 814 improves the functionality of video coding device 800 in addition to addressing problems that are specific to video coding techniques. Furthermore, coding module 814 achieves transformation of video coding device 800 into different states. Alternatively, coding module 814 may be implemented as instructions stored in memory 832 and executed by processor 830 (e.g., as a computer program product stored on a non-transitory medium).

[0105] Memory 832 may comprise one or more memory types such as a disk, a tape drive, a solid state drive, read only memory (ROM), random access memory (RAM), flash memory, ternary content addressable memory (TCAM), static random access memory (SRAM), etc. Memory 832 may be used as an overflow data storage device to store programs when such programs are selected for execution and to store instructions and data retrieved during program execution.

[0106] 9 is a flow chart of an example method 900 for encoding a bitstream of a picture, such as bitstream 500 and picture 600, respectively, to support extraction of a sub-bitstream of a sub-picture, such as sub-bitstream 501 and sub-picture 522, respectively, without rewriting slice headers by using explicit address signaling. Method 900 may be used by an encoder, such as codec system 200, encoder 300, and / or video coding device 800, when performing method 100.

[0107] The method 900 may begin when an encoder receives a video sequence including multiple pictures and decides to encode the video sequence into a bitstream, for example, based on a user input. The video sequence is partitioned into pictures / images / frames for further partitioning before encoding. At step 901, a picture of the video sequence is encoded into a bitstream. The picture may include multiple slices including a first slice. The first slice may be any slice in the picture, but is described as the first slice for clarity of discussion. As an example, the top left corner of the first slice may not be aligned with the top left corner of the picture.

[0108] In step 903, a slice header associated with the slice is encoded into the bitstream. The slice header includes a slice address of the first slice. The slice address may have a defined value, such as a numeric value selected by the encoder. Such values ​​may be arbitrary, but may increase in raster scan order (e.g., left-to-right and top-to-bottom) to support consistent coding functionality. The slice address may not have an index. In some examples, the slice address may be a slice_address syntax element.

[0109] In step 905, a PPS is encoded into the bitstream. An identifier of the first slice and a length of the slice address may be encoded into the PPS in the bitstream. The identifier may be a subpicture identifier. The length of the slice address may indicate the number of bits included in the slice address. For example, the length of the slice address in the PPS may include sufficient data to interpret the slice address from the slice header coded in step 903. In some examples, the length may be a subpic_id_len_minus1 syntax element. Furthermore, the identifier may include sufficient data to map the slice address from a picture-based location to a subpicture-based location. In some examples, the identifier may be a subPicIdx syntax element. For example, multiple subpicture-based identifiers may be included in the PPS. If a subpicture is extracted, the corresponding subpicture ID may be indicated in the PPS, for example, by using a flag / pointer and / or excluding unused subpicture IDs. In some examples, an explicit ID flag may also be coded in the parameter set. The flag may indicate to the decoder that a mapping is available to map slice addresses from picture-based locations to subpicture-based locations. In some examples, the mapping may be a SliceSubpicToPicIdx[SubPicIdx][slice_address] syntax element. Thus, the flag may indicate that the slice address is not an index. In some examples, the flag may be sps_subpic_id_present_flag.

[0110] In step 907, a sub-bitstream of the bitstream is extracted. For example, this may include extracting the first slice based on the slice address, the length of the slice address, and an identifier of the first slice without rewriting the slice header. As a specific example, such extraction may also include extracting a sub-picture of the picture, where the sub-picture includes the first slice. Parameter sets may also be included in the sub-bitstream. For example, the sub-bitstream may include a sub-picture, a slice header, a PPS, an SPS, etc.

[0111] The sub-bitstream is stored for communication to the decoder, step 909. The sub-bitstream may then be transmitted to the decoder, as desired.

[0112] 10 is a flow chart of an example method 1000 of decoding sub-picture sub-bitstreams, such as sub-bitstream 501 and sub-picture 522, extracted from picture bitstreams, such as bitstream 500 and picture 600, by using explicit address signaling. Method 1000 may be used by a decoder, such as codec system 200, decoder 400, and / or video coding device 800, when performing method 100.

[0113] Method 1000 may begin when a decoder begins receiving a sub-bitstream extracted from a bitstream, for example as a result of method 900. In step 1001, a sub-bitstream is received. The sub-bitstream includes a sub-picture of a picture. For example, the bitstream encoded at the encoder may include a picture, and the sub-bitstream is extracted from the bitstream at the encoder and / or slicer, where the sub-bitstream includes a sub-picture that includes one or more regions from a picture in the bitstream. The received sub-picture may be partitioned into a number of slices. The number of slices may include a slice designated as a first slice. The first slice may be any slice in the picture, but is described as the first slice for clarity of discussion. As an example, the top left corner of the first slice may not be aligned with the top left corner of the picture. The sub-picture includes a PPS that describes syntax related to the picture and therefore also describes syntax related to the sub-picture. The sub-bitstream also includes a slice header that describes the syntax associated with the first slice.

[0114] In step 1003, a parameter set such as a PPS and / or an SPS may be parsed to obtain an explicit ID flag. The ID flag may indicate that a mapping is available to map slice addresses from picture-based locations to sub-picture-based locations. Thus, the flag may indicate that the corresponding slice address has a defined value and does not have an index. In some examples, the flag may be sps_subpic_id_present_flag. Based on the value of the ID flag, the PPS may be parsed to obtain an identifier of the first slice and a length of the slice address. The identifier may be a sub-picture identifier. The length of the slice address may indicate the number of bits included in the corresponding slice address. For example, the length of the slice address in the PPS may include sufficient data to interpret the slice address from the slice header. In some examples, the length may be a subpic_id_len_minus1 syntax element. Furthermore, the identifier may include sufficient data to map slice addresses from picture-based locations to sub-picture-based locations. In some examples, the identifier may be a subPicIdx syntax element. For example, multiple sub-picture based identifiers may be included in the PPS, and when a sub-picture is extracted, the corresponding sub-picture ID may be indicated in the PPS, for example, by using a flag / pointer and / or by excluding unused sub-picture IDs.

[0115] In step 1005, a slice address of the first slice is determined from the slice header based on the identifier and the length of the slice address. For example, the length from the PPS may be used to determine a bit boundary for interpreting the slice address from the slice header. The identifier and slice address may then be used to map the slice address from a picture-based location to a sub-picture-based location. As an example, the mapping between the picture-based location and the sub-picture-based location may be used to align the slice header to the sub-picture. This allows the decoder to compensate for address mismatch between the slice header and the picture addressing scheme caused by the sub-bitstream extraction without the slice header having to be rewritten in the encoder and / or slicer. In some examples, the mapping may be a SliceSubpicToPicIdx[SubPicIdx][slice_address] syntax element.

[0116] In step 1007, the sub-bitstream may be decoded to generate a sub-picture video sequence. The sub-picture may include the first slice. Thus, the first slice is also decoded. The sub-picture video sequence including the decoded first slice may then be transmitted, for example, for display by a head mounted display or other display device.

[0117] 11 is a schematic diagram of an example system 1100 for transmitting sub-picture sub-bitstreams, such as sub-bitstream 501 and sub-picture 522, extracted from picture bitstreams, such as bitstream 500 and picture 600, by using explicit address signaling. System 1100 may be implemented by encoders and decoders, such as codec system 200, encoder 300, decoder 400, and / or video coding device 800. Furthermore, system 1100 may be used when implementing methods 100, 900, and / or 1000.

[0118] The system 1100 includes a video encoder 1102. The video encoder 1102 includes an encoding module 1101 for encoding a picture having a plurality of slices including a first slice into a bitstream, encoding a slice header including a slice address of the first slice into the bitstream, and encoding a PPS including an identifier and a length of the slice address of the first slice into the bitstream. The video encoder 1102 further includes an extraction module 1103 for extracting a sub-bitstream of the bitstream by extracting the first slice based on the slice address, the length of the slice address, and the identifier of the first slice without rewriting the slice header. The video encoder 1102 further includes a storage module 1105 for storing the sub-bitstream for communication to a decoder. The video encoder 1102 further includes a transmission module 1107 for transmitting the sub-bitstream including the slice header, the PPS, the first slice, and / or the corresponding sub-picture to the decoder. The video encoder 1102 may be further configured to perform any of the steps of the method 900.

[0119] The system 1100 also includes a video decoder 1110. The video decoder 1110 includes a receiving module 1111 for receiving a sub-bitstream including a sub-picture of a picture partitioned into a plurality of slices including a first slice, a PPS associated with the picture and the sub-picture, and a slice header associated with the first slice. The video decoder 1110 further includes a parsing module 1113 for parsing the PPS to obtain an identifier and a length of a slice address of the first slice. The video decoder 1110 further includes a determining module 1115 for determining a slice address of the first slice from the slice header based on the identifier and the length of the slice address. The video decoder 1110 further includes a decoding module 1117 for decoding the sub-bitstream to generate a video sequence of the sub-picture including the first slice. The video decoder 1110 further includes a transport module 1119 for transporting the video sequence of the sub-picture for display. The video decoder 1110 may be further configured to perform any of the steps of the method 1000.

[0120] A first component is directly coupled to a second component when there are no intervening components, other than lines, traces, or other media between the first and second components. A first component is indirectly coupled to a second component when there are other intervening components, other than lines, traces, or other media between the first and second components. The term "coupled" and variations thereof include both directly coupled and indirectly coupled. Use of the term "about," unless otherwise stated, means a range that includes ±10% of the number that follows.

[0121] It should be understood that the steps of the exemplary methods described herein do not necessarily have to be performed in the order described, and the ordering of steps in such methods is merely exemplary. Similarly, additional steps may be included in such methods, and certain steps may be omitted or combined in methods according to various embodiments of the present disclosure.

[0122] Although several embodiments are provided in this disclosure, it can be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the disclosure. The examples should be considered as illustrative and not limiting, and the intention should not be limited to the details given herein. For example, various elements or components may be combined or integrated into other systems or certain features may be omitted or not implemented.

[0123] Additionally, the techniques, systems, subsystems, and methods described and illustrated in various embodiments as separate or distinct may be combined or integrated with other systems, components, techniques, or methods without departing from the scope of the present disclosure. Other examples of modifications, substitutions, and alterations will be ascertainable by those skilled in the art and may be made without departing from the spirit and scope of what is disclosed herein.

Claims

1. A method implemented in a decoder, comprising: receiving a bitstream including a sub-picture of a picture partitioned into a number of slices, a slice header associated with a first slice among the number of slices, and a sequence parameter set (SPS), the SPS including a sub-picture identifier (ID) of the sub-picture; parsing the bitstream to obtain a parameter for deriving a length of a slice address of the first slice, where depending on the presence or absence of the parameter, the length of the slice address is inferred to be equal to Ceil(Log2(NumTilesInPic)), where NumTilesInPic contains the number of tiles in the picture; determining the slice address of the first slice from the slice header based on the length of the slice address; decoding the sub-picture including the first slice based on the slice address and the sub-picture ID; The method according to claim 1,

2. The length of the slice address indicates the number of bits contained in the slice address. The method of claim 1.

3. Determining the slice address of the first slice includes: using the length to determine a bit boundary for interpreting the slice address from the slice header; using the sub-picture ID and the slice address to map the slice address from a picture-based location to a sub-picture-based location; having The method according to claim 1 or 2.

4. the mapping between the picture-based positions and the sub-picture-based positions aligns the slice headers to the sub-pictures without the slice headers having to be rewritten. The method according to claim 3.

5. 1. A method implemented in an encoder, comprising: encoding a picture into a bitstream, the picture having sub-pictures partitioned into a number of slices; encoding into the bitstream a parameter for deriving a length of a slice address of a first slice among the plurality of slices, the length of the slice address being inferred to be equal to Ceil(Log2(NumTilesInPic)), where NumTilesInPic contains a number of tiles in the picture; encoding the slice address of the first slice based on the length of the slice address into a slice header of the bitstream, the slice header being associated with the first slice; encoding a sub-picture ID of the sub-picture into a sequence parameter set (SPS) of the bitstream; The method according to claim 1,

6. The length of the slice address indicates the number of bits contained in the slice address. The method according to claim 5.

7. the slice address has a defined value and does not have an index; The method according to claim 5 or 6.

8. extracting a sub-picture of the picture; the sub-picture includes the first slice, the bitstream comprises the subpicture, the slice header including the slice address, and a parameter set including the parameters; 7. The method according to any one of claims 5 to 6.

9. A processor; Memory, a receiver coupled to the processor; a transmitter coupled to the processor; having The processor, the memory, the receiver and the transmitter are configured to perform the method according to any one of claims 1 to 8. Video coding device.

10. 1. A computer program for use by a video coding device, comprising: Comprising computer-executable instructions which, when executed by a processor, cause the video coding device to perform the method of any one of claims 1 to 8. Computer program.

11. a receiving unit configured to receive a bitstream including a sub-picture of a picture partitioned into a plurality of slices, a slice header associated with a first slice among the plurality of slices, and a sequence parameter set (SPS), the SPS including a sub-picture identifier (ID) of the sub-picture; a parsing unit configured to parse the bitstream to obtain a parameter for deriving a length of a slice address of the first slice, where depending on the presence or absence of the parameter, the length of the slice address is inferred to be equal to Ceil(Log2(NumTilesInPic)), where NumTilesInPic contains a number of tiles in the picture; a determiner configured to determine the slice address of the first slice from the slice header based on the length of the slice address; a decoding unit configured to decode the sub-picture including the first slice based on the slice address and the sub-picture ID; A decoder having

12. the decoder further comprising a transport unit configured to transport the sub-picture for display; A decoder according to claim 11.

13. An encoding unit, the encoding unit comprising: Encoding a picture into a bitstream, the picture having sub-pictures partitioned into a number of slices; encoding into the bitstream a parameter for deriving a length of a slice address of a first slice among the plurality of slices, the length of the slice address being inferred to be equal to Ceil(Log2(NumTilesInPic)), where NumTilesInPic comprises a number of tiles in the picture; encoding the slice address of the first slice based on the length of the slice address in a slice header of the bitstream, the slice header being associated with the first slice; Encoding a sub-picture ID of the sub-picture into a sequence parameter set (SPS) of the bitstream. An encoder configured as follows.

14. the encoder further comprising a storage unit configured to store the bitstream for communication to a decoder; The encoder of claim 13.

15. A data structure for use by a computer, comprising: The data structure includes a bitstream, the bitstream comprising: data representing a sub-picture of a picture partitioned into a number of slices; a slice header associated with a first slice of the plurality of slices; a sequence parameter set (SPS) including a sub-picture identifier (ID) of the sub-picture; A parameter for deriving a length of a slice address of the first slice, Depending on the presence or absence of said parameter, the length of the slice address is inferred to be equal to Ceil(Log2(NumTilesInPic)), where NumTilesInPic contains the number of tiles in the picture, a processor of the computer determining the slice address of the first slice from the slice header based on the length of the slice address, and decoding the sub-picture including the first slice based on the slice address and the sub-picture ID. Data structure.

16. 1. A method for transmitting an encoded bitstream of video data, comprising: Obtaining a bitstream generated by the method of any one of claims 5 to 8 from a storage medium; transmitting the bitstream; The method according to claim 1,

17. 1. A system for transmitting an encoded bitstream of video data, comprising: A receiver arranged to obtain from a storage medium a bitstream generated by a method according to any one of claims 5 to 8; a transmission unit configured to transmit the bitstream; A system having

18. 1. A method for storing an encoded bitstream of video data, comprising the steps of: Receiving the bitstream generated by the method of any one of claims 5 to 8; storing said bitstream on a storage medium; The method according to claim 1,

19. 1. A system for storing an encoded bitstream of video data, comprising: A receiver arranged to receive the bitstream generated by the method of any one of claims 5 to 8; a storage medium configured to store said bitstream; A system having

Images