Explicit address indication in video coding

Abstract

A video coding mechanism is disclosed. The mechanism includes receiving a sub-bitstream, wherein the sub-bitstream includes a sub-picture in a picture, wherein the picture is partitioned into a plurality of slices including a first slice, a parameter set related to the picture and the sub-picture, and a slice header related to the first slice; parsing the parameter set to obtain an identifier and a slice address length of the first slice; determining a slice address of the first slice from the slice header according to the identifier and the slice address length; decoding the sub-bitstream to generate a sub-picture video sequence including the first slice; and sending the sub-picture video sequence including the first slice for display.

Description

[0001] This application is a divisional application. The original application has the application number 201980087375.7 and the original application date is December 31, 2019. The entire contents of the original application are incorporated herein by reference.

[0002] Cross-referencing related applications

[0003] This patent application claims the benefit of U.S. Provisional Patent Application No. 62 / 787,110, filed December 31, 2018, entitled "Explicit Tile Identifier (ID) Signaling," and also claims the benefit of U.S. Provisional Patent Application No. 62 / 883,537, filed August 6, 2019, entitled "Explicit Tile Identifier (ID) Signaling," the contents of which are incorporated herein by reference. Technical Field

[0004] This invention generally relates to video decoding, and more specifically to address management when extracting sub-images from an image in video decoding. Background Technology

[0005] Even relatively short videos require a significant amount of video data to describe, which can be challenging when streaming or otherwise transmitting data over communication networks with limited bandwidth. Therefore, video data is typically compressed before transmission over modern telecommunications networks. Video size can also be an issue when storing video on storage devices due to potentially limited memory resources. Video compression devices typically encode video data on the source side using software and / or hardware before transmission or storage, reducing the amount of data required to represent the digital video image. The compressed data is then received at the destination side by a video decompression device that decodes the video data. Given limited network resources and the growing demand for higher video quality, there is a need for improved compression and decompression techniques that can increase compression ratios with minimal impact on image quality. Summary of the Invention

[0006] In one embodiment, the present invention includes a method implemented in a decoder. The method includes: a receiver of the decoder receiving a sub-stream, wherein the sub-stream includes sub-images in an image, wherein the image is segmented into multiple stripes including a first stripe, a parameter set associated with the image and the sub-images, and a stripe header associated with the first stripe; a processor of the decoder parsing the parameter set to obtain an identifier and a stripe address length of the first stripe; the processor determining the stripe address of the first stripe from the stripe header based on the identifier and the stripe address length; the processor decoding the sub-stream to generate a video sequence including the sub-images of the first stripe; and the processor sending the video sequence including the sub-images for display. In some video decoding systems, slices (also called tile groups) can be addressed according to a set of indices. These indices can start from index 0 at the top left corner of the image, increment in raster scan order, and end at index N at the bottom right corner of the image, where N is the number of indices minus 1. This system is suitable for most applications. However, some applications, such as virtual reality (VR), only display sub-images within an image. Some systems, when streaming VR content, send only sub-streams of the bitstream to the decoder to improve encoding efficiency, where the sub-stream includes the sub-images to be displayed. In this case, since the top-left corner of the sub-image received by the decoder is typically some index other than 0, index-based addressing schemes may stop working properly. To address these issues, an encoder (or associated striper) might be needed to rewrite each stripe header to change the sub-image index so that the top-left index starts from 0, and adjust the remaining sub-image stripes accordingly. Dynamically rewriting the stripe header (e.g., based on each user request) could require significant computational work from the processor. The disclosed system employs an addressing scheme that extracts sub-streams including sub-images without rewriting the stripe header. Each stripe is addressed based on an identifier (ID) other than the index (e.g., a sub-image ID). In this way, regardless of which sub-image is received and its position relative to the top-left corner of the complete image, the decoder can consistently determine all relevant addresses. Since the ID is arbitrarily defined (e.g., chosen by the encoder), it is encoded in a variable-length field. Accordingly, the stripe address length is also indicated. The ID associated with the sub-image is also indicated. The length is used to resolve the stripe address, and the sub-image ID is used to map the stripe address from an image-based location to a sub-image-based location. By employing these mechanisms, the encoder, decoder, and / or associated stripes can be improved. For example, instead of extracting and sending the entire bitstream, sub-streams can be extracted and sent, thus reducing the use of network resources, memory resources, and / or processing resources.Furthermore, this method of extracting sub-streams avoids rewriting each stripe header based on each user request, which further reduces the use of network resources, memory resources, and / or processing resources.

[0007] Alternatively, according to any of the above aspects, in another implementation of said aspect, the identifier is associated with a sub-image.

[0008] Alternatively, according to any of the foregoing aspects, in another implementation of said aspect, the stripe address length represents the number of bits included in the stripe address.

[0009] Optionally, according to any of the foregoing aspects, in another implementation of said aspect, determining the stripe address of the first stripe includes: the processor using the length in the parameter set to determine bit boundaries to parse the stripe address from the stripe header; the processor using the identifier and the stripe address to map the stripe address from an image-based location to a sub-image-based location.

[0010] Optionally, according to any of the foregoing aspects, in another implementation of said aspect, the method further includes: the processor parsing the parameter set to obtain an identifier (ID) flag, wherein the ID flag indicates that the mapping relationship can be used to map the strip address from the image-based location to the sub-image-based location.

[0011] Alternatively, according to any of the foregoing aspects, in another implementation of said aspect, the mapping relationship between the image-based position and the sub-image-based position aligns the strip header with the sub-image, without requiring the strip header to be rewritten.

[0012] Alternatively, in any of the foregoing aspects, in another implementation of said aspect, the stripe address includes a defined value but does not include an index.

[0013] In one embodiment, the present invention includes a method implemented in an encoder. The method includes: a processor of the encoder encoding an image in a bitstream, wherein the image comprises multiple stripes, the multiple stripes including a first stripe; the processor encoding a stripe header in the bitstream, wherein the stripe header includes a stripe address of the first stripe; the processor encoding a parameter set in the bitstream, wherein the parameter set includes an identifier and a stripe address length of the first stripe; the processor extracting a sub-bitstream from the bitstream by: extracting the first stripe based on the stripe address of the first stripe, the stripe address length, and the identifier, without rewriting the stripe header; and storing the sub-bitstream in the encoder's memory for transmission to a decoder. In some video decoding systems, slices (also called tile groups) can be addressed according to a set of indices. These indices can start from index 0 at the top left corner of the image, increment sequentially according to the raster scan, and end at index N at the bottom right corner of the image, where N is the number of indices minus 1. This system is suitable for most applications. However, some applications, such as virtual reality (VR), only display sub-images within an image. Some systems, when streaming VR content, send only sub-streams of the bitstream to the decoder to improve encoding efficiency, where the sub-stream includes the sub-images to be displayed. In this case, since the top-left corner of the sub-image received by the decoder is typically some index other than 0, index-based addressing schemes may stop working properly. To address these issues, an encoder (or associated striper) might be needed to rewrite each stripe header to change the sub-image index so that the top-left index starts from 0, and adjust the remaining sub-image stripes accordingly. Dynamically rewriting the stripe header (e.g., based on each user request) could require significant computational work from the processor. The disclosed system employs an addressing scheme that extracts sub-streams including sub-images without rewriting the stripe header. Each stripe is addressed based on an identifier (ID) other than the index (e.g., a sub-image ID). In this way, regardless of which sub-image is received and its position relative to the top-left corner of the complete image, the decoder can consistently determine all relevant addresses. Since the ID is arbitrarily defined (e.g., chosen by the encoder), it is encoded in a variable-length field. Accordingly, the stripe address length is also indicated. The ID associated with the sub-image is also indicated. The length is used to resolve the stripe address, and the sub-image ID is used to map the stripe address from an image-based location to a sub-image-based location. By employing these mechanisms, the encoder, decoder, and / or associated slicer can be improved.For example, instead of extracting and sending the entire bitstream, sub-streams can be extracted and sent, reducing the use of network, memory, and / or processing resources. Furthermore, extracting sub-streams in this way avoids rewriting each stripe header for each user request, further reducing the use of network, memory, and / or processing resources.

[0014] Alternatively, according to any of the above aspects, in another implementation of said aspect, the identifier is associated with a sub-image.

[0015] Alternatively, according to any of the foregoing aspects, in another implementation of said aspect, the stripe address length represents the number of bits included in the stripe address.

[0016] Alternatively, according to any of the foregoing aspects, in another implementation of said aspect, the length of the parameter set includes data sufficient to parse the strip address from the strip header, and the identifier includes data sufficient to map the strip address from an image-based location to a sub-image-based location.

[0017] Optionally, according to any of the foregoing aspects, in another implementation of said aspect, the method further includes: the processor encoding an identifier (ID) flag in a parameter set, wherein the ID flag indicates a mapping relationship that can be used to map the strip address from the image-based location to the sub-image-based location.

[0018] Alternatively, according to any of the foregoing aspects, in another implementation of said aspect, the stripe address includes a defined value but does not include an index.

[0019] Optionally, according to any of the above aspects, in another implementation of said aspect, the extraction of a sub-bitstream from the bitstream includes: extracting a sub-image from the image, wherein the sub-image includes the first stripe, and the sub-bitstream includes the sub-image, the stripe header, and the parameter set.

[0020] In one embodiment, the present invention includes a video decoding device. The video decoding device includes: a processor, a memory, a receiver coupled to the processor, and a transmitter coupled to the processor, wherein the processor, memory, receiver, and transmitter are configured to perform the method according to any of the foregoing aspects.

[0021] In one embodiment, the present invention includes a non-transitory computer-readable medium. The non-transitory computer-readable medium includes a computer program product for use by a video decoding device; the computer program product includes computer-executable instructions stored in the non-transitory computer-readable medium; when a processor executes the computer-executable instructions, it causes the video decoding device to perform the method according to any of the foregoing aspects.

[0022] In one embodiment, the present invention includes a decoder. The decoder includes: a receiving module for receiving a sub-stream, wherein the sub-stream includes sub-images in an image, wherein the image is segmented into multiple stripes including a first stripe, a parameter set associated with the image and the sub-images, and a stripe header associated with the first stripe; a parsing module for parsing the parameter set to obtain an identifier and a stripe address length of the first stripe; a determining module for determining the stripe address of the first stripe from the stripe header based on the identifier and the stripe address length; a decoding module for decoding the sub-stream to generate a video sequence including the sub-images of the first stripe; and a sending module for sending the video sequence including the sub-images of the first stripe for display.

[0023] Alternatively, according to any of the foregoing aspects, in another implementation of said aspect, the decoder is also used to perform the method according to any of the foregoing aspects.

[0024] In one embodiment, the present invention includes an encoder. The encoder includes: an encoding module for: encoding an image in a bitstream, wherein the image includes multiple stripes, the multiple stripes including a first stripe; encoding a stripe header in the bitstream, wherein the stripe header includes a stripe address of the first stripe; encoding a parameter set in the bitstream, wherein the parameter set includes an identifier and a stripe address length of the first stripe; an extraction module for extracting a sub-bitstream from the bitstream by: extracting the first stripe based on the stripe address of the first stripe, the stripe address length, and the identifier, without rewriting the stripe header; and a storage module for storing the sub-bitstream for transmission to a decoder.

[0025] Alternatively, according to any of the foregoing aspects, in another implementation of said aspect, the encoder is further configured to perform the method described according to any of the foregoing aspects.

[0026] For clarity, any of the above embodiments may be combined with any or more of the other embodiments described above to create new embodiments within the scope of the present invention.

[0027] These and other features will be more clearly understood from the following detailed description in conjunction with the accompanying drawings and claims. Attached Figure Description

[0028] To gain a more thorough understanding of the present invention, the following description is provided in conjunction with the accompanying drawings and specific embodiments, wherein the same reference numerals denote the same parts.

[0029] Figure 1 A flowchart illustrating an exemplary method for encoding video signals.

[0030] Figure 2 This is a schematic diagram of an exemplary encoding and decoding (encoding / decoding) system used for video decoding.

[0031] Figure 3 This is a schematic diagram of an exemplary video encoder.

[0032] Figure 4 This is a schematic diagram of an exemplary video decoder.

[0033] Figure 5 This is a schematic diagram of an exemplary sub-bitstream extracted from a bitstream.

[0034] Figure 6 A schematic diagram for segmenting an exemplary image used for encoding.

[0035] Figure 7 This is a schematic diagram of an exemplary sub-image extracted from an image.

[0036] Figure 8 This is a schematic diagram of an exemplary video decoding device.

[0037] Figure 9 The flowchart illustrates an exemplary method for encoding a bitstream of an image, which extracts a sub-bitstream including sub-images without rewriting the stripe header by using explicit address indication.

[0038] Figure 10 This is a flowchart of an exemplary method for decoding a sub-stream including sub-images using explicit address indication.

[0039] Figure 11 This is a schematic diagram of an exemplary system for sending a sub-stream including a sub-image via an explicit address indication. Detailed Implementation

[0040] First, it should be understood that although illustrative implementations of one or more embodiments are provided below, the disclosed systems and / or methods can be implemented using any number of techniques, whether currently known or existing. The invention should not in any way be limited to the illustrative implementations, drawings, and techniques described below, including the exemplary designs and implementations illustrated and described herein, but can be modified within the full scope of the appended claims and their equivalents.

[0041] This paper uses various abbreviations, such as coding tree block (CTB), coding tree unit (CTU), coding unit (CU), coded videosequence (CVS), Joint Video Experts Team (JVET), motion constrained tile set (MCTS), maximum transfer unit (MTU), network abstraction layer (NAL), picture order count (POC), raw byte sequence payload (RBSP), sequence parameter set (SPS), versatile video coding (VVC), and working draft (WD).

[0042] Many video compression techniques can be used to reduce video file size while minimizing data loss. For example, video compression techniques may include performing spatial (e.g., intra-frame) prediction and / or temporal (e.g., inter-frame) prediction to reduce or remove data redundancy in a video sequence. For block-based video decoding, video stripes (e.g., video images or portions of video images) can be segmented into video blocks, which may also be referred to as tree blocks, coding tree blocks (CTBs), coding tree units (CTUs), coding units (CUs), and / or coding nodes. Video blocks in intra-frame coded (I) stripes within an image are encoded using spatial prediction based on reference samples from adjacent blocks within the same image. Video blocks in inter-frame coded one-way prediction (P) or two-way prediction (B) stripes within an image can be encoded using spatial prediction based on reference samples from adjacent blocks within the same image or temporal prediction based on reference samples from other reference images. An image may be referred to as a frame, and a reference image may be referred to as a reference frame. Spatial or temporal prediction produces prediction blocks that represent image blocks. Residual data represents the pixel difference between the original image block and the predicted block. Accordingly, inter-frame coded blocks are encoded based on motion vectors and residual data, where the motion vectors point to the blocks of reference samples that make up the predicted block, and the residual data indicates the difference between the coded block and the predicted block. Intra-frame coded blocks are encoded based on the intra-frame coding mode and residual data. For further compression, the residual data can be transformed from the pixel domain to the transform domain, resulting in residual transform coefficients that can then be quantized. The quantized transform coefficients are initially arranged in a two-dimensional array. The quantized transform coefficients can be scanned to produce a one-dimensional vector of transform coefficients. Entropy coding can be applied to achieve further compression. These video compression techniques are discussed in more detail below.

[0043] To ensure that encoded video can be correctly decoded, the video is encoded and decoded according to the corresponding video coding standards. Video coding standards include ITU-T H.261, MPEG-1 Part 2, ITU-T H.262 or ISO / IEC MPEG-2 Part 2, ITU-T H.263, ISO / IEC MPEG-4 Part 2, Advanced Video Coding (AVC) (also known as ITU-T H.264 or ISO / IEC MPEG-4 Part 10), and High Efficiency Video Coding (HEVC) (also known as ITU-T H.265 or MPEG-H Part 2). AVC includes extended versions such as Scalable Video Coding (SVC), Multiview Video Coding (MVC), Multiview Video Coding plus Depth (MVC+D), and three-dimensional (3D) AVC (3D-AVC). HEVC includes extended versions such as Scalable HEVC (SHVC), Multiview HEVC (MV-HEVC), and 3D HEVC (3D-HEVC). The Joint Video Experts Team (JVET) of ITU-T and ISO / IEC has begun developing a video coding standard called Versatile Video Coding (VVC). VVC is included in the Working Draft (WD), which includes JVET-L1001-v7.

[0044] To encode a video image, it is first segmented, and then each segment is encoded into a bitstream. Various image segmentation schemes exist. For example, an image can be segmented into regular slices, non-independent slices, tiles, and / or segmented according to Wavefront Parallel Processing (WPP). For simplicity, HEVC restricts the encoder to only using regular slices, non-independent slices, tiles, WPP, and combinations thereof when segmenting slices into CTB groups for video decoding. This segmentation supports maximum transfer unit (MTU) size matching, parallel processing, and reduced end-to-end latency. MTU represents the maximum amount of data that can be sent in a single packet. If the packet payload exceeds the MTU, the payload is divided into two packets through a process called fragmentation.

[0045] Regular slices, also simply called slices, are portions of an image obtained after segmentation. They can be reconstructed independently of other regular slices within the same image, but due to loop filtering operations, interdependencies still exist. Each regular slice is encapsulated within its own network abstraction layer (NAL) unit and transmitted. Furthermore, intra-frame prediction (intra-sample prediction, motion information prediction, coding pattern prediction) and entropy coding dependencies across slice boundaries may be disabled to support independent reconstruction. This independent reconstruction supports parallelization. For example, parallelization based on regular slices reduces inter-processor or inter-core communication. However, since regular slices are independent, each slice is associated with a separate slice header. Because each slice has a slice header bit cost and lacks prediction across slice boundaries, using regular slices incurs significant coding overhead. Additionally, regular slices can be used to support MTU size matching requirements. Specifically, since regular slices are encapsulated in separate NAL units and can be encoded independently, each regular slice needs to be smaller than the MTU in the MTU scheme to avoid splitting the slice into multiple packets. Therefore, in order to achieve parallelization and MTU size matching, the strip layout within the image will be contradictory.

[0046] Non-independent striping is similar to regular striping, but with a shorter stripe header, allowing segmentation of image tree block boundaries without disrupting intra-frame prediction. Correspondingly, non-independent striping can divide a regular stripe into multiple NAL units, thus completing the encoding of the entire regular stripe first, and then sending out a portion of that stripe, thereby reducing end-to-end latency.

[0047] A tile is a segmentation within an image formed by horizontal and vertical boundaries, which create tile columns and rows. Tiles can be encoded in raster scan order (right-to-left, top-to-bottom). The scan order of CTBs is the order in which scans are performed within a tile. Accordingly, CTBs within the first tile are encoded first in raster scan order, and then CTBs within the next tile are processed. Similar to regular stripes, tiles break the dependency on intra-frame prediction and entropy decoding. However, tiles may not be included in a single NAL unit; therefore, tiles cannot be used to implement MTU size matching. Each tile can be processed by a single processor / core, and inter-processor / core communication used for intra-frame prediction between processing units decoding adjacent tiles can be limited to sending a shared stripe header (when adjacent tiles are in the same stripe) and sharing reconstructed samples and metadata associated with loop filtering. When a stripe includes multiple tiles, the entry point byte offset for each tile can be indicated in the stripe header, in addition to the first entry point offset within the stripe. For each stripe and block, at least one of the following conditions must be met: (1) all coding tree blocks in the stripe belong to the same block; (2) all coding tree blocks in the block belong to the same stripe.

[0048] In WPP, the image is segmented into single-line CTBs. Entropy decoding and prediction mechanisms can utilize data from CTBs in other lines. Parallel processing can be achieved through parallel decoding of CTB lines. For example, the current line can be decoded in parallel with the previous line. However, the decoding of the current line is delayed by two CTBs compared to the decoding of the previous few lines. This delay ensures that data related to the CTBs above and to the right of the current CTB in the current line is available before decoding the current CTB. When represented graphically, this method resembles a wavefront. This interleaved start decoding can be parallelized using as many processors / cores as the number of CTB lines included in the image. Due to the support for intra-frame prediction between adjacent tree block lines within the image, significant inter-processor / inter-core communication may be required for intra-frame prediction. WPP segmentation does not consider NAL unit size. Therefore, WPP does not support MTU size matching. However, regular striping can be used in conjunction with WPP, incurring some coding overhead, to achieve MTU size matching as needed.

[0049] Tiles can also include motion-constrained tilesets (MCTS). A motion-constrained tileset (MCTS) is a set of tiles such that the associated motion vectors are restricted to pointing to integer pixel positions within the MCTS and fractional pixel positions to which interpolation only requires integer pixel positions within the MCTS. Furthermore, motion vector candidates derived from blocks outside the MCTS for temporal motion vector prediction are not allowed. This allows each MCTS to be decoded independently, without the need for tiles included within the MCTS. Temporal MCTS supplemental enhancement information (SEI) messages can be used to indicate the presence of an MCTS in the bitstream and signal that MCTS exists. The MCTS SEI message provides supplemental information (detailed as part of the SEI message semantics) that can be used to extract MCTS sub-bitstreams to generate a consistent bitstream of the MCTS set. This information includes the number of extractable sets, each defining multiple MCTS sets and including raw byte sequence payload (RBSP) bytes of the replacement video parameter set (VPS), sequence parameter set (SPS), and picture parameter set (PPS) to be used in the MCTS sub-stream extraction process. Because one or all of the syntax elements related to the slice address (including first_slice_segment_in_pic_flag and slice_segment_address) can take different values ​​in the extracted sub-stream, the parameter sets (VPS, SPS, and PPS) can be rewritten or replaced, and the slice header updated, when extracting the sub-stream according to the MCTS sub-stream extraction process.

[0050] The above scheme may have some problems. In some systems, when there are multiple blocks / strips within an image, syntax elements (such as `tile_group_address`) can be used to indicate the address of the block group as an index in the block group header. `tile_group_address` represents the block address of the first block in the block group. The length of `tile_group_address` can be determined as Ceil(Log2(NumTilesInPic)) bits, where NumTilesInPic includes the number of blocks in the image. The value of `tile_group_address` can range from 0 to NumTilesInPic–1 (inclusive), and the value of `tile_group_address` may not be equal to the value of `tile_group_address` of any other encoded block group NAL unit within the same encoded image. When `tile_group_address` does not exist in the bitstream, it can be inferred that `tile_group_address` is 0. The block address described above includes the block index. However, using the block index as the address of each block group may reduce coding efficiency to some extent.

[0051] For example, in some cases, it may be necessary to modify the AVC or HEVC slice segment header between encoding and decoding, either directly on the client side or within a network-based media processing entity before sending the sub-stream to the decoder. An example of this is chunk-based streaming. In chunked streaming, panoramic video is encoded using HEVC chunks, but the decoder only decodes a portion of these chunks. By rewriting the HEVC slice segment header (SSH) and SPS / PPS, the bitstream can be controlled to change the subset of chunks being decoded and their spatial arrangement within the decoded video frames. One reason for this CPU overhead is that the AVC and HEVC slice segment headers use variable-length fields and have a byte alignment field at the end. That is, whenever a field in the SSH is changed, the byte alignment field at the end of the SSH is affected, and then that byte alignment field must be rewritten. Moreover, since all fields are variable-length encoded, the only way to know the position of the byte alignment field is to parse all the preceding fields. This results in significant processing overhead, especially when using chunks, where each second of video may contain hundreds of NALs. Some systems support explicit chunk identifiers (IDs). However, some syntax elements may not be optimized, and unnecessary and / or redundant bits may be included when indicating the chunk ID. Furthermore, some constraints related to explicit chunk ID indication are not specified.

[0052] For example, the above mechanism allows for image segmentation and compression. For instance, an image can be segmented into strips, chunks, and / or chunk groups. In some examples, chunk groups can be used interchangeably with strips. Such strips and / or chunk groups can be addressed using a set of indices. These indices can start at index 0 at the top-left corner of the image, incrementing in raster scan order, and ending at index N at the bottom-right corner of the image. In this case, N is the number of indices minus 1. This system is suitable for most applications. However, some applications, such as virtual reality (VR), display only sub-images within an image. Such sub-images may be referred to as regions of interest in some contexts. Some systems, when streaming VR content, send only a sub-stream of the bitstream to the decoder to improve encoding efficiency, where the sub-stream includes the sub-image to be displayed. In this case, since the top-left corner of the sub-image received by the decoder is typically some index other than 0, the index-based addressing scheme may stop working properly. To address these issues, the encoder (or associated striper) might need to rewrite each strip header to change the sub-image indexes, starting the top-left index from 0, and adjust the remaining sub-image stripes accordingly. Dynamically rewriting the strip headers (e.g., based on each user request) could require significant computational work from the processor.

[0053] This paper discloses various mechanisms to improve coding efficiency and reduce processing overhead when extracting sub-bitstreams including sub-images from an encoded bitstream that includes images. The disclosed system employs an addressing scheme that can extract sub-bitstreams including sub-images without rewriting the slice header. Each slice / block group is addressed based on an ID other than an index. For example, a slice can be addressed by a value that can be mapped to an index and stored in the slice header. Thus, the decoder reads the slice address from the slice header and maps the address from an image-based location to a sub-image-based location. Since the slice address is not a predefined index, it is encoded in a variable-length field. Accordingly, the slice address length is also indicated. The ID associated with the sub-image is also indicated. The sub-image ID and length can be indicated in the PPS. A flag can also be indicated in the PPS to indicate that an explicit addressing scheme is used. After reading this flag, the decoder can obtain the length and sub-image ID. The length is used to parse the slice address from the slice header. Sub-image IDs are used to map stripe addresses from image-based locations to sub-image-based locations. This ensures the decoder consistently determines all relevant addresses regardless of which sub-image is received or its position relative to the top-left corner of the complete image. Furthermore, this mechanism allows these decisions to be made without rewriting the stripe header to change the stripe address values ​​and / or altering the byte alignment fields associated with the stripe addresses. This mechanism improves the encoder, decoder, and / or related slicers. For example, sub-streams can be extracted and sent instead of the entire stream, reducing network, memory, and / or processing resource usage. Moreover, extracting sub-streams avoids rewriting each stripe header for every user request, further reducing network, memory, and / or processing resource usage.

[0054] Figure 1 This is a flowchart of an exemplary method 100 for encoding a video signal. Specifically, an encoder encodes the video signal. During the encoding process, various mechanisms are employed to compress the video signal to reduce the video file size. The smaller file size allows the compressed video file to be sent to the user while reducing associated bandwidth overhead. A decoder then decodes the compressed video file to reconstruct the original video signal for display to the end user. The decoding process is typically the reverse of the encoding process, ensuring that the video signal reconstructed by the decoder is consistent with the video signal on the encoder side.

[0055] In step 101, a video signal is input to an encoder. For example, the video signal may be an uncompressed video file stored in memory. Alternatively, the video file may be captured by a video capture device (e.g., a camera) and encoded to support live video streaming. The video file may include audio and video components. The video components comprise a series of image frames. Viewing these image frames sequentially creates a visual effect of motion. These frames include pixels represented by light rays, referred to herein as luminance components (or luminance samples), and pixels represented by color, referred to herein as chrominance components (or color samples). In some examples, these frames may also include depth values ​​to support three-dimensional viewing.

[0056] In step 103, the video is segmented into blocks. Segmentation involves subdividing the pixels in each frame into square and / or rectangular blocks for compression. For example, in High Efficiency Video Coding (HEVC) (also known as H.265 and MPEG-H Part 2), frames can first be divided into coding tree units (CTUs), which are blocks of a predefined size (e.g., 64 pixels × 64 pixels). These CTUs include luma and chroma samples. CTUs can be divided into blocks using a coding tree, and these blocks can be repeatedly subdivided until a configuration that supports further coding is obtained. For example, the luma components of a frame can be subdivided until the blocks contain relatively uniform luma values. Similarly, the chroma components of a frame can be subdivided until the blocks contain relatively uniform chroma values. Therefore, the segmentation mechanism varies depending on the content of the video frame.

[0057] In step 105, various compression mechanisms are used to compress the image patches segmented in step 103. For example, inter-frame prediction and / or intra-frame prediction can be employed. Inter-frame prediction leverages the fact that objects in common scenes often appear in consecutive frames. Therefore, the block representing an object in a reference frame does not need to be repeatedly represented in adjacent frames. Specifically, an object (e.g., a table) may maintain a fixed position across multiple frames. Therefore, the table is represented once, and adjacent frames can rereference the reference frame. Pattern matching mechanisms can be used to match objects across multiple frames. Furthermore, moving objects can be represented across multiple frames due to object movement or camera movement. In a particular example, video could show a car moving across the screen across multiple frames. Motion vectors can be used to represent this movement. A motion vector is a two-dimensional vector that provides the offset between the object's coordinates in one frame and its coordinates in the reference frame. Therefore, inter-frame prediction can encode image patches in the current frame into a set of motion vectors that indicate the offset between the image patch in the current frame and the corresponding patch in the reference frame.

[0058] Intra-frame prediction is used to encode blocks within a common frame. Intra-frame prediction leverages the fact that luma and chroma components tend to cluster within a frame. For example, a patch of green in a section of a tree is often adjacent to several similar patches of green. Intra-frame prediction employs various directional prediction modes (e.g., 33 in HEVC), planar modes, and direct current (DC) modes. These directional modes indicate that the sample of the current block is similar / identical to the sample of its neighboring blocks in the corresponding direction. Planar modes indicate that a series of blocks in a row / column (e.g., a plane) can be interpolated based on neighboring blocks at the edges of that row. Planar modes essentially represent light / color transitions smoothly between rows / columns using a relatively constant slope with numerical variations. DC modes are used for boundary smoothing, indicating that a block is similar / identical to the average of the samples of all its neighboring blocks, which are angularly related to the directional prediction modes. Therefore, intra-frame predicted blocks can represent image blocks as values ​​of various relational prediction modes rather than actual values. Furthermore, inter-frame predicted blocks can represent image blocks as motion vector values ​​rather than actual values. In either case, the predicted block may not accurately represent the image block in some situations. All differences are stored in residual blocks. These residual blocks can be transformed to further compress the file.

[0059] In step 107, various filtering techniques can be used. In HEVC, filters are used according to the in-loop filtering scheme. The block-based prediction described above may produce a blocky image on the decoder side. Furthermore, the block-based prediction scheme can encode blocks and then reconstruct the encoded blocks for subsequent use as reference blocks. The in-loop filtering scheme iteratively applies noise suppression filters, deblocking filters, adaptive loop filters, and sample adaptive offset (SAO) filters to blocks / frames. These filters reduce block artifacts, allowing for accurate reconstruction of the encoded file. Furthermore, these filters reduce artifacts in the reconstructed reference block, making it less likely that artifacts will generate other artifacts in subsequent blocks encoded based on the reconstructed reference block.

[0060] Once the video signal has been segmented, compressed, and filtered, the resulting data is encoded into a bitstream in step 109. The bitstream includes the data described above, as well as any indicative data required to support proper video signal reconstruction at the decoder side. For example, this data may include segmentation data, prediction data, residual blocks, and various flags that provide encoding instructions to the decoder. The bitstream may be stored in memory for transmission to the decoder upon request. The bitstream may also be broadcast and / or multicast to multiple decoders. The generation of the bitstream is an iterative process. Therefore, steps 101, 103, 105, 107, and 109 can be performed consecutively and / or simultaneously across multiple frames and blocks. Figure 1 The order shown is for clarity and ease of discussion, and is not intended to limit the video decoding process to a specific order.

[0061] In step 111, the decoder receives the bitstream and begins the decoding process. Specifically, the decoder uses an entropy decoding scheme to convert the bitstream into corresponding syntax data and video data. In step 111, the decoder uses the syntax data in the bitstream to determine the segmented portions of frames. The segmentation should match the result of block segmentation in step 103. The entropy encoding / decoding used in step 111 is described below. The encoder makes many choices during compression, such as selecting a block segmentation scheme from several possible options based on the spatial location of values ​​in one or more input images. Indicating the exact choice may use a large number of binary symbols (bins). As used herein, a "binary symbol" is a binary value as a variable (e.g., a bit value that may vary depending on the context). Entropy encoding causes the encoder to discard any options that are obviously unsuitable for a particular situation, leaving a set of available options. A codeword is then assigned to each available option. The length of the codeword depends on the number of available options (e.g., one binary symbol corresponds to two options, two binary symbols correspond to three to four options, and so on). The encoder then encodes the codewords for the selected options. This scheme reduces the codeword size because the codeword is as large as expected, thus uniquely indicating a selection from a small subset of available options, rather than uniquely indicating a selection from a potentially large set of all possible options. The decoder then decodes the options by determining this set of available options in a similar manner to the encoder. By determining this set of available options, the decoder can read the codeword and determine the selection made by the encoder.

[0062] In step 113, the decoder performs block decoding. Specifically, the decoder performs an inverse transform to generate residual blocks. Then, the decoder uses the residual blocks and corresponding prediction blocks to reconstruct image blocks based on the segmentation. The prediction blocks may include intra-frame prediction blocks and inter-frame prediction blocks generated by the encoder in step 105. The reconstructed image blocks are then placed in frames of the reconstructed video signal based on the segmentation data determined in step 111. The syntax for step 113 can also be indicated in the bitstream via entropy coding as described above.

[0063] In step 115, the frames of the reconstructed video signal are filtered in a manner similar to step 107 on the encoder side. For example, noise suppression filters, deblocking filters, adaptive loop filters, and SAO filters can be applied to the frames to remove block artifacts. Once the frames have been filtered, in step 117, the video signal can be output to a display for viewing by the end user.

[0064] Figure 2This is a schematic diagram of an exemplary encoding and decoding (encoding / decoding) system 200 for video decoding. Specifically, the encoding / decoding system 200 functions to implement operation method 100. The encoding / decoding system 200 is generally applicable to describing components used in both encoders and decoders. The encoding / decoding system 200 receives a video signal and segments it, as described in steps 101 and 103 of operation method 100, to obtain a segmented video signal 201. When acting as an encoder, the encoding / decoding system 200 compresses the segmented video signal 201 into an encoded bitstream, as described in steps 105, 107, and 109 of method 100. When acting as a decoder, the encoding / decoding system 200 generates an output video signal from the bitstream, as described in steps 111, 113, 115, and 117 of operation method 100. The encoding / decoding system 200 includes a general decoder control component 211, a transform scaling and quantization component 213, an intra-frame estimation component 215, an intra-frame prediction component 217, a motion compensation component 219, a motion estimation component 221, a scaling and inverse transform component 229, a filter control and analysis component 227, an intra-loop filter component 225, a decoded image buffer component 223, and a header format and context-adaptive binary arithmetic coding (CABAC) component 231. These components are coupled as shown in the figure. Figure 2 In the diagram, black lines represent the motion of the data to be encoded / decoded, while dashed lines represent the motion of control data that controls the operation of other components. All components in the encoding / decoding system 200 can reside in the encoder. The decoder may include a subset of the components in the encoding / decoding system 200. For example, the decoder may include an intra-frame prediction component 217, a motion compensation component 219, a scaling and inverse transform component 229, an in-loop filter component 225, and a decoded image buffer component 223. These components are described below.

[0065] The segmented video signal 201 is a captured video sequence that has been segmented into pixel blocks by a coding tree. The coding tree uses various partitioning modes to further subdivide the pixel blocks into smaller pixel blocks. These blocks can then be further subdivided into smaller blocks. These blocks can be called nodes on the coding tree. Larger parent nodes are divided into smaller child nodes. The number of times a node is subdivided is called the node / coding tree depth. In some cases, the resulting blocks can be included in a coding unit (CU). For example, a CU can be a sub-part of a CTU, including a luma block, one or more red chromatic aberration (Cr) blocks, and one or more blue chromatic aberration (Cb) blocks, as well as the syntax instructions corresponding to the CU. Partitioning modes can include binary trees (BT), triple trees (TT), and quad trees (QT) used to divide nodes into two, three, or four child nodes of different shapes (depending on the partitioning mode used). The segmented video signal 201 is sent to the overall encoder control component 211, the transform scaling and quantization component 213, the intra-frame estimation component 215, the filter control analysis component 227, and the motion estimation component 221 for compression.

[0066] The overall decoder control component 211 is used to determine how to encode images from a video sequence into the bitstream based on application constraints. For example, the overall decoder control component 211 manages optimizations for bitrate / bitstream size and reconstruction quality. These decisions can be made based on storage space / bandwidth availability and image resolution requests. The overall decoder control component 211 also manages buffer utilization based on transmission speed to mitigate buffer underloading and overloading. To address these issues, the overall decoder control component 211 manages segmentation, prediction, and filtering performed by other components. For example, the overall decoder control component 211 can dynamically increase compression complexity to improve resolution and bandwidth utilization, or decrease compression complexity to reduce resolution and bandwidth utilization. Therefore, the overall decoder control component 211 controls other components in the encoding / decoding system 200 to balance video signal reconstruction quality and bitrate. The overall decoder control component 211 generates control data that controls the operation of other components. This control data is also sent to the header format and CABAC component 231 for encoding into the bitstream, thereby instructing the decoder to use the parameters for decoding.

[0067] The segmented video signal 201 is also sent to the motion estimation component 221 and the motion compensation component 219 for inter-frame prediction. Frames or stripes in the segmented video signal 201 can be divided into multiple video blocks. The motion estimation component 221 and the motion compensation component 219 perform inter-frame prediction coding on the received video blocks relative to one or more blocks in one or more reference frames to provide temporal prediction. The encoding / decoding system 200 can perform multiple decoding rounds to select an appropriate encoding mode for each block in the video data, and so on.

[0068] Motion estimation component 221 and motion compensation component 219 can be highly integrated, but are described separately for conceptual purposes. Motion estimation performed by motion estimation component 221 is the process of generating motion vectors, which are used to estimate the motion of video blocks. For example, motion vectors can represent the displacement of an encoded object relative to a prediction block. A prediction block is a block that closely matches the block to be encoded in terms of pixel differences. A prediction block can also be called a reference block. This pixel difference can be determined using the sum of absolute differences (SAD), sum of square differences (SSD), or other difference measures. HEVC employs several encoded objects, including CTUs, coding tree blocks (CTBs), and CUs. For example, a CTU can be divided into a CTB, and then the CTB can be divided into CBs to be included in a CU. CUs can be encoded as prediction units (PUs) that include prediction data and / or transform units (TUs) that include transform residual data of the CUs. Motion estimation component 221 performs rate-distortion analysis as part of a rate-distortion optimization process, generating motion vectors, PUs, and TUs. For example, the motion estimation component 221 can determine multiple reference blocks, multiple motion vectors, etc. for the current block / frame, and can select the reference blocks, motion vectors, etc. with the best rate-distortion characteristics. The best rate-distortion characteristics can maintain a balance between the quality of video reconstruction (e.g., the amount of data loss caused by compression) and coding efficiency (e.g., the size of the final code).

[0069] In some examples, the codec system 200 can calculate the values ​​of sub-integer pixel positions of the reference image stored in the decoded image buffer component 223. For example, the video codec system 200 can interpolate the reference image at quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions. Therefore, the motion estimation component 221 can perform motion search relative to the integer pixel positions and fractional pixel positions, and output motion vectors with fractional pixel precision. The motion estimation component 221 compares the position of the PU with the position of the predicted block in the reference image to calculate the motion vector for the PU of the video block in the inter-frame coded strip. The motion estimation component 221 outputs the calculated motion vector as motion data to the header format and CABAC component 231 for encoding, and also outputs it as motion data to the motion compensation component 219.

[0070] Motion compensation performed by motion compensation component 219 may include acquiring or generating prediction blocks based on motion vectors determined by motion estimation component 221. Similarly, in some examples, motion estimation component 221 and motion compensation component 219 may be functionally integrated. After receiving the motion vector of the PU for the current video block, motion compensation component 219 can locate the prediction block to which the motion vector points. The pixel values ​​of the prediction block are then subtracted from the pixel values ​​of the currently encoded video block to obtain the pixel difference, thereby forming a residual video block. Generally, motion estimation component 221 performs motion estimation relative to the luma component, and motion compensation component 219 uses the motion vectors calculated based on the luma component for both the chroma and luma components. The prediction block and residual block are then sent to transform scaling and quantization component 213.

[0071] The segmented video signal 201 is also sent to the intra-frame estimation component 215 and the intra-frame prediction component 217. Similar to the motion estimation component 221 and the motion compensation component 219, the intra-frame estimation component 215 and the intra-frame prediction component 217 can be highly integrated, but are described separately for conceptual purposes. The intra-frame estimation component 215 and the intra-frame prediction component 217 perform intra-frame prediction relative to the blocks in the current frame, instead of the inter-frame prediction performed between frames by the motion estimation component 221 and the motion compensation component 219 as described above. Specifically, the intra-frame estimation component 215 determines an intra-frame prediction mode to encode the current block. In some examples, the intra-frame estimation component 215 selects a suitable intra-frame prediction mode from multiple tested intra-frame prediction modes to encode the current block. The selected intra-frame prediction mode is then sent to the header format and CABAC component 231 for encoding.

[0072] For example, the intra-frame estimation component 215 performs rate-distortion analysis on various tested intra-frame prediction modes, calculates rate-distortion values, and selects the intra-frame prediction mode with the best rate-distortion characteristics among the tested modes. Rate-distortion analysis is generally used to determine the amount of distortion (or error) between a coded block and the original uncoded block encoded to produce the coded block, and to determine the bit rate (e.g., number of bits) used to generate the coded block. The intra-frame estimation component 215 calculates the ratio based on the distortion and rate of various coded blocks to determine the intra-frame prediction mode with the optimal rate-distortion value for obtaining the block. Additionally, the intra-frame estimation component 215 can be used to encode depth blocks in a depth image using a depth-modeling mode (DMM) based on rate-distortion optimization (RDO).

[0073] When implemented on the encoder, the intra-prediction component 217 can generate residual blocks from the prediction blocks based on the intra-prediction mode determined by the intra-estimation component 215, or, when implemented on the decoder, read residual blocks from the bitstream. The residual blocks comprise the difference between the prediction blocks and the original blocks, represented as a matrix. The residual blocks are then sent to the transform-scaling and quantization component 213. The intra-estimation component 215 and the intra-prediction component 217 can operate on the luma and chroma components.

[0074] Transform scaling and quantization component 213 is used to further compress the residual block. Transform scaling and quantization component 213 performs transforms on the residual block such as discrete cosine transform (DCT), discrete sine transform (DST), or conceptually similar transforms, thereby producing video blocks that include the residual transform coefficient values. Wavelet transform, integer transform, subband transform, or other types of transforms can also be performed. Transforms can convert residual information from the pixel value domain to the transform domain, such as the frequency domain. Transform scaling and quantization component 213 is also used to scale the transform residual information according to frequencies, etc. This scaling involves applying a scaling factor to the residual information to quantize different frequency information at different granularities, which may affect the final visual quality of the reconstructed video. Transform scaling and quantization component 213 is also used to quantize the transform coefficients to further reduce the bit rate. The quantization process can reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be changed by adjusting the quantization parameters. In some examples, transform scaling and quantization component 213 can then perform a scan on the matrix including the quantized transform coefficients. The quantization transform coefficients are sent to the header format and CABAC component 231 for encoding in the bitstream.

[0075] The scaling and inverse transform component 229 performs the opposite operation to the transform scaling and quantization component 213 to support motion estimation. The scaling and inverse transform component 229 performs inverse scaling, inverse transform, and / or inverse quantization to reconstruct a residual block in the pixel domain, which can then be used as a reference block. This reference block may become the prediction block for another current block. The motion estimation component 221 and / or the motion compensation component 219 can add the residual block back to the corresponding prediction block to compute the reference block for motion estimation of subsequent blocks / frames. Filters are applied to the reconstructed reference block to reduce artifacts generated during scaling, quantization, and transform. These artifacts can cause inaccurate predictions (and generate additional artifacts) when predicting subsequent blocks.

[0076] The filter control analysis component 227 and the in-loop filter component 225 apply filters to residual blocks and / or reconstructed image blocks. For example, a transformed residual block from the scaling and inverse transform component 229 can be combined with a corresponding predicted block from the intra-prediction component 217 and / or the motion compensation component 219 to reconstruct the original image block. The filter can then be applied to the reconstructed image block. In some examples, the filter can be applied to the residual block. Figure 2 Like other components, the filter control analysis component 227 and the in-loop filtering component 225 are highly integrated and can be implemented together, but for conceptual purposes, they are described separately. Filters applied to the reconstructed reference block are applied to specific spatial regions; these filters include multiple parameters to adjust how they are used. The filter control analysis component 227 analyzes the reconstructed reference block to determine the locations where these filters need to be used and sets the corresponding parameters. This data is sent to the header format and CABAC component 231 for encoding as filter control data. The in-loop filtering component 225 uses these filters according to the filter control data. These filters may include deblocking filters, noise suppression filters, SAO filters, and adaptive loop filters. These filters may be applied in the spatial / pixel domain (e.g., reconstructing pixel blocks) or the frequency domain, depending on the example.

[0077] When operating as an encoder, the filtered reconstructed image blocks, residual blocks, and / or prediction blocks are stored in the decoded image buffer component 223 for subsequent use in motion estimation as described above. When operating as a decoder, the decoded image buffer component 223 stores the filtered reconstructed blocks and sends them to the display as part of the output video signal. The decoded image buffer component 223 can be any storage device capable of storing prediction blocks, residual blocks, and / or reconstructed image blocks.

[0078] The header format and CABAC component 231 receive data from various components in the encoding / decoding system 200 and encode this data in an encoded bitstream for transmission to the decoder. Specifically, the header format and CABAC component 231 generate various headers to encode control data (e.g., overall control data and filter control data). Furthermore, prediction data (including intra-frame prediction and motion data) and residual data in the form of quantized transform coefficients are encoded in the bitstream. The final bitstream contains all the information required by the decoder to reconstruct the original segmented video signal 201. This information may also include an intra-frame prediction mode index table (also called a codeword map), definitions of the coding context for various blocks, indications of the most likely intra-frame prediction modes, indications of segmentation information, etc. This data may be encoded using entropy coding. For example, the above information can be encoded using context adaptive variable length coding (CAVLC), CABAC, syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding techniques. After entropy coding, the encoded bitstream can be sent to another device (e.g., a video decoder) or archived for subsequent transmission or retrieval.

[0079] Figure 3 This is a block diagram of an exemplary video encoder 300. The video encoder 300 can be used to implement the encoding function of the encoding / decoding system 200 and / or perform steps 101, 103, 105, 107, and / or 109 in the operation method 100. The encoder 300 segments the input video signal to obtain a segmented video signal 301 that is substantially similar to the segmented video signal 201. Then, components in the encoder 300 compress and encode the segmented video signal 301 in a bitstream.

[0080] Specifically, the segmented video signal 301 is sent to the intra-prediction component 317 for intra-frame prediction. The intra-prediction component 317 can be substantially similar to the intra-estimation component 215 and the intra-prediction component 217. The segmented video signal 301 is also sent to the motion compensation component 321 for inter-frame prediction based on the reference block in the decoded image buffer component 323. The motion compensation component 321 can be substantially similar to the motion estimation component 221 and the motion compensation component 219. The predicted block and residual block from the intra-prediction component 317 and the motion compensation component 321 are sent to the transform and quantization component 313 for transforming and quantizing the residual block. The transform and quantization component 313 can be substantially similar to the transform scaling and quantization component 213. The transform quantized residual block and the corresponding predicted block (along with associated control data) are sent to the entropy coding component 331 for encoding in the bitstream. The entropy coding component 331 can be substantially similar to the header format and CABAC component 231.

[0081] The transformed and quantized residual block and / or the corresponding prediction block are also sent from the transform and quantization component 313 to the inverse transform and quantization component 329 to reconstruct a reference block for use by the motion compensation component 321. The inverse transform and quantization component 329 can be substantially similar to the scaling and inverse transform component 229. According to the example, an intra-loop filter in the intra-loop filter component 325 is also applied to the residual block and / or the reconstructed reference block. The intra-loop filter component 325 can be substantially similar to the filter control analysis component 227 and the intra-loop filter component 225. The intra-loop filter component 325 may include multiple filters, as described in conjunction with the intra-loop filter component 225. The filtered block is then stored in the decoded image buffer component 323 as a reference block for use by the motion compensation component 321. The decoded image buffer component 323 can be substantially similar to the decoded image buffer component 223.

[0082] Figure 4 This is a block diagram of an exemplary video decoder 400. The video decoder 400 can be used to implement the decoding function of the encoding / decoding system 200 and / or perform steps 111, 113, 115, and / or 117 in the operation method 100. The decoder 400 receives a bitstream from the encoder 300, etc., and generates a reconstructed output video signal based on the bitstream for display to the end user.

[0083] The bitstream is received by entropy decoding component 433. Entropy decoding component 433 performs entropy decoding schemes, such as CAVLC, CABAC, SBAC, PIPE encoding, or other entropy coding techniques. For example, entropy decoding component 433 can use header information to provide context for parsing other data encoded as codewords in the bitstream. The aforementioned decoding information includes any information required to decode the video signal, such as overall control data, filter control data, segmentation information, motion data, prediction data, and quantization transform coefficients in the residual block. The quantization transform coefficients are sent to inverse transform and quantization component 429 to reconstruct the residual block. Inverse transform and quantization component 429 may be similar to inverse transform and quantization component 329.

[0084] The reconstructed residual block and / or predicted block are sent to the intra-prediction component 417 to reconstruct image blocks according to the intra-prediction operation. The intra-prediction component 417 can be similar to the intra-estimation component 215 and the intra-prediction component 217. Specifically, the intra-prediction component 417 uses a prediction mode to locate a reference block in the frame and applies the residual block to the above result to reconstruct the intra-predicted image block. The reconstructed intra-predicted image block and / or residual block, along with the corresponding inter-frame prediction data, are sent to the decoding image buffer component 423 via the intra-loop filter component 425. The decoding image buffer component 423 and the intra-loop filter component 425 can be substantially similar to the decoding image buffer component 223 and the intra-loop filter component 225, respectively. The intra-loop filter component 425 filters the reconstructed image block, residual block, and / or predicted block. This information is stored in the decoding image buffer component 423. The reconstructed image block from the decoding image buffer component 423 is sent to the motion compensation component 421 for inter-frame prediction. Motion compensation component 421 can be substantially similar to motion estimation component 221 and / or motion compensation component 219. Specifically, motion compensation component 421 uses the motion vector of a reference block to generate a prediction block and applies the residual block to the result to reconstruct an image block. The resulting reconstructed block can also be sent to decoding image buffer component 423 via in-loop filter component 425. Decoding image buffer component 423 continues to store other reconstructed image blocks. These reconstructed image blocks can be reconstructed into frames using segmentation information. These frames can also be placed in a sequence. The sequence is output to a display as a reconstructed output video signal.

[0085] Figure 5 This is a schematic diagram of an exemplary bitstream 500 comprising an encoded video sequence. For example, bitstream 500 may be generated by codec system 200 and / or encoder 300, and decoded by codec system 200 and / or decoder 400. As another example, bitstream 500 may be generated by encoder in step 109 of method 100 and used by decoder in step 111.

[0086] Bitstream 500 includes a sequence parameter set (SPS) 510, multiple picture parameter sets (PPS) 512, multiple strip headers 514, and image data 520. SPS 510 includes sequence data shared by all images in the video sequence included in bitstream 500. This data may include image size, bit depth, coding tool parameters, bitrate constraints, etc. PPS 512 includes one or more image-specific parameters. Therefore, each image in the video sequence can refer to a PPS 512. PPS 512 may represent the coding tools, quantization parameters, offsets, and image-specific coding tool parameters (e.g., filter control parameters) available for the corresponding blocks in the image. Strip headers 514 include one or more parameters specific to a corresponding strip in the image. Therefore, each strip in the video sequence can refer to a strip header 514. Strip headers 514 may include strip type information, picture order count (POC), a list of reference images, prediction weights, block entry points, deblocking filter parameters, etc. In some examples, a stripe may be referred to as a chunk group. In this case, the stripe header 514 may be referred to as the chunk group header.

[0087] Image data 520 includes video data encoded according to inter-frame prediction and / or intra-frame prediction, as well as corresponding transform and quantization residual data. This image data 520 is ordered according to the segmentation pattern used to segment the images prior to encoding. For example, a video sequence is divided into images 521, images 521 are divided into stripes 523, stripes 523 can be further divided into blocks and / or CTUs, CTUs are further divided into coding blocks according to a coding tree, and then the coding blocks can be encoded / decoded according to a prediction mechanism. For example, images 521 may include one or more stripes 523. Images 521 refer to PPS 512, and stripes 523 refer to strip header 514. Each stripe 523 may include one or more blocks. Each stripe 523 and / or image 521 may include multiple CTUs.

[0088] Each image 521 may include the entire visual dataset associated with the video sequence at the corresponding moment. The VR system can display the region selected by the user in image 521. This creates a feeling of being in the scene depicted by image 521. When the bitstream 500 is encoded, the region the user might want to view is unknown. Therefore, image 521 may include every possible region the user is likely to view. However, in a VR scene, the corresponding codec can be designed based on the assumption that the user only views the region selected in image 521 and discards the rest of image 521.

[0089] Each strip 523 can be a rectangle defined by the top-left CTU and the bottom-right CTU. In some examples, strip 523 includes a series of blocks and / or CTUs that are raster scanned from left to right and from top to bottom. In other examples, strip 523 is a rectangular strip. A rectangular strip may not traverse the entire width of the image in raster scan order. Instead, a rectangular strip may include rectangular and / or square regions in image 521 defined by CTUs and / or block rows and CTUs and / or block columns. Strip 523 is the smallest unit that the decoder can display individually. Therefore, strips 523 in image 521 can be assigned to different sub-images 522 to depict desired areas of image 521 respectively. For example, in a VR scene, image 521 may include the entire visible data range, but the user can only view sub-images 522 that include one or more strips 523 on a head-mounted display.

[0090] As described above, the video codec can assume that non-selected regions of image 521 will be discarded on the decoder side. Therefore, a sub-stream 501 can be extracted from the bitstream 500. The extracted sub-stream 501 may include selected sub-images 522 and associated syntax. The non-selected regions of image 521 may be sent at a lower resolution or ignored to improve coding efficiency. Sub-image 522 is the selected region of image 521 and may include one or more associated stripes 524. Stripes 524 are subsets of stripes 523 that depict the selected regions of image 521 associated with sub-image 522. Sub-stream 501 also includes SPS 510, PPS 512, stripe header 514, and / or sub-parts therein associated with sub-images 522 and stripes 524.

[0091] Sub-streams 501 can be extracted from bitstream 500. For example, a user using a decoder can watch a video. The user can select a corresponding region of image 521. The decoder can request subsequent sub-images 522 associated with the region the user is currently watching. The encoder can then send the sub-image 522 associated with the selected region at a higher resolution and the remaining region of image 521 at a lower resolution. To achieve this functionality, the decoder can extract 529 one or more sub-streams 501 from bitstream 500. Extraction 529 includes storing the sub-images 522 (including stripes 524 within the sub-images 522) in sub-stream 501. Extraction 529 also includes storing associated SPS 510, PPS 512, and stripe headers 514 in the sub-stream as needed to support decoding of sub-images 522 and stripes 524.

[0092] One problem with extracting 529 from sub-stream 501 is that addressing relative to image 521 may differ from addressing relative to sub-image 522. The addressing problem is described in detail below. In some systems, the stripe header 514 may be rewritten to accommodate these addressing differences. However, sub-stream 501 may include many stripe headers 514 (e.g., one or two orders of magnitude for each image 521), and these stripe headers 514 are dynamically rewritten for each user. Therefore, rewriting the stripe headers 514 in this way may require significant computation from the processor. This invention includes a mechanism that allows extracting 529 from the stripe headers 514 into sub-stream 501 without rewriting them.

[0093] In the system that rewrites the stripe header 514, stripes 523 and 524 are addressed based on index values ​​such as stripe index, block index, and CTU index. These index values ​​increment sequentially according to the raster scan order. To address the addressing mismatch problem, the disclosed embodiments use ID values ​​defined for each stripe, block, and / or CTU. These defined IDs can be default values ​​and / or selected by the encoder. The defined IDs can uniformly increment sequentially according to the raster scan order, but they do not have to be monotonically increasing. Therefore, the defined IDs can have value gaps to enable address management. For example, the index can be monotonically increasing (e.g., 0, 1, 2, 3, etc.), while the defined IDs can increment by a defined multiple (e.g., 0, 10, 20, 30, etc.). The encoder can carry the mapping relationship 535 in the bitstream 500 and sub-bitstream 501, so that the decoder can map the defined IDs to indices that the decoder can parse.

[0094] The parameter set, such as SPS 510 and / or PPS 512, may include an ID flag 531. The ID flag 531 may be configured to indicate that a mapping relationship 535 can be used to map a stripe address from a position based on image 521 to a position based on sub-image 522. Accordingly, the ID flag 531 may be configured to indicate to the decoder the public mechanism used in bitstream 500 and sub-bitstream 501. For example, the ID flag 531 may be encoded as an explicit chunk ID flag, `sps_subpic_id_present_flag`, or other syntax elements. The ID flag 531 may be encoded in bitstream 500 and extracted 529 into sub-bitstream 501.

[0095] The parameter set, such as SPS 510 and / or PPS 512, may also include the syntax element ID 532. ID 532 may indicate a sub-image 522 within image 521. For example, some IDs 532 may be included in the PPS 512 of bitstream 500. When sub-bitstream 501 is extracted (529), one or more IDs 532 associated with one or more sub-images 522 to be sent to the decoder may be included in the PPS 512 of sub-bitstream 501. In other examples, a point pointing to the associated ID 532 may be inserted into the PPS 512 of sub-bitstream 501 so that the decoder can determine the correct ID 532. For example, ID 532 may be encoded as SubPicIdx, Tile_id_val[i], or other syntax elements indicating the boundaries of sub-image 522.

[0096] The parameter sets such as SPS 510 and / or PPS 512 may also include the syntax element slice address length 533. Additionally, the slice header 514 may include the slice address 534 of slice 523. The slice address 534 is included as a defined ID value. The slice address 534 can be directly extracted 529 into the slice header 514 of the sub-stream 501 without modification to avoid rewriting the slice header 514. For example, the slice address 534 can be encoded as slice_address, tile_group_address, or other syntax elements indicating the boundaries of slice 523 and slice 524. The slice address length 533 can then be used to parse the slice address 534. For example, the slice address 534 includes values ​​defined by the encoder, and is therefore encoded as a variable-length value before the byte alignment field. The slice address length 533 can indicate the number of bits included in the corresponding slice address 534, thus indicating the boundaries of the slice address 534 to the decoder. Therefore, the decoder can use the stripe address length 533 (e.g., from PPS 512) to resolve stripe address 534. Thus, the stripe header 514 does not need to be rewritten to adjust the byte alignment field after stripe address 534. For example, stripe address length 533 can be encoded as subpic_id_len_minus1, tile_id_len_minus1, or other syntax elements indicating stripe address length 533. Stripe address length 533 can be included in the PPS 512 of bitstream 500, and then extracted to the PPS 512 of sub-bitstream 501.

[0097] Mapping relation 535 can also be sent via parameter sets such as SPS 510, PPS 512, and / or slice header 514. Mapping relation 535 represents a mechanism for mapping slice addresses from a position based on image 521 to a position based on sub-image 522. Mapping relation 535 can be encoded in bitstream 500 and extracted 529 to the corresponding parameter set of sub-bitstream 501. For example, mapping relation 535 can be encoded as the syntax element SliceSubpicToPicIdx[SubPicIdx][slice_address], the syntax element tileIdToIdx[Tile_group_address], or other syntax elements representing a mechanism for mapping slice addresses from a position based on image 521 to a position based on sub-image 522.

[0098] Accordingly, the decoder can read sub-stream 501 and obtain ID flag 531 to determine that stripe 524 is addressed by a defined address rather than an index. The decoder can obtain ID 532 to determine the sub-image 522 included in sub-stream 501. The decoder can also obtain one or more stripe addresses 534 and stripe address length 533 to parse one or more stripe addresses 534. Then, the decoder can obtain mapping relation 53 to map one or more stripe addresses 534 to a format that the decoder can parse. Finally, the decoder can use one or more stripe addresses 534 when decoding and displaying sub-image 522 and corresponding stripe 524.

[0099] Figure 6 This is a schematic diagram of segmenting an exemplary image 600 for encoding. For example, image 600 may be encoded in and decoded from bitstream 500 by encoding / decoding system 200, encoder 300 and / or decoder 400, etc. Furthermore, image 600 may be segmented into and / or included in sub-images of sub-bitstream 501 to implement the encoding and decoding in method 100.

[0100] Image 600 can be segmented into strips 623, which are substantially similar to strip 523. Strip 623 can be further segmented into blocks 625 and CTU 627. Figure 6 In the image, stripes 623 are represented by thick lines, graphically distinguished by alternating white backgrounds and hashes. Blocks 625 are represented by dashed lines. Block boundaries on the boundaries of stripes 623 are shown as thick dashed lines, while those not on the boundaries of stripes 623 are shown as thin dashed lines. CTU 627 boundaries are shown as thin solid lines, except for CTU 627 boundaries covered by either block boundaries or stripe boundaries. In this example, image 600 includes 9 stripes 623, 24 blocks 625, and 216 CTUs 627.

[0101] As shown in the figure, stripe 623 is a rectangle with boundaries, which may be defined by the included blocks 625 and / or CTU 627. Stripe 623 may not span the entire width of image 600. Blocks 625 can be generated in stripe 623 according to rows and columns. CTU 627 can be segmented from blocks 625 and / or stripe 623 to produce segmented portions of image 600, which can be further subdivided into coding blocks for encoding according to inter-frame prediction and / or intra-frame prediction. Image 600 can be encoded in a bitstream such as bitstream 500. Regions of image 600 can be included in sub-images and extracted into sub-images 522 and sub-bitstreams such as sub-bitstream 501.

[0102] Figure 7 This is a schematic diagram of an exemplary sub-image 722 extracted from image 700. For example, image 700 may be substantially similar to image 600. Furthermore, image 700 may be encoded in bitstream 500 by encoding / decoding system 200 and / or encoder 300, etc. Sub-image 722 may be extracted into and decoded from sub-bitstream 501 by encoding / decoding system 200, encoder 300, and / or decoder 400, etc. Additionally, image 700 may be used to implement the encoding and decoding in method 100.

[0103] As shown in the figure, image 700 includes a top-left corner 702 and a bottom-right corner 704. Sub-image 722 includes one or more stripes 723 in image 700. When using indexing, the top-left corner 702 and the bottom-right corner 704 are associated with the first and last indexes, respectively. However, the decoder can only display sub-image 722, not the entire image 700. Furthermore, the stripe address 734 of the first stripe 723a may not be aligned with the top-left corner 702, and the stripe address 734 of the third stripe 723c may not be aligned with the bottom-right corner 704. Therefore, the stripe address 734 relative to sub-image 722 is not aligned with the stripe address 734 relative to image 700. This invention uses an ID defined for stripe address 734 instead of an index. The decoder can use a mapping relationship to map the stripe address 734 from a position based on image 700 to a position based on sub-image 722. Then, the decoder can use the mapped stripe address 734 to place the first stripe 723a in the upper left corner 702 of the decoder display, place the third stripe 723c in the lower left corner 704 of the decoder display, and place the second stripe 723b between the first stripe 723a and the third stripe 723c.

[0104] As described herein, this invention describes an improvement to explicit chunk ID indication in video decoding, where chunking is used for image segmentation. The above technical description is based on VVC developed by ITU-T and ISO / IEC JVET. However, these techniques are also applicable to other video codec specifications. Exemplary embodiments described herein are as follows.

[0105] The concepts of chunk index and chunk ID are distinct. A chunk's chunk ID may or may not be equal to its chunk index. When the chunk ID differs from the chunk index, the mapping between them can be indicated in the PPS. Instead of using the chunk index, the chunk ID can be used to indicate the chunk group address in the chunk group header. This ensures the chunk ID value remains consistent when extracting chunk groups from the original bitstream. This is achieved by updating the mapping between the chunk ID and chunk index in the PPS referenced by the chunk group. This method addresses the issue that the chunk index value might change depending on the sub-bitmap being extracted. It should be noted that when performing MCTS-based sub-bitstream extraction, it may still be necessary to rewrite other parameter sets (e.g., parameter sets other than the stripe header).

[0106] The above can be achieved by using flags in a parameter set indicating tile information. For example, PPS can be used as a parameter set. For example, explicit_tile_id_flag can be used for this purpose. explicit_tile_id_flag can be indicated regardless of the number of tiles in the image, and explicit_tile_id_flag can indicate that explicit tile indication is used. Syntax elements can also be used to indicate the number of bits used to indicate the tile ID value (e.g., the mapping between tile index and tile ID). Such syntax elements can also be used to indicate the tile ID / address in the tile group header. For example, the syntax element tile_id_len_minus1 can be used for this purpose. When explicit_tile_id_flag is equal to 0 (e.g., when the tile ID is set to the tile index), tile_id_len_minus1 may not exist. When tile_id_len_minus1 does not exist, it can be inferred that the value of tile_id_len_minus1 is equal to the value of Ceil(Log2(NumTilesInPic)). Another constraint requires that the bitstream extracted as a sub-bitstream by MCTS can include explicit_tile_id_flag with the active PPS set to 1, unless the sub-bitstream includes the top-left block from the original bitstream.

[0107] In an exemplary embodiment, the video coding syntax can be modified as follows to achieve the functionality described herein. An exemplary CTB raster and block scanning process can be described as follows. The list TileId[ctbAddrTs] (where ctbAddrTs ranges from 0 to PicSizeInCtbsY–1, inclusive) represents the conversion from CTB address to block ID under block scanning, and the list NumCtusInTile[tileIdx] (where tileIdx ranges from 0 to PicSizeInCtbsY–1, inclusive) represents the conversion from block index to the number of CTUs in the block. The derivation of both is as follows:

[0108]

[0109] The list NumCtusInTile[tileIdx] (where tileIdx ranges from 0 to PicSizeInCtbsY-1, inclusive) represents the conversion from the tile index to the number of CTUs in the tile, which can be derived as follows:

[0110]

[0111] The set TileIdToIdx[tileId] for a group of NumTilesInPic tileId values ​​represents the conversion from tile ID to tile index, which can be derived as follows:

[0112]

[0113] An exemplary image parameter set RBSP syntax can be described as follows.

[0114]

[0115]

[0116] An example block header syntax can be described as follows.

[0117]

[0118] An exemplary block grouping data syntax can be described as follows.

[0119]

[0120]

[0121] The semantics of an exemplary image parameter set (RBSP) can be described as follows. `explicit_tile_id_flag` is set to 1, indicating that the tile ID for each tile is explicitly indicated. `explicit_tile_id_flag` is set to 0, indicating that the tile ID is not explicitly indicated. For the bitstream extracted as a sub-bitstream from MCTS, the value of `explicit_tile_id_flag` can be set to 1 for activating the PPS, unless the resulting bitstream includes the top-left tile from the original bitstream. `Tile_id_len_minus1+1` represents the number of bits used to represent the `tile_id_val[i]` and `tile_group_address` syntax elements in the tile group header representing the reference PPS. The value of `tile_id_len_minus1` can range from `Ceil(Log2(NumTilesInPic)` to 15 (inclusive). When `tile_id_len_minus1` does not exist, it can be inferred that its value is equal to `Ceil(Log2(NumTilesInPic)`. It should be noted that in some cases, the value of `tile_id_len_minus1` can be greater than `Ceil(Log2(NumTilesInPic)`. This is because the current bitstream may be the result of MCTS sub-bitstream extraction. In this case, the block ID can be the block index in the original bitstream, which can be obtained using `Ceil(Log2(OrgNumTilesInPic)`. Pic is represented by (NumTilesInPic) bits, where OrgNumTilesInPic is the NumTilesInPic of the original bitstream, which is greater than the NumTilesInPic of the current bitstream. tile_id_val[i] represents the tile ID of the i-th tile in the reference PPS image. The length of tile_id_val[i] is (tile_id_len_minus1+1) bits. For any integers m and n in the range of 0 to NumTilesInPic-1 (inclusive), when m is not equal to n, tile_id_val[m] may not be equal to tile_id_val[n], and when m is less than n, tile_id_val[m] may be less than tile_id_val[n].

[0122] The following variables can be derived by calling CTB raster and block scan conversion: The list ColWidth[i] (where i ranges from 0 to num_tile_columns_minus1, inclusive) represents the width of the i-th block column, in CTB units; the list RowHeight[j] (where j ranges from 0 to num_tile_rows_minus1, inclusive) represents the height of the j-th block row, in CTB units; the list ColBd[i] (where i ranges from 0 to num_tile_columns_minus1+1, inclusive) represents the bit position of the i-th block column boundary. The list RowBd[j] (where j ranges from 0 to num_tile_rows_minus1+1, inclusive) represents the position of the j-th block row boundary, in CTB units; the list CtbAddrRsToTs[ctbAddrRs] (where ctbAddrRs ranges from 0 to PicSizeInCtbsY-1, inclusive) represents the conversion of the CTB address under CTB raster scanning to the CTB address under block scanning; the list CtbAddrTsToRs[ctbAddrTs] (where ctbAddrTs ranges from 0 to PicSizeInCtbsY-1) represents the conversion of the CTB address under CTB raster scanning to the CTB address under block scanning. InCtbsY-1 (including end values) represents the conversion from CTB address under block scanning to CTB address under image CTB raster scanning; the list TileId[ctbAddrTs] (ctbAddrTs ranges from 0 to PicSizeInCtbsY-1, including end values) represents the conversion from CTB address under block scanning to block ID; the list NumCtusInTile[tileIdx] (tileIdx ranges from 0 to PicSizeInCtbsY-1, including end values) represents the conversion from block index to the number of CTUs in the block; the list FirstCtbAddrTs[ti `tileIdx` (where `tileIdx` ranges from 0 to NumTilesInPic-1, inclusive) represents the conversion from block ID to the CTB address of the first CTB in a block under block scanning; `TileIdToIdx` (a set of NumTilesInPic `tileId` values) represents the conversion from block ID to block index; `FirstCtbAddrTs` (where `tileIdx` ranges from 0 to NumTilesInPic-1, inclusive) represents the conversion from block ID to the CTB address of the first CTB in a block under block scanning.The list `ColumnWidthInLumaSamples[i]` (where `i` ranges from 0 to `num_tile_columns_minus1`, inclusive) represents the width of the `i`-th column, in units of brightness samples; the list `RowHeightInLumaSamples[j]` (where `j` ranges from 0 to `num_tile_rows_minus1`, inclusive) represents the height of the `j`-th row, in units of brightness samples.

[0123] `tile_group_address` represents the tile ID of the first tile in the tile group. The length of `tile_group_address` is (tile_id_len_minus1+1) bits. The value of `tile_group_address` can range from 0 to 2`tile_id_len_minus1+1-1` (inclusive). The value of `tile_group_address` does not have to be equal to the value of `tile_group_address` of any other NAL unit in the same coded tile group within the same coded image.

[0124] Figure 8 This is a schematic diagram of an exemplary video decoding device 800. The video decoding device 800 is suitable for implementing the disclosed examples / embodiments described herein. The video decoding device 800 includes a downlink port 820, an uplink port 850, and / or a transceiver unit (Tx / Rx) 810. The transceiver unit 810 includes a transmitter and / or a receiver for data communication in the uplink and / or downlink via a network. The video decoding device 800 also includes a processor 830 and a memory 832. The processor 830 includes a logic unit and / or a central processing unit (CPU) for processing data. The memory 832 is used to store the data. The video decoding device 800 may also include electrical components, optical-to-electrical (OE) components, electro-optical (EO) components, and / or wireless communication components coupled to the uplink port 850 and / or downlink port 820 for data communication via an electrical communication network, an optical communication network, or a wireless communication network. The video decoding device 800 may also include input and / or output (I / O) devices 860 for data communication with the user. I / O devices 860 may include output devices, such as a monitor displaying video data, a speaker outputting audio data, etc. I / O devices 860 may also include input devices such as a keyboard, mouse, trackball, etc., and / or corresponding interfaces for interacting with the aforementioned output devices.

[0125] Processor 830 is implemented in both hardware and software. Processor 830 can be implemented as one or more CPU chips, one or more cores (e.g., implemented as a multi-core processor), one or more field-programmable gate arrays (FPGAs), one or more application-specific integrated circuits (ASICs), and one or more digital signal processors (DSPs). Processor 830 communicates with downlink port 820, transceiver unit 810, uplink port 850, and memory 832. Processor 830 includes a decoding module 814. Decoding module 814 implements the disclosed embodiments described herein, such as method 100, method 900, and / or method 1000, which may employ bitstream 500, image 600, and / or image 700. Decoding module 814 can also implement any other methods / mechanisms described herein. Furthermore, decoding module 814 can implement codec system 200, encoder 300, and / or decoder 400. For example, when acting as an encoder, decoding module 814 can indicate flags, sub-image IDs, and lengths in the PPS. The decoding module 814 can also encode the stripe address in the stripe header. Then, the decoding module 814 can extract a sub-stream, including sub-images, from the image's bitstream without rewriting the stripe header. When acting as a decoder, the decoding module 814 can read flags to determine whether to use an explicit stripe address instead of an index. The decoding module 814 can also read the length and sub-image ID from the PPS and the stripe address from the stripe header. Then, the decoding module 814 can use the length to resolve the stripe address and use the sub-image ID to map the stripe address from an image-based address to a sub-image-based address. Therefore, the decoding module 814 can determine the desired position of the stripe, regardless of the selected sub-image, and without rewriting the stripe header to accommodate sub-image-based address changes. Therefore, when segmenting and encoding video data, the decoding module 814 enables the video decoding device 800 to provide additional functions, avoid certain processes to reduce processing overhead, and / or improve encoding efficiency. Accordingly, the decoding module 814 improves the functionality of the video decoding device 800 while solving problems specific to the field of video encoding. Furthermore, the decoding module 814 affects the transitions of the video decoding device 800 to different states. Alternatively, the encoding module 814 can be implemented as instructions stored in memory 832 and executed by processor 830 (e.g., implemented as a computer program product stored in a non-transitory medium).

[0126] Memory 832 includes one or more memory types, such as disks, tape drives, solid-state drives, read-only memory (ROM), random access memory (RAM), flash memory, ternary content-addressable memory (TCAM), static random-access memory (SRAM), etc. Memory 832 can be used as an overflow data storage device to store programs when a program is selected for execution, as well as instructions and data read during program execution.

[0127] Figure 9 A flowchart of an exemplary method 900 for encoding a bitstream (e.g., bitstream 500) of an image (e.g., image 600) to support the extraction of a sub-bitstream (e.g., sub-bitstream 501) that includes a sub-image (e.g., sub-image 522) without requiring explicit address indication to rewrite the stripe header. Method 900 can be executed by an encoder such as codec system 200, encoder 300, and / or video decoding device 800 when performing method 100.

[0128] Method 900 may begin with: an encoder receiving a video sequence comprising multiple images and determining, based on user input, to encode the video sequence in a bitstream. The video sequence is segmented into pictures / images / frames for further segmentation prior to encoding. In step 901, the images in the video sequence are encoded in the bitstream. The images may include multiple stripes, the multiple stripes including a first stripe. The first stripe may be any stripe in the image, but for clarity, it is described as the first stripe. For example, the top-left corner of the first stripe may not be aligned with the top-left corner of the image.

[0129] In step 903, a slice header associated with the slice is encoded in the bitstream. The slice header includes the slice address of the first slice. The slice address may include a defined value, such as a numerical value selected by the encoder. This value can be arbitrary, but can increment in raster scan order (e.g., from left to right and from top to bottom) to support consistent encoding. The slice address may not include an index. In some examples, the slice address can be the syntax element `slice_address`.

[0130] In step 905, the PPS is encoded in the bitstream. An identifier and the stripe address length of the first stripe can be encoded in the PPS of the bitstream. The identifier can be a sub-image identifier. The stripe address length can represent the number of bits included in the stripe address. For example, the stripe address length in the PPS can include enough data to parse the stripe address from the stripe header (encoded in step 903). In some examples, the length can be the syntax element `subpic_id_len_minus1`. Furthermore, the identifier can include enough data to map the stripe address from an image-based location to a sub-image-based location. In some examples, the identifier can be the syntax element `subPicIdx`. For example, multiple sub-image-based identifiers can be included in the PPS. When extracting a sub-image, the corresponding sub-image ID can be indicated in the PPS by using a flag / pointer and / or removing unused sub-image IDs. In some examples, an explicit ID flag can also be encoded in a parameter set. The flag can indicate to the decoder that a mapping can be used to map the stripe address from the image-based location to the sub-image-based location. In some examples, the mapping can be the syntax element `SliceSubpicToPicIdx[SubPicIdx][slice_address]`. Accordingly, the flag can indicate that the stripe address is not an index. In some examples, the flag can be `sps_subpic_id_present_flag`.

[0131] In step 907, sub-streams are extracted from the bitstream. For example, extracting sub-streams may include: extracting the first stripe without rewriting the stripe header, based on the stripe address, the stripe address length, and the identifier. In a specific example, this extraction may also include extracting sub-images from the image. In this case, the sub-image includes the first stripe. The parameter set may also be included in the sub-stream. For example, the sub-stream may include the sub-image, the stripe header, the PPS, SPS, etc.

[0132] In step 909, the sub-stream is stored for transmission to the decoder. The sub-stream can then be transmitted to the decoder as needed.

[0133] Figure 10This is a flowchart of an exemplary method 1000 for decoding a sub-bitstream (e.g., sub-bitstream 501) that includes a sub-image (e.g., sub-image 522), the sub-bitstream being extracted from a bitstream (e.g., bitstream 500) of an image (e.g., image 600) via an explicit address indication. Method 1000 may be executed by a decoder such as a codec system 200, a decoder 400, and / or a video decoding device 800 when performing method 100.

[0134] Method 1000 may begin with: a decoder starting to receive a sub-bitstream extracted from the bitstream, such as the result of method 900. In step 1001, the sub-bitstream is received. The sub-bitstream includes sub-images within an image. For example, a bitstream encoded on the encoder side may include an image from which the encoder and / or striper extracts the sub-bitstream, the sub-bitstream including sub-images comprising one or more regions of an image in the bitstream. The received sub-images may be segmented into multiple stripes. The multiple stripes may include a strip designated as a first stripe. The first stripe may be any stripe in the image, but for clarity, it is described as the first stripe. For example, the top-left corner of the first stripe may not be aligned with the top-left corner of the image. The sub-bitstream also includes a PPS describing the syntax associated with the image, and therefore also the syntax associated with the sub-images. The sub-bitstream also includes a stripe header describing the syntax associated with the first stripe.

[0135] In step 1003, parameter sets such as PPS and / or SPS are parsed to obtain an explicit ID flag. The ID flag can indicate a mapping relationship that can be used to map a stripe address from an image-based location to a sub-image-based location. Accordingly, the flag can indicate that the corresponding stripe address includes a defined value but does not include an index. In some examples, the flag can be `sps_subpic_id_present_flag`. Based on the value of the ID flag, the PPS can be parsed to obtain an identifier and the stripe address length of the first stripe. The identifier can be a sub-image identifier. The stripe address length can represent the number of bits included in the corresponding stripe address. For example, the stripe address length in the PPS can include enough data to parse the stripe address from the stripe header. In some examples, the length can be the syntax element `subpic_id_len_minus1`. Furthermore, the identifier can include enough data to map the stripe address from an image-based location to a sub-image-based location. In some examples, the identifier can be the syntax element `subPicIdx`. For example, multiple sub-image-based identifiers can be included in the PPS. When extracting sub-images, the corresponding sub-image ID can be indicated in the PPS by using a flag / pointer and / or removing unused sub-image IDs.

[0136] In step 1005, the slice address of the first slice is determined from the slice header based on the identifier and the slice address length. For example, the length in the PPS can be used to determine bit boundaries to parse the slice address from the slice header. The identifier and the slice address can then be used to map the slice address from an image-based location to a sub-image-based location. For example, the mapping between the image-based location and the sub-image-based location can be used to align the slice header with the sub-image. In this way, the decoder can resolve the address mismatch between the slice header and the image addressing scheme caused by the encoder and / or slicer performing sub-stream extraction without needing to rewrite the slice header. In some examples, the mapping can be the syntax element SliceSubpicToPicIdx[SubPicIdx][slice_address].

[0137] In step 1007, the sub-stream is decoded to generate a video sequence of sub-images. The sub-images may include the first stripe. Accordingly, the first stripe is also decoded. The video sequence including the sub-images (including the decoded first stripe) can then be sent for display on a head-mounted display or other display device.

[0138] Figure 11This is a schematic diagram of an exemplary system 1100 for transmitting a sub-stream (e.g., sub-stream 501) that includes a sub-image (e.g., sub-image 522). The sub-stream including the sub-image is extracted from the bitstream (e.g., bitstream 500) of an image (e.g., image 600) via an explicit address indication. System 1100 may be implemented by encoders and decoders such as codec system 200, encoder 300, decoder 400, and / or video decoding device 800. Furthermore, system 1100 may be used to implement method 100, method 900, and / or method 1000.

[0139] System 1100 includes a video encoder 1102. The video encoder 1102 includes an encoding module 1101, configured to: encode an image in a bitstream, wherein the image includes multiple stripes, the multiple stripes including a first stripe; encode a stripe header in the bitstream, wherein the stripe header includes the stripe address of the first stripe; and encode a PPS in the bitstream, wherein the PPS includes an identifier and the stripe address length of the first stripe. The video encoder 1102 further includes an extraction module 1103, configured to extract a sub-bitstream from the bitstream without rewriting the stripe header, based on the stripe address of the first stripe, the stripe address length, and the identifier. The video encoder 1102 further includes a storage module 1105, configured to store the bitstream for transmission to a decoder. The video encoder 1102 further includes a transmission module 1107, configured to transmit the sub-bitstream to the decoder, wherein the sub-bitstream includes the stripe header, the PPS, the first stripe, and / or the corresponding sub-image. The video encoder 1102 can also be used to perform any step of method 900.

[0140] System 1100 further includes a video decoder 1110. The video decoder 1110 includes a receiving module 1111 for receiving a sub-stream, wherein the sub-stream includes sub-images in an image, wherein the image is segmented into multiple stripes including a first stripe, PPS associated with the image and the sub-images, and a stripe header associated with the first stripe. The video decoder 1110 further includes a parsing module 1113 for parsing the PPS to obtain an identifier and the stripe address length of the first stripe. The video decoder 1110 further includes a determining module 1115 for determining the stripe address of the first stripe from the stripe header based on the identifier and the stripe address length. The video decoder 1110 further includes a decoding module 1117 for decoding the sub-stream to generate a video sequence including the sub-images of the first stripe. The video decoder 1110 further includes a sending module 1119 for sending the video sequence including the sub-images of the first stripe for display. The video decoder 1110 can also be used to perform any step of method 1000.

[0141] When there is no intermediate component between the first component and the second component other than a line, trace, or other medium, the first component and the second component are directly coupled. When there is an intermediate component between the first component and the second component other than a line, trace, or other medium, the first component and the second component are indirectly coupled. The term "coupled" and its variations include direct coupling and indirect coupling. Unless otherwise stated, the term "about" is used to mean a range including ±10% of the following figure.

[0142] It should also be understood that the steps of the exemplary methods described herein do not necessarily need to be performed in the order described, and the order of the steps of these methods should be understood as merely exemplary. Similarly, in methods consistent with various embodiments of the present invention, these methods may include other steps, and some steps may be omitted or combined.

[0143] While this invention provides several embodiments, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the invention. The examples of this invention should be considered illustrative rather than restrictive, and the invention is not limited to the details given herein. For example, various elements or components may be combined or integrated into another system, or certain features may be omitted or not implemented.

[0144] Furthermore, without departing from the scope of the invention, the techniques, systems, subsystems, and methods described and illustrated as discrete or separate in the various embodiments can be combined or integrated with other systems, components, techniques, or methods. Other examples of variations, substitutions, and modifications can be determined by those skilled in the art and are given without departing from the spirit and scope of the disclosure herein.

Claims

1. A method implemented in a decoder, characterized in that, The method includes: Receive a bitstream, wherein the bitstream includes sub-images in an image, a set of parameters associated with the image and the sub-images, and a strip header associated with a first stripe, wherein the image is segmented into multiple stripes including the first stripe; The bitstream is parsed to obtain a sequence parameter set (SPS) and an image parameter set (PPS). The PPS includes a first parameter, which is used to deduce the strip address length of the first strip. The strip address length is deduced to be equal to Ceil(Log2(NumTilesInPic)) based on the presence or absence of the first parameter, where NumTilesInPic represents the number of blocks in the image. The SPS includes a second parameter, which is used to indicate the identifier (ID) of the sub-image in the image. The stripe address of the first stripe is determined from the stripe header based on the stripe address length; Based on the stripe address and the second parameter, decode the sub-image containing the first stripe.

2. The method according to claim 1, characterized in that, The method further includes: A video sequence including the sub-images is sent for display.

3. The method according to claim 1 or 2, characterized in that, The stripe address length represents the number of bits included in the stripe address.

4. The method according to claim 1 or 2, characterized in that, Determining the stripe address of the first stripe includes: The length in the parameter set is used to determine the bit boundaries in order to parse the stripe address from the stripe header; The stripe address is mapped from an image-based location to a sub-image-based location using the identifier and the stripe address.

5. The method according to claim 1 or 2, characterized in that, The striped address includes defined values ​​but does not include indexes.

6. A method implemented in an encoder, characterized in that, The method includes: The image is encoded into a bitstream, wherein the image is segmented into multiple stripes including a first stripe; The parameter set is encoded into the bitstream. The parameter set includes a sequence parameter set (SPS) and an image parameter set (PPS). The PPS includes a first parameter, which is used to deduce the strip address length of the first strip. The strip address length is deduced to be equal to Ceil(Log2(NumTilesInPic)) based on the presence or absence of the first parameter. The NumTilesInPic represents the number of blocks in the image. The SPS includes a second parameter, which is used to indicate the identifier (ID) of the sub-images in the image. Based on the length of the stripe address and the second parameter, the stripe address of the first stripe is encoded into the stripe header of the bitstream.

7. The method according to claim 6, characterized in that, The method further includes: The bitstream is stored in at least one storage medium.

8. The method according to claim 6 or 7, characterized in that, The stripe address length represents the number of bits included in the stripe address.

9. The method according to claim 6 or 7, characterized in that, The striped address includes defined values ​​but does not include indexes.

10. A decoder, characterized in that, The decoder includes: A receiver is configured to receive a bitstream, wherein the bitstream includes sub-images in an image, a set of parameters associated with the image and the sub-images, and a strip header associated with a first stripe, wherein the image is segmented into a plurality of stripes including the first stripe; Processor, used for: The bitstream is parsed to obtain a sequence parameter set (SPS) and an image parameter set (PPS). The PPS includes a first parameter, which is used to deduce the strip address length of the first strip. The strip address length is deduced to be equal to Ceil(Log2(NumTilesInPic)) based on the presence or absence of the first parameter, where NumTilesInPic represents the number of blocks in the image. The SPS includes a second parameter, which is used to indicate the identifier (ID) of the sub-image in the image. The stripe address of the first stripe is determined from the stripe header based on the stripe address length; Based on the stripe address and the second parameter, decode the sub-image containing the first stripe.

11. The decoder according to claim 10, characterized in that, The processor is also used for: A video sequence including the sub-images is sent for display.

12. The decoder according to claim 10 or 11, characterized in that, The stripe address length represents the number of bits included in the stripe address.

13. The decoder according to claim 10 or 11, characterized in that, The processor is specifically used for: The length in the parameter set is used to determine the bit boundaries in order to parse the stripe address from the stripe header; The stripe address is mapped from an image-based location to a sub-image-based location using the identifier and the stripe address.

14. The decoder according to claim 10 or 11, characterized in that, The striped address includes defined values ​​but does not include indexes.

15. An encoder, characterized in that, The encoder includes: Processor, used for: The image is encoded into a bitstream, wherein the image is segmented into multiple stripes including a first stripe; The parameter set is encoded into the bitstream. The parameter set includes a sequence parameter set (SPS) and an image parameter set (PPS). The PPS includes a first parameter, which is used to deduce the strip address length of the first strip. The strip address length is deduced to be equal to Ceil(Log2(NumTilesInPic)) based on the presence or absence of the first parameter. The NumTilesInPic represents the number of blocks in the image. The SPS includes a second parameter, which is used to indicate the identifier (ID) of the sub-images in the image. Based on the length of the stripe address and the second parameter, the stripe address of the first stripe is encoded into the stripe header of the bitstream; A transmitter for sending the bitstream to a decoder.

16. The encoder according to claim 15, characterized in that, The bitstream is stored in at least one storage medium.

17. The encoder according to claim 15 or 16, characterized in that, The stripe address length represents the number of bits included in the stripe address.

18. The encoder according to claim 15 or 16, characterized in that, The striped address includes defined values ​​but does not include indexes.

19. A video decoding device, characterized in that, The video decoding device includes: A processor, a memory, a receiver coupled to the processor, and a transmitter coupled to the processor, wherein the processor, memory, receiver, and / or transmitter are configured to perform the method according to any one of claims 1 to 9.

20. A non-transitory computer-readable medium, characterized in that, It stores computer-executable instructions thereon, which, when executed by a processor, cause the video decoding device to perform the method of any one of claims 1 to 9.

21. A non-transitory computer-readable medium, characterized in that, It stores a computer program and a bitstream thereon, characterized in that the computer program, when executed by a processor, implements the method as described in any one of claims 6 to 9 to generate the bitstream.

Images