Indication of single slice per sub-picture in sub-picture based video coding

Abstract

This invention provides a video decoding method. The method includes: receiving a bitstream, wherein the bitstream includes a picture parameter set (PPS), and the PPS includes a single stripe marker for each sub-image; determining whether the value of the single stripe marker for each sub-image is a first value or a second value, wherein the first value indicates that each sub-image of the PPS includes one and only one rectangular stripe, and the second value indicates that each sub-image of the PPS may include one or more rectangular stripes; when the value of the single stripe marker for each sub-image is the first value, decoding the one and only one rectangular stripe; when the value of the single stripe marker for each sub-image is the second value, decoding the one or more rectangular stripes.

Description

[0001] Cross-referencing related applications

[0002] This patent application claims the benefit of U.S. Provisional Patent Application No. 62 / 904,481, filed September 23, 2019, entitled "Indication of One Slice Per Subpicture in Subpicture-Based Video Coding," and U.S. Provisional Patent Application No. 62 / 905,122, filed September 24, 2019, also entitled "Indication of One Slice Per Subpicture in Subpicture-Based Video Coding," which are incorporated herein by reference. Technical Field

[0003] This invention generally relates to video decoding, and more specifically to the transmission of instructions in video decoding based on sub-images. Background Technology

[0004] Even relatively short videos can require a large amount of video data to describe, which can cause difficulties when streaming or otherwise transmitting the data over bandwidth-constrained communication networks. Therefore, video data is typically compressed before transmission over modern telecommunications networks. The size of the video can also be an issue when storing it on storage devices due to potentially limited memory resources. Video compression devices typically encode video data using software and / or hardware on the source side before transmission or storage, reducing the amount of data required to represent the digital video image. Video decompression devices then decode the video data and receive the compressed data at the destination. Given limited network resources and the growing demand for higher video quality, there is a need to improve compression and decompression technologies to increase compression ratios with minimal impact on image quality. Summary of the Invention

[0005] The first aspect relates to a method implemented by a decoder. The method includes: the decoder receiving a bitstream, wherein the bitstream includes a picture parameter set (PPS), the PPS including a single stripe marker for each sub-image; the decoder determining whether the value of the single stripe marker for each sub-image is a first value or a second value, wherein the first value indicates that each sub-image referencing the PPS includes one and only one rectangular stripe, and the second value indicates that each sub-image referencing the PPS may include one or more rectangular stripes; when the value of the single stripe marker for each sub-image is the first value, the decoder decodes the one and only one rectangular stripe to obtain a decoded image; when the value of the single stripe marker for each sub-image is the second value, the decoder decodes the one or more rectangular stripes to obtain a decoded image.

[0006] This method provides a technique for using flags in the bitstream to indicate whether each sub-image in an image contains one and only one rectangular stripe. Because a sub-image is identical to a stripe, when the flag indicates that each sub-image contains only a single rectangular stripe, syntax elements indicating the number and layout of sub-images (e.g., the position and size of each sub-image) can be omitted from the bitstream. In other words, when the flag indicates that each sub-image contains only a single rectangular stripe, the bitstream only needs to include the syntax elements for the stripe. This reduces the amount of information carried in the bitstream, thereby reducing the use of memory, processor, and network resources and improving decoding efficiency. Therefore, the encoder / decoder (also known as a codec) in video decoding represents an improvement over existing codecs. In fact, the improved video decoding process provides a better user experience when the video is sent, received, and / or viewed.

[0007] Alternatively, according to any of the above aspects, in another implementation of said aspect, the single-slice flag per subpic is represented as pps_single_slice_per_subpic_flag.

[0008] Alternatively, according to any of the above aspects, in another implementation of said aspect, the first value is 1 (1) and the second value is 0 (0).

[0009] Optionally, according to any of the above aspects, in another implementation of said aspect, the PPS further includes a stripe count flag in the image, and the value of the stripe count flag plus 1 indicates the number of stripes in each image referencing the PPS.

[0010] Alternatively, according to any of the above aspects, in another implementation of said aspect, the number of stripes in the image is represented as pps_num_slices_in_pic_minus1.

[0011] Alternatively, according to any of the above aspects, in another implementation of said aspect, each sub-image includes a rectangular region of one or more stripes within the image.

[0012] Optionally, according to any of the above aspects, in another implementation of said aspect, the method further includes: when the value of the single stripe marker of each sub-image is the first value, generating an image based on the one and only one rectangular stripe for display; and when the value of the single stripe marker of each sub-image is the second value, generating an image based on the one or more rectangular stripes for display.

[0013] The second aspect relates to a method implemented by an encoder. The method includes: when each sub-image of a reference image parameter set (PPS) includes one and only one rectangular stripe, the encoder sets a single stripe flag for each sub-image to a first value; when each sub-image of the reference PPS may include one or more rectangular stripes, the encoder sets the single stripe flag for each sub-image to a second value; the encoder encodes the PPS into a bitstream, wherein the PPS includes the single stripe flag for each sub-image; the encoder stores the bitstream for transmission to a decoder.

[0014] This method provides a technique for using flags in the bitstream to indicate whether each sub-image in an image contains one and only one rectangular stripe. Because a sub-image is identical to a stripe, when the flag indicates that each sub-image contains only a single rectangular stripe, syntax elements indicating the number and layout of sub-images (e.g., the position and size of each sub-image) can be omitted from the bitstream. In other words, when the flag indicates that each sub-image contains only a single rectangular stripe, the bitstream only needs to include the syntax elements for the stripe. This reduces the amount of information carried in the bitstream, thereby reducing the use of memory, processor, and network resources and improving decoding efficiency. Therefore, the encoder / decoder (also known as a codec) in video decoding represents an improvement over existing codecs. In fact, the improved video decoding process provides a better user experience when the video is sent, received, and / or viewed.

[0015] Alternatively, according to any of the above aspects, in another implementation of said aspect, the single-slice flag per subpic is represented as pps_single_slice_per_subpic_flag.

[0016] Alternatively, according to any of the above aspects, in another implementation of said aspect, the first value is 1 (1) and the second value is 0 (0).

[0017] Optionally, according to any of the above aspects, in another implementation of said aspect, the PPS further includes a stripe count flag in the image, and the value of the stripe count flag plus 1 indicates the number of stripes in each image referencing the PPS.

[0018] Alternatively, according to any of the above aspects, in another implementation of said aspect, the number of stripes in the image is represented as pps_num_slices_in_pic_minus1.

[0019] Alternatively, according to any of the above aspects, in another implementation of said aspect, each sub-image includes a rectangular region of one or more stripes within the image.

[0020] Optionally, according to any of the foregoing aspects, in another implementation of said aspect, the method further includes: sending the bitstream to a video decoder.

[0021] The third aspect relates to a decoding device. The decoding device includes: a receiver for receiving a bitstream, wherein the bitstream includes a picture parameter set (PPS), the PPS including a single stripe marker for each sub-image; a memory coupled to the receiver, wherein the memory stores instructions; and a processor coupled to the memory, wherein the processor executes the instructions to cause the decoding device to perform the following operations: determining whether the value of the single stripe marker for each sub-image is a first value or a second value, wherein the first value indicates that each sub-image referencing the PPS includes one and only one rectangular stripe, and the second value indicates that each sub-image referencing the PPS may include one or more rectangular stripes; when the value of the single stripe marker for each sub-image is the first value, decoding the one and only one rectangular stripe to obtain a decoded image; and when the value of the single stripe marker for each sub-image is the second value, decoding the one or more rectangular stripes to obtain a decoded image.

[0022] This decoding device provides a technique that uses flags in the bitstream to indicate whether each sub-image in an image contains one and only one rectangular stripe. Because a sub-image is identical to a stripe, when the flag indicates that each sub-image contains only a single rectangular stripe, syntax elements indicating the number and layout of sub-images (e.g., the position and size of each sub-image) can be omitted from the bitstream. In other words, when the flag indicates that each sub-image contains only a single rectangular stripe, the bitstream only needs to include the syntax elements for the stripe. This reduces the amount of information carried in the bitstream, thereby reducing the use of memory, processor, and network resources and improving decoding efficiency. Therefore, the encoder / decoder (also known as a codec) in video decoding represents an improvement over existing codecs. In fact, the improved video decoding process provides a better user experience when video is sent, received, and / or viewed.

[0023] Alternatively, according to any of the above aspects, in another implementation of said aspect, the first value is 1 (1) and the second value is 0 (0).

[0024] Optionally, according to any of the above aspects, in another implementation of said aspect, the PPS further includes a stripe count flag in the image, and the value of the stripe count flag plus 1 indicates the number of stripes in each image referencing the PPS.

[0025] Alternatively, according to any of the above aspects, in another implementation of said aspect, the number of stripes in the image is represented as pps_num_slices_in_pic_minus1.

[0026] Optionally, according to any of the above aspects, in another implementation of said aspect, the processor is further configured to: generate an image for display based on the one and only one rectangular strip when the value of the single strip marker for each sub-image is the first value, and generate an image for display based on the one or more rectangular strips when the value of the single strip marker for each sub-image is the second value.

[0027] The fourth aspect relates to an encoding device. The encoding device includes: a memory including instructions; a processor coupled to the memory, wherein the processor is configured to execute the instructions to cause the encoding device to perform the following operations: encoding a bitstream, wherein the bitstream includes a picture parameter set (PPS), the PPS including a single stripe flag for each sub-image; setting the single stripe flag for each sub-image to a first value when each sub-image of the PPS includes one and only one rectangular stripe, and setting the single stripe flag for each sub-image to a second value when each sub-image of the PPS may include one or more rectangular stripes; and a transmitter coupled to the processor, wherein the transmitter is configured to transmit the bitstream to a video decoder.

[0028] This encoding device provides a technique that uses flags in the bitstream to indicate whether each sub-image in an image contains one and only one rectangular stripe. Because a sub-image is identical to a stripe, when the flag indicates that each sub-image contains only a single rectangular stripe, syntax elements indicating the number and layout of sub-images (e.g., the position and size of each sub-image) can be omitted from the bitstream. In other words, when the flag indicates that each sub-image contains only a single rectangular stripe, the bitstream only needs to include the syntax elements for the stripe. This reduces the amount of information carried in the bitstream, thereby reducing the use of memory, processor, and network resources and improving decoding efficiency. Therefore, the encoder / decoder (also known as a codec) in video decoding represents an improvement over existing codecs. In fact, the improved video decoding process provides a better user experience when video is sent, received, and / or viewed.

[0029] Alternatively, according to any of the above aspects, in another implementation of said aspect, the first value is 1 (1) and the second value is 0 (0).

[0030] Optionally, according to any of the above aspects, in another implementation of said aspect, the PPS further includes a stripe count flag in the image, and the value of the stripe count flag plus 1 indicates the number of stripes in each image referencing the PPS.

[0031] Alternatively, according to any of the above aspects, in another implementation of said aspect, the number of stripes in the image is represented as pps_num_slices_in_pic_minus1.

[0032] Alternatively, according to any of the foregoing aspects, in another implementation of said aspect, the memory stores the bitstream before sending the bitstream to the decoder.

[0033] The fifth aspect relates to a decoding apparatus. The decoding apparatus includes: a receiver for receiving an image for encoding or receiving a bitstream for decoding; a transmitter coupled to the receiver, wherein the transmitter is used to send the bitstream to the decoder or to send the decoded image to a display; a memory coupled to at least one of the receiver or the transmitter, wherein the memory is used to store instructions; and a processor coupled to the memory, wherein the processor is used to execute the instructions stored in the memory to perform any of the methods disclosed herein.

[0034] This decoding device provides a technique for using flags in the bitstream to indicate whether each sub-image in an image contains one and only one rectangular stripe. Because a sub-image is identical to a stripe, when the flag indicates that each sub-image contains only a single rectangular stripe, syntax elements indicating the number and layout of sub-images (e.g., the position and size of each sub-image) can be omitted from the bitstream. In other words, when the flag indicates that each sub-image contains only a single rectangular stripe, the bitstream only needs to include the syntax elements for the stripe. This reduces the amount of information carried in the bitstream, thereby reducing the use of memory, processor, and network resources and improving decoding efficiency. Therefore, the encoder / decoder (also known as a codec) in video decoding represents an improvement over existing codecs. In fact, the improved video decoding process provides a better user experience when video is sent, received, and / or viewed.

[0035] Alternatively, according to any of the foregoing aspects, in another implementation of said aspect, the decoding apparatus further includes a display for displaying the decoded image.

[0036] The sixth aspect relates to a system. The system includes: an encoder; and a decoder in communication with the encoder, wherein the encoder or the decoder includes the decoding device, the encoding device, or the decoding apparatus disclosed herein.

[0037] This system provides a technique for using flags in the bitstream to indicate whether each sub-image in an image contains one and only one rectangular stripe. Because a sub-image is identical to a stripe, when the flag indicates that each sub-image contains only a single rectangular stripe, the syntax elements indicating the number and layout of sub-images (e.g., the position and size of each sub-image) can be omitted from the bitstream. In other words, when the flag indicates that each sub-image contains only a single rectangular stripe, the bitstream only needs to include the syntax elements for the stripe. This reduces the amount of information carried in the bitstream, thereby reducing the use of memory, processor, and network resources and improving decoding efficiency. Therefore, the encoder / decoder (also known as a codec) in video decoding represents an improvement over existing codecs. In fact, the improved video decoding process provides a better user experience when the video is sent, received, and / or viewed.

[0038] The seventh aspect relates to a decoding module. The decoding module includes: a receiving module for receiving an image for encoding or receiving a bitstream for decoding; a transmitting module coupled to the receiving module, wherein the transmitting module is used to transmit the bitstream to the decoding module or to transmit the decoded image to a display module; a storage module coupled to at least one of the receiving module or the transmitting module, wherein the storage module is used to store instructions; and a processing module coupled to the storage module, wherein the processing module is used to execute the instructions stored in the storage module to perform any of the methods disclosed herein.

[0039] This decoding module provides a technique for using flags in the bitstream to indicate whether each sub-image in an image contains one and only one rectangular stripe. Because a sub-image is identical to a stripe, when the flag indicates that each sub-image contains only a single rectangular stripe, syntax elements indicating the number and layout of sub-images (e.g., the position and size of each sub-image) can be omitted from the bitstream. In other words, when the flag indicates that each sub-image contains only a single rectangular stripe, the bitstream only needs to include the syntax elements for the stripe. This reduces the amount of information carried in the bitstream, thereby reducing the use of memory, processor, and network resources and improving decoding efficiency. Therefore, the encoder / decoder (also known as a codec) in video decoding represents an improvement over existing codecs. In fact, the improved video decoding process provides a better user experience when the video is sent, received, and / or viewed.

[0040] For clarity, any of the above embodiments may be combined with any one or more of the other embodiments described above to create new embodiments within the scope of the present invention.

[0041] These and other features will become clearer from the following detailed description taken in conjunction with the accompanying drawings and claims. Attached Figure Description

[0042] To gain a more complete understanding of the invention, reference is now made to the following brief description taken in conjunction with the accompanying drawings and specific embodiments, wherein similar reference numerals denote similar parts.

[0043] Figure 1 A flowchart illustrating an exemplary method for decoding video signals;

[0044] Figure 2 This is a schematic diagram of an exemplary encoding and decoding (encoding / decoding) system for video decoding;

[0045] Figure 3 This is a schematic diagram of an exemplary video encoder;

[0046] Figure 4 This is a schematic diagram of an exemplary video decoder;

[0047] Figure 5 This is a schematic diagram of an exemplary bitstream that includes an encoded video sequence;

[0048] Figure 6 This is an example of a method for decoding a decoded video stream;

[0049] Figure 7 This is an example of a method for encoding a decoded video stream;

[0050] Figure 8 A schematic diagram of a video decoding device;

[0051] Figure 9 This is a schematic diagram of one embodiment of the decoding module. Detailed Implementation

[0052] First, it should be understood that although illustrative implementations of one or more embodiments are provided below, the disclosed systems and / or methods can be implemented using any number of techniques, whether currently known or existing. The invention should not be limited in any way to the illustrative implementations, drawings, and techniques described below, including the exemplary designs and implementations illustrated and described herein, but can be modified within the full scope of the appended claims and their equivalents.

[0053] The following terms are defined as follows, unless used in the opposite context herein. Specifically, the following definitions are intended to describe the invention more clearly. However, terms may be described differently in different contexts. Therefore, the following definitions should be considered supplementary information and not as limiting of any other definitions provided herein for these terms.

[0054] A bitstream is a sequence of bits containing video data that is compressed for transmission between an encoder and a decoder. An encoder is a device used to compress video data into a bitstream through an encoding process. A decoder is a device used to reconstruct video data from the bitstream for display through a decoding process. An image is an array of luminance and / or chrominance samples that create its frames or fields. For clarity, the image being encoded or decoded can be referred to as the current image. A reference image is an image that includes reference samples that can be used when decoding other images using references based on inter-frame prediction and / or inter-layer prediction. A reference image list is a list of reference images used for inter-frame prediction and / or inter-layer prediction. Some video decoding systems use two reference image lists, which can be represented as reference image list 1 and reference image list 0. A reference image list structure is an addressable syntax structure that contains multiple reference image lists. Inter-frame prediction is a mechanism for decoding samples of the current image by referencing indicator samples in a reference image that is different from the current image, where the reference image and the current image are at the same layer. A reference image list structure entry is an addressable location in the reference image list structure that represents a reference image associated with the reference image list. A stripe header is a portion of the decoded stripe and includes data elements associated with all video data within the blocks represented in the stripe. A stripe consists of four sides connected by right angles. These four sides include two pairs of parallel sides. Furthermore, the sides in each pair of parallel sides are of equal length. Therefore, a stripe can be any rectangular shape, with a square being a special case of a rectangle where all four sides are of equal length. A rectangular stripe is a stripe with two pairs of parallel sides, where each pair of parallel sides is of equal length, but one pair of parallel sides is longer than the other. A sequence parameter set (SPS) is a set of parameters containing data associated with the image sequence. A picture parameter set (PPS) is a syntax structure that includes syntax elements applicable to zero or more complete decoded images, determined by the syntax elements present in each picture header.

[0055] A flag is a variable or unit syntax element that can take one of two possible values: 0 and 1. A subimage is a rectangular region of one or more stripes within an image. A subimage identifier (ID) is a number, letter, or other marker that uniquely identifies a subimage. Subimage IDs (also known as block identifiers) are used to identify specific subimages using subimage indices; this can be referred to here as a subimage ID mapping.

[0056] An access unit (AU) is a collection of one or more encoded images associated with the same display time (e.g., the same image sequence number), which are output from the decoded picture buffer (DPB) (e.g., displayed to the user). An access unit delimiter (AUD) is an indicator or data structure used to indicate the start of an AU or the boundaries between AUs. A decoded video sequence is a series of images that have been reconstructed by the decoder in preparation for display to the user.

[0057] A coded video sequence (CVS) is a sequence of access units (AUs) that, in decoding order, includes a coded video sequence start (CVSS) AU followed by zero or more AUs that are not CVSS AUs. These non-CVSS AUs include all subsequent AUs, up to but not including any subsequent AUs that are CVSS AUs. A CVSS AU is an AU in which each layer specified by a video parameter set (VPS) has a prediction unit (PU), and the encoded image in each PU is the coded layer video sequence start (CLVSS) image. In one embodiment, each image is within an AU. A PU is a set of network abstraction layer (NAL) units associated with each other according to specified classification rules, the NAL units being consecutive in decoding order and containing exactly one encoded image.

[0058] The following abbreviations are used in this paper: Adaptive Loop Filter (ALF), Coding Tree Block (CTB), Coding Tree Unit (CTU), Coding Unit (CU), Coded Video Sequence (CVS), Joint Video Experts Team (JVET), Motion-Constrained Tile Set (MCTS), Maximum Transfer Unit (MTU), Network Abstraction Layer (NAL), Picture Order Count (POC), Picture Parameter Set (PPS), Raw Byte Sequence Payload (RBSP), Sample Adaptive Offset (SAO), Sequence Parameter Set (SPS), Temporal Motion Vector Prediction (TMVP), Versatile Video Coding (VVC), and Working Draft. Draft, WD).

[0059] Many video compression techniques can be used to reduce the size of video files while minimizing data loss. For example, video compression techniques may include performing spatial (e.g., intra-frame) prediction and / or temporal (e.g., inter-frame) prediction to reduce or remove data redundancy in a video sequence. For block-based video decoding, video stripes (e.g., video images or portions of video images) can be segmented into video blocks, which may also be referred to as tree blocks, coding tree blocks (CTBs), coding tree units (CTUs), coding units (CUs), and / or coding nodes. Video blocks in intra-frame decoded (I) stripes of an image are decoded using spatial prediction relative to reference samples in adjacent blocks within the same image, while video blocks in inter-frame decoded one-way prediction (P) or two-way prediction (B) stripes of an image may be decoded using spatial prediction relative to reference samples in adjacent blocks within the same image, or they may be decoded using temporal prediction relative to reference samples in other reference images. An image may be referred to as a frame, and a reference image may be referred to as a reference frame. Spatial or temporal prediction produces predicted blocks representing image patches. Residual data represents the pixel difference between the original image patch and the predicted patch. Therefore, inter-frame decoded blocks are encoded based on motion vectors and residual data, where the motion vectors point to the block of reference samples forming the predicted patch, and the residual data represents the difference between the decoded block and the predicted patch; intra-frame decoded blocks are encoded based on the intra-frame decoding mode and residual data. For further compression, the residual data can be transformed from the pixel domain to the transform domain. This produces quantizable residual transform coefficients. The quantized transform coefficients can initially be arranged in a two-dimensional array. The quantized transform coefficients can be scanned to produce a one-dimensional vector of transform coefficients. Entropy decoding can be used to achieve further compression. These video compression techniques are discussed in more detail below.

[0060] To ensure that encoded video can be correctly decoded, the video is encoded and decoded according to the corresponding video coding standards. Video coding standards include ITU Standardization Sector (ITU-T) H.261, ISO / IEC Motion Picture Experts Group (MPEG) Part 2, ITU-T H.262 or ISO / IEC MPEG-2 Part 2, ITU-T H.263, ISO / IEC MPEG-4 Part 2, Advanced Video Coding (AVC) (also known as ITU-T H.264 or ISO / IEC MPEG-4 Part 10), and High Efficiency Video Coding (HEVC) (also known as ITU-T H.265 or MPEG-H Part 2). AVC includes extended versions such as Scalable Video Coding (SVC), Multiview Video Coding (MVC), Multiview Video Coding plus Depth (MVC+D), and three-dimensional (3D) AVC (3D-AVC). HEVC includes extended versions such as Scalable HEVC (SHVC), Multiview HEVC (MV-HEVC), and 3D-HEVC (3D-HEVC).

[0061] There is also a new video coding standard called Versatile Video Coding (VVC), which is being developed by the Joint Video Experts Team (JVET) of ITU-T and ISO / IEC. Although there are several working drafts of the VVC standard, this article specifically refers to one working draft (WD), namely "Versatile Video Coding (Draft 5)," proposed by B. Bross, J. Chen, and S. Liu in JVET-N1001-v3 at the 13th JVET meeting on March 27, 2019.

[0062] HEVC includes four different image segmentation schemes: regular striping, non-independent striping, block segmentation, and wavefront parallel processing (WPP). These schemes can be applied to maximum transfer unit (MTU) size matching, parallel processing, and reduction of end-to-end latency.

[0063] Regular stripes in HEVC are similar to those in H.264 / AVC. Each regular strip is encapsulated in its own NAL unit, and intra-frame prediction (intra-sample prediction, motion information prediction, coding mode prediction) and entropy coding dependencies across stripe boundaries are disabled. Therefore, regular stripes can be reconstructed independently of other regular stripes within the same image (however, due to cyclic filtering operations, interdependencies may still exist).

[0064] Regular striping is the only tool available for parallelization, and it exists in almost the same form in H.264 / AVC. Parallelization based on regular striping requires less inter-processor or inter-core communication (except for inter-processor or inter-core data sharing for motion compensation when decoding a predictive decoded image, which typically requires more communication than inter-processor or inter-core data sharing due to intra-frame prediction). However, for the same reason, using regular striping can incur significant decoding overhead due to the bit cost of the stripe header and the lack of prediction across stripe boundaries. Furthermore, because regular striping is intra-frame independent and each regular stripe is encapsulated in its own NAL unit, it also serves as a key mechanism for bitstream segmentation to match MTU size requirements, compared to other tools mentioned below. In many cases, achieving both parallelization and MTU size matching presents conflicting requirements for stripe layout within the image. Therefore, addressing this situation necessitates the development of the parallelization tools mentioned below.

[0065] Non-independent slices have short slice headers and can segment the bitstream at tree block boundaries without interrupting any intra-frame prediction. Essentially, correlated slices divide a regular slice into multiple NAL units, which reduces end-to-end latency by sending a portion of the regular slice before the entire regular slice's encoding is complete.

[0066] In WPP, the image is segmented into single rows of coding tree blocks (CTBs). Entropy decoding and prediction can utilize data from CTBs in other segmented parts. Parallel decoding of CTB rows enables parallel processing, where the start of CTB row decoding is delayed by two CTBs to ensure that data related to the CTBs above and to the right of the current CTB is available before decoding of the current CTB. This staggered start (resembling a wavefront in graphical representation) allows for parallelization when there are as many processors / cores as the number of CTB rows included in the image. Intra-frame prediction between adjacent tree block rows within the image may require significant inter-processor / inter-core communication. Applying WPP segmentation does not generate additional NAL units compared to not applying WPP segmentation; therefore, WPP cannot achieve MTU size matching. However, if MTU size matching is required, regular striping can be used with WPP, which incurs some decoding overhead.

[0067] Block segmentation defines the horizontal and vertical boundaries for dividing an image into block columns and rows. First, the CTB scan is changed to be performed within a block (i.e., according to the CTB raster scan order of a block), then the top-left CTB in the next block is decoded according to the block raster scan order of the image. Similar to regular striping, block segmentation breaks intra-prediction dependencies and entropy decoding dependencies. However, these blocks may not be contained in individual NAL units (similar to WPP in this respect), therefore, blocks may not be used for MTU size matching. Each block can be processed by a single processor / core, and the inter-processor / inter-core communication used for intra-prediction between processing units decoding adjacent blocks is limited to sending a shared stripe header (when multiple blocks are in the same stripe) and loop filtering related sharing for performing reconstructed samples and metadata. When a stripe includes multiple blocks or WPP segments, the entry point byte offset for each block and WPP segment, except for the first block or WPP segment in the stripe, is indicated in the stripe header.

[0068] For simplicity, HEVC imposes restrictions on the application of the four different image segmentation schemes. A given coded video sequence cannot include blocks and wavefronts from most of the profiles detailed in HEVC. For each strip and block, one or both of the following conditions must be met: (1) all decoded tree blocks in a strip belong to the same block; (2) all decoded tree blocks in a block belong to the same strip. Finally, a wavefront segment comprises exactly one CTB line, and when using WPP, if a strip begins within a CTB line, it must end within the same CTB line.

[0069] The JCT-VC output document JCTVC-AC1005, "HEVC Supplemental Enhancement Information (Draft 4)" (October 24, 2017), edited by J. Boyce, A. Ramasubramonian, R. Skupin, G. J. Sullivan, A. Tourapis, Y.-K. Wang, etc., details a recent modification to HEVC. A public version of the draft can be obtained from the following link: http: / / phenix.int-evry.fr / jct / doc_end_user / documents / 29_Macau / wg11 / JCTVC-AC1005-v2.zip. Including this modification, HEVC defines three MCTS-related supplemental enhancement information (SEI) messages: the temporal MCTS SEI message, the MCTS extracted information set SEI message, and the MCTS extracted information nested SEI message.

[0070] The temporal MCTS SEI message indicates the presence of an MCTS in the bitstream and identifies that MCTS. For each MCTS, the motion vector is constrained to point to the full sample position inside the MCTS and to the fractional sample position, which only requires interpolation of the full sample position inside the MCTS. Furthermore, motion vector candidates are not allowed for temporal motion vector prediction derived from blocks outside the MCTS. This allows each MCTS to be decoded independently, without needing blocks not included in the MCTS.

[0071] The MCTS Extraction Information Set (SEI) message provides supplementary information (represented as part of the SEI message semantics) that can be used for MCTS substream extraction to generate a consistent bitstream for the MCTS set. This information includes multiple extraction information sets, each defining multiple MCTS sets and including RBSP bytes for the replacement video parameter set (VPS), SPS, and PPS to be used during MCTS substream extraction. When extracting substreams according to the MCTS substream extraction process, the parameter sets (VPS, SPS, and PPS) need to be rewritten or replaced, and the slice header needs to be slightly updated, because one or all slice address-related syntax elements (including first_slice_segment_in_pic_flag and slice_segment_address) typically need to have different values.

[0072] In the latest VVC draft specification, an image can be segmented into multiple sub-images, each covering a rectangular region and comprising an integer number of complete stripes. Sub-image segmentation persists across all images within the coded video sequence (CVS), and the segmentation information (i.e., the position and size of the sub-images) is indicated in the SPS. Sub-images can be represented as being decoded without using sample values ​​from any other sub-images for motion compensation.

[0073] In JVET document JVET-O0141 (publicly available at: http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 13_Marrakech / wg11 / JVET-M0261-v1.zip), the sub-image design is similar to that in the latest VVC draft specification, but with some differences. One of these is the explicit indication of the sub-image ID for each sub-image in the SPS and in each stripe header to enable sub-image-based sub-stream extraction without altering the decoded stripe NAL units. In this approach, the sub-image ID for each sub-image remains constant across all images in the CVS.

[0074] In typical use cases, each sub-image contains only one stripe. However, unfortunately, the ability to indicate this typical use case is lacking or absent. In fact, knowing that each sub-image contains only one stripe can prove useful. For example, when the layout of the stripes in an image is known, the bits used to indicate the sub-image layout can be completely omitted from the bitstream, since the sub-image layout is exactly the same as the stripe layout.

[0075] This paper discloses a technique for using flags in the bitstream to indicate whether each sub-image in an image contains one and only one rectangular stripe. Because a sub-image is identical to a stripe, when the flag indicates that each sub-image contains only a single rectangular stripe, syntax elements indicating the number and layout of sub-images (e.g., the position and size of each sub-image) can be omitted in the bitstream. In other words, when the flag indicates that each sub-image contains only a single rectangular stripe, the bitstream only needs to include the syntax elements for the stripe. This reduces the amount of information carried in the bitstream, thereby reducing the use of memory, processor, and network resources and improving decoding efficiency. Therefore, the encoder / decoder (also known as a codec) in video decoding represents an improvement over existing codecs. In fact, the improved video decoding process provides a better user experience when video is sent, received, and / or viewed.

[0076] Figure 1 This is a flowchart of an exemplary method 100 for decoding a video signal. Specifically, the video signal is encoded on the encoder side. The encoding process compresses the video signal using various mechanisms to reduce the size of the video file. A smaller file size allows the compressed video file to be sent to the user while reducing associated bandwidth overhead. The decoder then decodes the compressed video file to reconstruct the original video signal for display to the end user. The decoding process is generally the reverse of the encoding process so that the video signal reconstructed by the decoder can be consistent with the video signal on the encoder side.

[0077] In step 101, the video signal is input into the encoder. For example, the video signal can be an uncompressed video file stored in memory. Alternatively, the video file can be captured by a video capture device such as a camera and encoded to support live video streaming. The video file can include both audio and video components. The video components consist of a series of image frames. When these image frames are viewed sequentially, they create a visual effect of motion. These frames include pixels represented by light, referred to herein as luminance components (or luminance samples), and pixels represented by color, referred to herein as chrominance components (or chrominance samples). In some examples, these frames may also include depth values ​​to support three-dimensional viewing.

[0078] In step 103, the video is segmented into blocks. Segmentation involves subdividing the pixels in each frame into square and / or rectangular blocks for compression. For example, in High Efficiency Video Coding (HEVC) (also known as H.265 and MPEG-H Part 2), frames can first be divided into coding tree units (CTUs), which are blocks of a predefined size (e.g., 64 pixels × 64 pixels). These CTUs include both luma and chroma samples. The coding tree can be used to divide the CTUs into blocks, and then these blocks are repeatedly subdivided until a configuration that supports further coding is obtained. For example, the luma component of a frame can be subdivided until the blocks contain relatively uniform luma values. Similarly, the chroma component of a frame can be subdivided until the blocks contain relatively uniform chroma values. Therefore, the segmentation mechanism varies depending on the content of the video frame.

[0079] In step 105, various compression mechanisms are used to compress the image patches segmented in step 103. For example, inter-frame prediction and / or intra-frame prediction can be used. Inter-frame prediction is designed to take advantage of the fact that objects in a typical scene often appear in consecutive frames. Accordingly, the blocks describing objects in a reference frame do not need to be described repeatedly in adjacent frames. Specifically, an object (e.g., a table) can remain in a fixed position across multiple frames. Therefore, the table is described once, and adjacent frames can rereference the reference frame. Pattern matching mechanisms can be used to match objects across multiple frames. Furthermore, moving objects can be represented across multiple frames due to object movement or camera movement. In a specific example, a video can display a car moving across the screen across multiple frames. Motion vectors can be used to describe this movement. A motion vector is a two-dimensional vector that provides the offset of an object's coordinates in one frame to its coordinates in the reference frame. Therefore, inter-frame prediction can encode image patches in the current frame into a set of motion vectors representing the offset of the image patch in the current frame from the corresponding patch in the reference frame.

[0080] Intra-frame prediction encodes blocks within a common frame. Intra-frame prediction leverages the fact that luma and chroma components tend to cluster within a single frame. For example, a patch of green in a section of a tree is often adjacent to several similar patches of green. Intra-frame prediction employs various directional prediction modes (e.g., 33 in HEVC), planar modes, and direct current (DC) modes. These directional modes indicate that samples in the current block are similar / identical to samples in neighboring blocks in the corresponding direction. Planar modes represent a series of blocks in a row / column (e.g., a plane) that can be interpolated based on neighboring blocks at the edges of that row. Planar modes essentially represent a smooth transition of light / color across rows / columns by employing a relatively constant slope for the changing values. DC modes are used for boundary smoothing and indicate that a block is similar / identical to the average of samples in all neighboring blocks associated with the angular direction of the directional prediction mode. Accordingly, intra-frame predicted blocks can represent image blocks as values ​​of various relational prediction modes rather than actual values. Furthermore, inter-frame predicted blocks can represent image blocks as motion vector values ​​rather than actual values. In either case, the predicted block may not accurately represent the image block in some situations. All differences are stored in residual blocks. Transformations can be applied to the residual blocks to further compress the file.

[0081] In step 107, various filtering techniques can be applied. In HEVC, filters are applied according to an in-loop filtering scheme. The block-based prediction described above may produce a blocky image on the decoder side. Furthermore, the block-based prediction scheme can encode blocks and then reconstruct the encoded blocks for subsequent use as reference blocks. The in-loop filtering scheme iteratively applies noise suppression filters, deblocking filters, adaptive loop filters, and sample adaptive offset (SAO) filters to blocks / frames. These filters reduce block artifacts, allowing for accurate reconstruction of the encoded file. Furthermore, these filters reduce artifacts in the reconstructed reference block, making it less likely that artifacts will generate other artifacts in subsequent blocks encoded based on the reconstructed reference block.

[0082] Once the video signal has been segmented, compressed, and filtered, in step 109, the resulting data is encoded into a bitstream. The bitstream includes the data described above, as well as any signaling data required to support proper video signal reconstruction at the decoder side. For example, this data may include segmentation data, prediction data, residual blocks, and various flags that provide decoding instructions to the decoder. The bitstream can be stored in memory for transmission to the decoder upon request. The bitstream can also be broadcast and / or multicast to multiple decoders. Creating the bitstream is an iterative process. Accordingly, steps 101, 103, 105, 107, and 109 can be performed consecutively and / or simultaneously across multiple frames and blocks. Figure 1 The order shown is presented for clarity and ease of description and is not intended to restrict the video decoding process to a specific order.

[0083] In step 111, the decoder receives the bitstream and begins the decoding process. Specifically, the decoder uses an entropy decoding scheme to convert the bitstream into corresponding syntax data and video data. In step 111, the decoder uses the syntax data in the bitstream to determine the segmented portions of the frames. The segmentation should match the result of the block segmentation in step 103. The entropy encoding / decoding used in step 111 is described below. The encoder makes many choices during compression, for example, selecting a block segmentation scheme from several possible choices based on the spatial placement of values ​​in one or more input images. Indicating the exact choice may take a large number of bits (bins). As used herein, a “bit” is a binary value as a variable (e.g., a bit value that may vary depending on the content). Entropy encoding causes the encoder to discard any options that are obviously unsuitable for a particular situation, leaving a set of available options. A codeword is then assigned to each available option. The length of the codeword depends on the number of available options (e.g., one binary symbol corresponds to two options, two binary symbols correspond to three to four options, and so on). The encoder then encodes the codewords for the selected options. This scheme reduces the codeword size because the codeword is as large as expected, thus uniquely indicating a selection from a small subset of available options, rather than uniquely indicating a selection from a potentially large set of all possible options. The decoder then decodes this selection by determining this set of available options in a similar manner to the encoder. By determining this set of available options, the decoder can read the codeword and determine the selection made by the encoder.

[0084] In step 113, the decoder performs block decoding. Specifically, the decoder employs an inverse transform to generate residual blocks. Then, the decoder uses the residual blocks and corresponding prediction blocks to reconstruct image blocks based on the segmentation. The prediction blocks may include intra-frame prediction blocks and inter-frame prediction blocks generated by the encoder in step 105. Next, the reconstructed image blocks are placed in frames of the reconstructed video signal based on the segmentation data determined in step 111. The syntax used in step 113 can also be indicated in the bitstream via entropy coding as described above.

[0085] In step 115, the frames of the reconstructed video signal are filtered in a manner similar to step 107 on the encoder side. For example, noise suppression filters, deblocking filters, adaptive loop filters, and SAO filters can be applied to the frames to remove block artifacts. Once the frames have been filtered, in step 117, the video signal can be output to a display for viewing by the end user.

[0086] Figure 2This is a schematic diagram of an exemplary encoding and decoding (encoding / decoding) system 200 for video decoding. Specifically, the encoding / decoding system 200 provides functionality to implement the operation method 100. The encoding / decoding system 200 is used broadly to describe components used on both the encoder and decoder sides. The encoding / decoding system 200 receives a video signal and segments it, as described with reference to steps 101 and 103 in the operation method 100, to obtain a segmented video signal 201. Then, when acting as an encoder, the encoding / decoding system 200 compresses the segmented video signal 201 into an encoded bitstream, as described with reference to steps 105, 107, and 109 in the method 100. When acting as a decoder, the encoding / decoding system 200 generates an output video signal from the bitstream, as described with reference to steps 111, 113, 115, and 117 in the operation method 100. The encoding / decoding system 200 includes a general decoder control component 211, a transform scaling and quantization component 213, an intra-frame estimation component 215, an intra-frame prediction component 217, a motion compensation component 219, a motion estimation component 221, a scaling and inverse transform component 229, a filter control and analysis component 227, an intra-loop filter component 225, a decoded image buffer component 223, and a header format and context-adaptive binary arithmetic coding (CABAC) component 231. These components are coupled as shown in the figure. Figure 2 In the diagram, black lines represent the movement of data to be encoded / decoded, while dashed lines represent the movement of control data controlling the operation of other components. All components in the encoding / decoding system 200 can reside in the encoder. The decoder may include a subset of the components in the encoding / decoding system 200. For example, the decoder may include an intra-frame prediction component 217, a motion compensation component 219, a scaling and inverse transform component 229, an in-loop filter component 225, and a decoded image buffer component 223. These components are described below.

[0087] The segmented video signal 201 is a captured video sequence that has been segmented into pixel blocks by a coding tree. The coding tree uses various partitioning patterns to subdivide pixel blocks into smaller pixel blocks. These blocks can then be further subdivided into even smaller blocks. These blocks can be called nodes on the coding tree. Larger parent nodes are divided into smaller child nodes. The number of times a node is subdivided is called the node / coding tree depth. In some cases, the resulting blocks can be included in a coding unit (CU). For example, a CU can be a sub-part of a CTU, including a luma block, one or more red chromatic aberration (Cr) blocks, and one or more blue chromatic aberration (Cb) blocks, as well as the corresponding syntax instructions for the CU. Partitioning patterns can include binary trees (BT), triple trees (TT), and quadtrees (QT), used to divide nodes into two, three, or four child nodes of different shapes, depending on the partitioning pattern used. The segmented video signal 201 is forwarded to the general decoder control component 211, the transform scaling and quantization component 213, the intra-frame estimation component 215, the filter control and analysis component 227, and the motion estimation component 221 for compression.

[0088] The general-purpose decoder control component 211 makes decisions related to encoding images from a video sequence into a bitstream based on application constraints. For example, the general-purpose decoder control component 211 manages the optimization of bitrate / bitstream size relative to reconstruction quality. These decisions can be made based on storage space / bandwidth availability and image resolution requests. The general-purpose decoder control component 211 also manages buffer utilization based on transmission speed to mitigate buffer underloading and overloading issues. To address these issues, the general-purpose decoder control component 211 manages segmentation, prediction, and filtering performed by other components. For example, the general-purpose decoder control component 211 can dynamically increase compression complexity to improve resolution and increase bandwidth utilization, or decrease compression complexity to decrease resolution and bandwidth utilization. Therefore, the general-purpose decoder control component 211 controls other components in the encoding / decoding system 200 to balance video signal reconstruction quality with bitrate. The general-purpose decoder control component 211 generates control data that is used to control the operation of other components. It also forwards control data to the header format and CABAC component 231 for encoding into the bitstream, thereby instructing the decoder to use the parameters when decoding.

[0089] The segmented video signal 201 is also sent to the motion estimation component 221 and the motion compensation component 219 for inter-frame prediction. The frames or stripes of the segmented video signal 201 can be divided into multiple video blocks. The motion estimation component 221 and the motion compensation component 219 perform inter-frame prediction decoding on the received video blocks relative to one or more blocks in one or more reference frames to provide timing prediction. The encoding / decoding system 200 can execute multiple decoding rounds to select an appropriate decoding mode for each video data block, and so on.

[0090] Motion estimation component 221 and motion compensation component 219 can be highly integrated, but for conceptual purposes, they are described separately. Motion estimation performed by motion estimation component 221 is the process of generating motion vectors, which are used to estimate the motion of video blocks. For example, motion vectors can represent the displacement of a decoded object relative to a prediction block. A prediction block is a block found to be highly matched to the block to be decoded in terms of pixel differences. A prediction block can also be referred to as a reference block. This pixel difference can be determined by the sum of absolute differences (SAD), the sum of square differences (SSD), or other difference measures. HEVC employs several decoded objects, including CTUs, coding tree blocks (CTBs), and CUs. For example, a CTU can be divided into multiple CTBs, and then a CTB can be divided into CBs to be included in a CU. CUs can be encoded as prediction units (PUs) that include prediction data and / or transform units (TUs) that include transform residual data of the CUs. Motion estimation component 221 uses rate-distortion analysis as part of the rate-distortion optimization process to generate motion vectors, PU, ​​and TU. For example, motion estimation component 221 can determine multiple reference blocks, multiple motion vectors, etc., for the current block / frame, and can select reference blocks, motion vectors, etc., with optimal rate-distortion characteristics. Optimal rate-distortion characteristics balance the quality of video reconstruction (e.g., the amount of data loss due to compression) and decoding efficiency (e.g., the size of the final encoded value).

[0091] In some examples, the codec system 200 can calculate the values ​​of sub-integer pixel positions of the reference image stored in the decoded image buffer component 223. For example, the video codec system 200 can interpolate the values ​​of quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference image. Therefore, the motion estimation component 221 can perform motion search relative to integer and fractional pixel positions and output motion vectors with fractional pixel precision. The motion estimation component 221 calculates the motion vector of the PU for a video block in the inter-frame decoded strip by comparing the position of the PU with the position of the predicted block in the reference image. The motion estimation component 221 outputs the calculated motion vector as motion data to the header format and CABAC component 231 for encoding, and then outputs it as motion data to the motion compensation component 219.

[0092] The motion compensation performed by motion compensation component 219 may involve acquiring or generating prediction blocks based on motion vectors determined by motion estimation component 221. Similarly, in some examples, motion estimation component 221 and motion compensation component 219 may be functionally integrated. Upon receiving the motion vector of the PU for the current video block, motion compensation component 219 can locate the prediction block to which the motion vector points. Then, the pixel values ​​of the prediction block are subtracted from the pixel values ​​of the current video block being decoded to obtain the pixel difference, thereby forming the residual video block. Generally, motion estimation component 221 performs motion estimation relative to the luma component, while motion compensation component 219 uses the motion vectors calculated based on the luma component for both the chroma and luma components. The prediction block and residual block are then forwarded to transform scaling and quantization component 213.

[0093] The segmented video signal 201 is also sent to the intra-frame estimation component 215 and the intra-frame prediction component 217. Similar to the motion estimation component 221 and the motion compensation component 219, the intra-frame estimation component 215 and the intra-frame prediction component 217 can be highly integrated, but for conceptual purposes, they are described separately. The intra-frame estimation component 215 and the intra-frame prediction component 217 perform intra-frame prediction relative to the blocks in the current frame, instead of the inter-frame prediction performed between frames by the motion estimation component 221 and the motion compensation component 219 as described above. Specifically, the intra-frame estimation component 215 determines an intra-frame prediction mode for encoding the current block. In some examples, the intra-frame estimation component 215 selects a suitable intra-frame prediction mode from multiple tested intra-frame prediction modes to encode the current block. The selected intra-frame prediction mode is then forwarded to the header format and CABAC component 231 for encoding.

[0094] For example, intra-frame estimation component 215 performs rate-distortion analysis on various tested intra-prediction modes to calculate rate-distortion values ​​and selects the intra-prediction mode with the best rate-distortion characteristics among the tested modes. Rate-distortion analysis typically determines the amount of distortion (or error) between a coded block and the original uncoded block encoded to produce the coded block, as well as the bit rate (e.g., number of bits) used to generate the coded block. Intra-frame estimation component 215 calculates the ratio based on the distortion and rate of various coded blocks to determine the intra-prediction mode that exhibits the best rate-distortion value for the block. Additionally, intra-frame estimation component 215 can be used to decode depth blocks of a depth image using a depth-modeling mode (DMM) according to rate-distortion optimization (RDO).

[0095] When implemented on the encoder, intra-prediction component 217 can generate residual blocks from the prediction blocks according to the selected intra-prediction mode determined by intra-estimation component 215, or when implemented on the decoder, it can read residual blocks from the bitstream. The residual blocks comprise the difference between the prediction blocks and the original blocks, represented as a matrix. The residual blocks are then forwarded to transform-scaling and quantization component 213. Both intra-estimation component 215 and intra-prediction component 217 can operate on both the luma and chroma components.

[0096] Transform scaling and quantization component 213 is used to further compress the residual block. Transform scaling and quantization component 213 applies transforms such as discrete cosine transform (DCT), discrete sine transform (DST), or conceptually similar transforms to the residual block, thereby producing a video block that includes the values ​​of the residual transform coefficients. Wavelet transform, integer transform, subband transform, or other types of transforms can also be used. Transforms can convert residual information from the pixel value domain to the transform domain, such as the frequency domain. Transform scaling and quantization component 213 is also used to scale the transform residual information according to frequencies, etc. This scaling involves applying a scaling factor to the residual information to quantize different frequency information at different granularities, which may affect the final visual quality of the reconstructed video. Transform scaling and quantization component 213 is also used to quantize the transform coefficients to further reduce the bit rate. The quantization process can reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, transform scaling and quantization component 213 can then scan the matrix that includes the quantized transform coefficients. The quantization transform coefficients are forwarded to the header format and CABAC component 231 for encoding into the bitstream.

[0097] The scaling and inverse transform component 229 performs the opposite operation to the transform scaling and quantization component 213 to support motion estimation. The scaling and inverse transform component 229 applies inverse scaling, inverse transform, and / or inverse quantization to reconstruct the residual block in the pixel domain. For example, the residual block is subsequently used as a reference block. This reference block can become the prediction block for another current block. The motion estimation component 221 and / or the motion compensation component 219 can compute the reference block by adding the residual block back to the corresponding prediction block for motion estimation in subsequent blocks / frames. Filters are applied to the reconstructed reference block to reduce artifacts generated during scaling, quantization, and transform. These artifacts can cause inaccurate predictions (and generate additional artifacts) when predicting subsequent blocks.

[0098] The filter control analysis component 227 and the in-loop filter component 225 apply filters to residual blocks and / or reconstructed image blocks. For example, a transformed residual block from the scaling and inverse transform component 229 can be combined with a corresponding predicted block from the intra-prediction component 217 and / or the motion compensation component 219 to reconstruct the original image block. The filter can then be applied to the reconstructed image block. In some examples, the filter can also be applied to the residual block. As... Figure 2 The other components, filter control analysis component 227 and in-loop filter component 225, are highly integrated and can be implemented together, but for conceptual purposes, they are described separately. Filters applied to the reconstructed reference block are applied to specific spatial regions. These filters include multiple parameters to adjust how they are used. Filter control analysis component 227 analyzes the reconstructed reference block to determine the locations where these filters need to be used and sets the corresponding parameters. This data is forwarded as filter control data to the header format and CABAC component 231 for encoding. In-loop filter component 225 applies these filters based on the filter control data. These filters may include deblocking filters, noise suppression filters, SAO filters, and adaptive loop filters. These filters may be applied in the spatial / pixel domain (e.g., for reconstructed pixel blocks) or the frequency domain, depending on the example.

[0099] When operating as an encoder, filtered reconstructed image blocks, residual blocks, and / or prediction blocks are stored in the decoded image buffer component 223 for subsequent use in motion estimation, as described above. When operating as a decoder, the decoded image buffer component 223 stores the reconstructed and filtered blocks and forwards them to the display as part of the output video signal. The decoded image buffer component 223 can be any storage device capable of storing prediction blocks, residual blocks, and / or reconstructed image blocks.

[0100] The header format and CABAC component 231 receive data from various components in the encoding / decoding system 200 and encode this data into an encoded bitstream for transmission to the decoder. Specifically, the header format and CABAC component 231 generate various headers to encode control data (e.g., general control data and filter control data). Furthermore, prediction data (including intra-frame prediction data and motion data) and residual data in the form of quantized transform coefficient data are encoded into the bitstream. The final bitstream contains all the information required by the decoder to reconstruct the original segmented video signal 201. This information may also include an intra-frame prediction mode index table (also called a codeword map), definitions of the coding context of various blocks, indications of the most likely intra-frame prediction modes, indications of segmentation information, etc. This data may be encoded using entropy coding. For example, this information can be encoded using context adaptive variable length coding (CAVLC), CABAC, syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding techniques. After entropy coding, the encoded bitstream can be sent to another device (e.g., a video decoder) or archived for later transmission or retrieval.

[0101] Figure 3 This is a block diagram of an exemplary video encoder 300. The video encoder 300 can be used to implement the encoding function of the encoding / decoding system 200 and / or perform steps 101, 103, 105, 107, and / or 109 in the operation method 100. The encoder 300 segments the input video signal to obtain a segmented video signal 301, which is substantially similar to the segmented video signal 201. Then, the segmented video signal 301 is compressed by components in the encoder 300 and encoded into a bitstream.

[0102] Specifically, the segmented video signal 301 is forwarded to the intra-prediction component 317 for intra-frame prediction. The intra-prediction component 317 can be substantially similar to the intra-estimation component 215 and the intra-prediction component 217. The segmented video signal 301 is also forwarded to the motion compensation component 321 for inter-frame prediction based on the reference block in the decoded image buffer component 323. The motion compensation component 321 can be substantially similar to the motion estimation component 221 and the motion compensation component 219. The predicted block and residual block from the intra-prediction component 317 and the motion compensation component 321 are forwarded to the transform and quantization component 313 for transforming and quantizing the residual block. The transform and quantization component 313 can be substantially similar to the transform scaling and quantization component 213. The transform quantized residual block and the corresponding predicted block (along with the associated control data) are forwarded to the entropy coding component 331 for encoding in the bitstream. The entropy coding component 331 can be substantially similar to the header format and CABAC component 231.

[0103] The transformed and quantized residual block and / or the corresponding prediction block are also forwarded from the transform and quantization component 313 to the inverse transform and dequantization component 329 to be reconstructed as a reference block for use by the motion compensation component 321. The inverse transform and dequantization component 329 can be substantially similar to the scaling and inverse transform component 229. According to the example, the intra-loop filter in the intra-loop filter component 325 is also applied to the residual block and / or the reconstructed reference block. The intra-loop filter component 325 can be substantially similar to the filter control analysis component 227 and the intra-loop filter component 225. The intra-loop filter component 325 may include multiple filters as described with reference to the intra-loop filter component 225. The filtered block is then stored in the decoded image buffer component 323 as a reference block for use by the motion compensation component 321. The decoded image buffer component 323 can be substantially similar to the decoded image buffer component 223.

[0104] Figure 4 This is a block diagram of an exemplary video decoder 400. The video decoder 400 can be used to implement the decoding function of the encoding / decoding system 200 and / or perform steps 111, 113, 115, and / or 117 in the operation method 100. The decoder 400 receives a bitstream from the encoder 300, etc., and generates a reconstructed output video signal based on the bitstream for display to the end user.

[0105] The bitstream is received by entropy decoding component 433. Entropy decoding component 433 is used to perform entropy decoding schemes, such as CAVLC, CABAC, SBAC, PIPE decoding, or other entropy decoding techniques. For example, entropy decoding component 433 can use header information to provide context for parsing additional data encoded as codewords in the bitstream. Decoded information includes any information required to decode the video signal, such as general control data, filter control data, segmentation information, motion data, prediction data, and quantization transform coefficients in the residual block. The quantization transform coefficients are forwarded to inverse transform and dequantization component 429 to reconstruct the residual block. Inverse transform and dequantization component 429 may be similar to inverse transform and dequantization component 329.

[0106] The reconstructed residual block and / or predicted block are forwarded to the intra-prediction component 417 for reconstruction into image blocks based on the intra-prediction operation. The intra-prediction component 417 can be similar to the intra-estimation component 215 and the intra-prediction component 217. Specifically, the intra-prediction component 417 uses a prediction mode to locate a reference block in the frame and applies the residual block to the above result to reconstruct the intra-predicted image block. The reconstructed intra-predicted image block and / or residual block, along with the corresponding inter-frame prediction data, are forwarded to the decoding image buffer component 423 via the intra-loop filter component 425. The decoding image buffer component 423 and the intra-loop filter component 425 can be substantially similar to the decoding image buffer component 223 and the intra-loop filter component 225, respectively. The intra-loop filter component 425 filters the reconstructed image block, the residual block, and / or the predicted block. This information is stored in the decoding image buffer component 423. The reconstructed image block from the decoding image buffer component 423 is forwarded to the motion compensation component 421 for inter-frame prediction. Motion compensation component 421 can be substantially similar to motion estimation component 221 and / or motion compensation component 219. Specifically, motion compensation component 421 uses the motion vectors of a reference block to generate a prediction block and applies the residual block to the result to reconstruct the image block. The resulting reconstructed block can also be forwarded to decoding image buffer component 423 via in-loop filter component 425. Decoding image buffer component 423 continues to store other reconstructed image blocks. These reconstructed image blocks can be reconstructed into frames using segmentation information. These frames can also be placed in a sequence. This sequence is output as a reconstructed output video signal to a display.

[0107] Figure 5 This is a schematic diagram of an exemplary bitstream 500 comprising an encoded video sequence. For example, bitstream 500 may be generated by codec system 200 and / or encoder 300, and decoded by codec system 200 and / or decoder 400. In another example, bitstream 500 may be generated by encoder in step 109 of method 100 and used by decoder in step 111.

[0108] Bitstream 500 includes a sequence parameter set (SPS) 510, multiple picture parameter sets (PPS) 512, a strip header 514, and picture data 520. SPS 510 contains sequence data shared by all pictures in the video sequence contained in bitstream 500. This data may include image size, bit depth, encoding tool parameters, bitrate constraints, etc. PPS 512 includes one or more parameters specific to each picture. Therefore, each picture in the video sequence can reference a PPS 512. PPS 512 can represent the decoding tools, quantization parameters, offsets, and picture-specific decoding tool parameters (e.g., filter control parameters) available for the strip (or block) in the corresponding picture. Strip header 514 includes parameters specific to each strip in the picture. Therefore, each strip in the video sequence can have a strip header 514. Strip header 514 may include strip information, picture order count (POC), a list of reference pictures, prediction weights, block entry points, deblocking effect parameters, etc. It should be noted that some systems refer to the strip header 514 as the chunk group header and use this information to support chunk groups rather than stripes.

[0109] Image data 520 includes video data encoded according to inter-frame prediction and / or intra-frame prediction, as well as corresponding transform and quantization residual data. This type of image data 520 is categorized according to the segmentation type used to segment the images prior to encoding. For example, the images in image data 520 include one or more images 521. A sequence or series of images 521 may be referred to as CVS 527. As used herein, CVS 527 is a sequence of access units (AUs) that, in decoding order, includes a coded video sequence start (CVSS) AU followed by zero or more AUs that are not CVSS AUs, including all subsequent AUs up to but not including any subsequent AUs that are CVSS AUs. A CVSS AU is an AU in which each layer specified by the video parameter set (VPS) has a prediction unit (PU), and the encoded image in each PU is the coded layer video sequence start (CLVSS) image. In one embodiment, each image 521 is within an AU. A PU is a set of NAL units that are associated with each other according to a specified classification rule. The NAL units are consecutive in the decoding order and contain exactly one encoded image.

[0110] CVS is the encoded video sequence of each coded layer video sequence (CLVS) in a video stream. It's important to note that CVS and CLVS are the same when the video stream consists of a single layer. They only differ when the video stream contains multiple layers.

[0111] Each image 521 can be divided into sub-images 523. Sub-image 523 is a segmented portion of the image generated by horizontal and vertical boundaries. Sub-image 523 can be rectangular and / or square. Specifically, sub-image 523 comprises four sides connected by right angles. These four sides include two pairs of parallel sides. Furthermore, the sides in each pair of parallel sides are of equal length. Therefore, sub-image 523 can be any rectangular shape, with a square being a special case of a rectangle where all four sides are of equal length.

[0112] Each sub-image 523 can be divided into strips 529. It should be noted that in some applications, strips 529 may be referred to as blocks and / or groups of blocks. Strips 529 can be rectangular and / or square, meaning that the strip has a rectangular or square shape (as opposed to, for example, raster scan strips that may not have a rectangular or square shape). Specifically, strip 529 comprises four sides connected by right angles. These four sides include two pairs of parallel sides. Furthermore, the sides in each pair of parallel sides are of equal length. Therefore, strip 529 can be any rectangular shape, with squares being a special case of rectangles where all four sides are of equal length.

[0113] In one embodiment, each sub-image 523 includes a rectangular region of one or more stripes 529 within an image 521. That is, each sub-image 523 is divided into one or more stripes 529, and each image 521 is divided into one or more sub-images 523.

[0114] According to the example, strip 529 may or may not allow prediction based on other strips 529. Within sub-image 523, each strip 529 may have a unique strip index. The strip index is a numerical identifier chosen by the program and used to distinguish one strip 529 from other strips. For example, the strip index may increment numerically from top to bottom or from left to right.

[0115] It should be noted that in some examples, a stripe identifier (ID) can also be assigned to stripe 529. The stripe ID is an assigned identifier used to distinguish one stripe 529 from other stripes. In some examples, calculations can use the stripe ID instead of the stripe index. Furthermore, in some examples, the stripe ID can be assigned the same value as the stripe index. The stripe index and / or stripe ID can be used to indicate a group of stripes that includes stripe 529. For example, the stripe index and / or stripe ID can be used to map image data associated with stripe 529 to the appropriate location for display. Multiple stripes 529 can be referred to as a tile set. A tile set is a related collection of stripes 529 that can be extracted and decoded individually, for example, to support the display of regions of interest and / or to support parallel processing. Stripes 529 in a tile set can be decoded without referencing stripes 529 outside the tile set. Each stripe 529 can be assigned to a corresponding tile set, therefore image 521 can include multiple tile sets.

[0116] Slice 529 is further divided into coding tree units (CTUs). Each CTU is further divided into coding blocks based on the coding tree. These coding blocks can then be encoded / decoded according to a prediction mechanism.

[0117] In one embodiment, PPS 512 includes a single stripe flag 532 for each sub-image. In one embodiment, the value of the single stripe flag 532 for each sub-image is a first value (e.g., 1). The first value indicates that each sub-image (e.g., sub-image 523) of the reference PPS 512 (as opposed to another PPS in the bitstream 500) may include one and only one rectangular stripe (e.g., stripe 529). That is, each sub-image includes a single rectangular stripe. Therefore, the sub-image and the rectangular stripe are the same size.

[0118] In one embodiment, the value of the single-slice-per-subpic flag 532 is a second value (e.g., 0). This second value indicates that each subpicture (e.g., subpicture 523) of the reference PPS 512 (as opposed to another PPS in the bitstream 500) may include one or more rectangular slices (e.g., slice 529). That is, each subpicture may include a single rectangular slice or multiple slices. When a subpicture includes multiple slices, the size of the subpicture differs from the size of a single slice. In one embodiment, the single-slice-per-subpic flag 532 is represented as pps_single_slice_per_subpic_flag.

[0119] In one embodiment, PPS 512 further includes a stripe count flag 534 in the image. The stripe count flag 534 is used to determine the number of rectangular stripes (e.g., stripe 529) in each image (e.g., image 521) of the reference PPS 512.

[0120] In one embodiment, when the value of the single stripe flag 532 for each sub-image is a second value, the stripe quantity flag 534 in the image is used to indicate the number of rectangular stripes in each image of the reference PPS.

[0121] In one embodiment, the value of the stripe count flag 534 plus 1 indicates the number of stripes in each image of the reference PPS. For example, when the value of the stripe count flag 534 is 3, the total number of stripes in the image is 4 (e.g., 3+1).

[0122] In one embodiment, the number of stripes in the image, 534, is represented as pps_num_slices_in_pic_minus1.

[0123] Figure 6 This is one embodiment of a decoding method 600 implemented by a video decoder (e.g., video decoder 400). Method 600 can be executed after receiving a decoded bitstream directly or indirectly from a video encoder (e.g., video encoder 300). Method 600 improves the decoding process by using a flag in the bitstream indicating whether each sub-image in an image contains one and only one rectangular stripe. Because a sub-image is the same as a stripe, when the flag indicates that each sub-image contains only a single rectangular stripe, syntax elements indicating the number and layout of sub-images (e.g., the position and size of each sub-image) can be omitted in the bitstream. That is, when the flag indicates that each sub-image contains only a single rectangular stripe, the bitstream only needs to include the syntax elements for the stripe. This reduces the amount of information carried in the bitstream, reduces redundancy, and improves decoding efficiency. Therefore, the encoder / decoder (also called a codec) in video decoding is an improvement over existing codecs. In fact, the improved video decoding process provides a better user experience when the video is sent, received, and / or viewed.

[0124] In step 602, the video decoder receives a bitstream (e.g., bitstream 500), wherein the bitstream includes a PPS (e.g., PPS 512), and the PPS includes a single slice flag per subpic (e.g., single slice flag per subpic 532). In one embodiment, the single slice flag per subpic is represented as pps_single_slice_per_subpic_flag.

[0125] In one embodiment, the PPS further includes a stripe count flag (e.g., stripe count flag 534). The value of the stripe count flag plus 1 indicates the number of stripes in each image referencing the PPS. In one embodiment, the stripe count flag is represented as pps_num_slices_in_pic_minus1.

[0126] In step 604, the video decoder determines whether the value of the single stripe flag for each sub-image is a first value or a second value. The first value indicates that each sub-image referencing the PPS includes one and only one rectangular stripe. The second value indicates that each sub-image referencing the PPS may include one or more rectangular stripes (e.g., one or more rectangular stripes). In one embodiment, the first value is 1 (1) and the second value is 0 (0). In one embodiment, each sub-image includes a rectangular region of one or more stripes within the image.

[0127] In step 606, when the value of the single stripe flag for each sub-image is the first value, the video decoder decodes the one and only one rectangular stripe; when the value of the single stripe flag for each sub-image is the second value, the video decoder decodes the one or more rectangular stripes. Once decoded, the one and only one rectangular stripe or the one or more rectangular stripes (as appropriate) are used to generate or produce an image or video sequence for display to a user on the monitor or screen of an electronic device (e.g., a smartphone, tablet, laptop, personal computer, etc.).

[0128] Figure 7 This is an embodiment of a method 700 for encoding a video bitstream, implemented by a video encoder (e.g., video encoder 300). Method 700 can be performed when an image (e.g., an image in a video) is to be encoded into a video bitstream and then sent to a video decoder (e.g., video decoder 400). Method 700 improves the encoding process by using a flag in the bitstream indicating whether each sub-image in the image contains one and only one rectangular stripe. Because a sub-image is the same as a stripe, when the flag indicates that each sub-image contains only a single rectangular stripe, syntax elements indicating the number and layout of sub-images (e.g., the position and size of each sub-image) can be omitted in the bitstream. That is, when the flag indicates that each sub-image contains only a single rectangular stripe, only the syntax elements for the stripe need to be included in the bitstream. This reduces the amount of information carried in the bitstream, reduces redundancy, and improves decoding efficiency. Therefore, the encoder / decoder (also called a codec) in video decoding represents an improvement over existing codecs. In fact, the improved video decoding process provides a better user experience when videos are sent, received, and / or watched.

[0129] In step 702, when each sub-image (e.g., sub-image 523) of the reference PPS (e.g., PPS 512) includes one and only one rectangular stripe, the video encoder sets the single stripe flag per sub-image (e.g., single stripe flag per sub-image 532) to a first value, and when each sub-image of the reference PPS may include one or more rectangular stripes, the video encoder sets the single stripe flag per sub-image to a second value.

[0130] In one embodiment, the single-slice flag per subpic is represented as pps_single_slice_per_subpic_flag. In one embodiment, the first value is 1 (1) and the second value is 0 (0).

[0131] In one embodiment, the PPS further includes a slice count flag (e.g., slice count flag 534). In one embodiment, the video encoder sets the slice count flag to a certain value. The value of the slice count flag plus 1 indicates the number of slices in each image referenced by the PPS. In one embodiment, the slice count flag is represented as pps_num_slices_in_pic_minus1. In one embodiment, each sub-image includes a rectangular region of one or more slices within the image.

[0132] In step 704, the video encoder encodes the PPS into a bitstream (e.g., bitstream 500), wherein the PPS includes a single stripe marker for each sub-image. In step 706, the video encoder stores the bitstream for transmission to the decoder. The bitstream may be stored in memory until it is transmitted to the video decoder. Upon receiving the encoded video bitstream, the video decoder can decode it (e.g., as described above) to generate or produce an image or video sequence for display to a user on the monitor or screen of an electronic device (e.g., a smartphone, tablet, laptop, personal computer, etc.).

[0133] The following syntax and semantics can be used to implement the embodiments disclosed herein. The following description is relative to the base text, which is the latest VVC draft specification, assuming that the indication information for sub-image information is sent based on the stripe layout. In other words, only additions and modifications are described, while text in the base text not mentioned below applies as is.

[0134] The PPS syntax is as follows.

[0135]

[0136]

[0137] PPS semantics are as follows.

[0138] A value of 1 for brick_idx_delta_sign_flag[i] indicates a positive sign for bottom_right_brick_idx_delta[i]. A value of 0 for brick_idx_delta_sign_flag[i] indicates a negative sign for bottom_right_brick_idx_delta[i].

[0139] The requirement for stream consistency is that a stripe should include multiple complete blocks, or only a continuous sequence of complete bricks in a single block.

[0140] The variables TopLeftBrickIdx[i], BottomRightBrickIdx[i], NumBricksInSlice[i], BrickToSliceMap[i], SliceColBd[i], SliceRowBd[i], SliceWidth[i], and SliceHeight[i] represent the brick index of the top-left corner of the i-th strip, the brick index of the bottom-right corner of the i-th strip, the number of bricks in the i-th strip, the strip index including the j-th brick, the position of the left boundary of the i-th strip in CTB units, the position of the top boundary of the i-th strip in CTB units, the width of the i-th strip in CTB units, and the height of the i-th strip in CTB units, respectively. Their derivation is as follows:

[0141]

[0142]

[0143] A subpics_present_flag value of 1 indicates the presence of subpics parameters in the PPS RBSP syntax. A subpics_present_flag value of 0 indicates the absence of subpics parameters in the PPS RBSP syntax. When a subpics_present_flag is absent, its value is inferred to be 0.

[0144] It should be noted that when the bitstream is the result of a sub-bitstream extraction process based on sub-images (where the input bitstream contains multiple sub-images for each image, and during the extraction process, VCL NAL units can only be extracted or discarded, but cannot be modified), and the bitstream contains only the correct subset of the sub-images of the input bitstream, the value of subpics_present_flag in the RBSP of the PPS in the bitstream must be equal to 1, even if there is only one sub-image in each image.

[0145] `single_slice_per_subpic_flag` equal to 1 indicates that each sub-image in each image of the reference PPS comprises a rectangular strip. `single_slice_per_subpic_flag` equal to 0 indicates that sub-images in the reference PPS may include multiple rectangular strips. When `rect_slice_flag` equals 1 and `num_slices_in_pic_minus1` equals 0, the value of `single_slice_per_subpic_flag` is inferred to be 1.

[0146] `num_subpics_minus1` plus 1 indicates the number of subpicks in each image of the reference PPS. `num_subpics_minus1` should be in the range of 0 to `num_slices_in_pic_minus1` (inclusive). When `subpics_present_flag` equals 0, the value of `num_subpics_minus1` is inferred to be 0. When the value of `num_subpics_minus1` does not exist and `single_slice_per_subpic_flag` equals 1, then the value of `num_subpics_minus1` is inferred to be equal to `num_slices_in_pic_minus1`. The variable `NumSubPics` is deduced to be equal to `num_subpics_minus1+1`.

[0147] The increment of 1 in bottom_right_slice_idx_length_minus1 indicates the number of bits used to represent the syntax element bottom_right_slice_idx_delta[i]. The value of bottom_right_slice_idx_length_minus1 should be in the range of 0 to Ceil(Log2(num_slices_in_pic_minus1+1))–1 (inclusive).

[0148] When i is greater than 0, bottom_right_slice_idx_delta[i] represents the difference between the slice index of the bottom right corner of the i-th sub-image and the slice index of the bottom right corner of the (i-1)-th sub-image. bottom_right_slice_idx_delta[0] represents the slice index of the bottom right corner of the 0-th sub-image. When single_slice_per_subpic_flag equals 1, the value of bottom_right_slice_idx_delta[i] is inferred to be 1. The value of BottomRightSliceIdx[num_subpics_minus1] is inferred to be equal to num_slices_in_pic_minus1. The length of the syntax element bottom_right_slice_idx_delta[i] is (bottom_right_slice_idx_length_minus1+1) bits.

[0149] A slice_idx_delta_sign_flag[i] equal to 1 indicates a positive sign for bottom_right_slice_idx_delta[i]. A slice_idx_delta_sign_flag[i] equal to 0 indicates a negative sign for bottom_right_brick_idx_delta[i].

[0150] The variables TopLeftSliceIdx[i], BottomRightSliceIdx[i], NumSlicesInSubpic[i], SliceToSubpicMap[j], and SliceSubpicToPicIdx[i][k] represent the slice index of the top-left corner of the i-th sub-image, the slice index of the bottom-right corner of the i-th sub-image, the number of slices in the i-th sub-image, the sub-image index of the sub-image including the j-th slice, and the image-level slice index of the k-th slice in the i-th sub-image, respectively. Their derivation is as follows:

[0151]

[0152]

[0153] The variables SubpicLeft[i], SubpicTop[i], SubpicWidth, and SubpicHeight[i] represent the left boundary position, top boundary position, width, and height of the i-th sub-image in CTB units, respectively. Their derivation is as follows (where each value of i is in the range of 0 to NumSubPics–1 (inclusive)):

[0154] SubpicLeft[i]=SliceColBd[TopLeftSliceIdx[i]]

[0155] SubpicWidth[i]=SliceColBd[BottomRightSliceIdx[i]]+SliceWidth[BottomRightSliceIdx[i]]–SubpicLeft[i]

[0156] SubpicTop[i]=SliceColBd[TopLeftSliceIdx[i]]

[0157] SubpicHeight[i]=SliceColBd[BottomRightSliceIdx[i]]+SliceHeight[BottomRightSliceIdx[i]]–SubpicTop[i]

[0158] A subpic_treated_as_pic_flag[i] equal to 1 indicates that the i-th subpic of each encoded image in CVS is treated as an image during decoding, excluding in-loop filtering operations. A subpic_treated_as_pic_flag[i] equal to 0 indicates that the i-th subpic of each encoded image in CVS is not treated as an image during decoding, excluding in-loop filtering operations. When it does not exist, the value of subpic_treated_as_pic_flag[i] is inferred to be 0.

[0159] A value of 1 for `loop_filter_across_subpic_enabled_flag[i]` indicates that in-loop filtering can be performed across the boundary of the i-th subpic of each encoded image in the CVS. A value of 0 for `loop_filter_across_subpic_enabled_flag[i]` indicates that in-loop filtering is not performed across the boundary of the i-th subpic of each encoded image in the CVS. When `loop_filter_across_subpic_enabled_pic_flag[i]` is not present, its value is inferred to be 1.

[0160] The following constraints apply to the requirement for stream consistency.

[0161] For any two subpics subpicA and subpicB, when the index of subpicA is less than the index of subpicB, any encoded NAL unit of subpicA should be placed after any encoded NAL unit of subpicB in the decoding order.

[0162] The shape of the sub-images should ensure that each sub-image, when decoded, has its entire left and entire top boundaries composed of the image boundary or the boundaries of the previously decoded sub-images.

[0163] The values ​​of SubpicLeft[i], SubpicTop[i], SubpicWidth[i], and SubpicHeight[i] should be the same (where each value of i is in the range of 0 to NumSubPics–1 (inclusive)), regardless of which PPS RBSP the CLVS coding stripe NAL unit references.

[0164] In all PPS RBSPs referenced by the CLVS coding strip NAL unit, the values ​​of subpic_treated_as_pic_flag[i] and loop_filter_across_subpic_enabled_flag[i] should remain the same, where each value of i is in the range of 0 to NumSubPics–1 (inclusive).

[0165] A value of 1 for `subpic_ids_constant_in_clvs_flag` indicates that the values ​​of `subpic_id_len_minus1` and `pps_subpic_id[i]` remain the same across all PPS RBSPs referenced by the NAL unit of the CLVS coding stripe, where each value of `i` is in the range of 0 to `num_subpics_minus1` (inclusive). A value of 0 for `subpic_ids_constant_in_clvs_flag` indicates that the values ​​of `subpic_id_len_minus1` and `pps_subpic_id[i]` may or may not be constrained.

[0166] Increasing 1 to subpic_id_len_minus1 indicates the length of the syntax elements pps_subpic_id[i] and slice_subpic_id, in bits. The value of subpic_id_len_minus1 should be in the range of 3 to 31 (inclusive). Values ​​in the range of 0 to 2 (inclusive) of subpic_id_len_minus1 are reserved for future use by ITU-T|ISO / IEC.

[0167] pps_subpic_id[i] represents the identifier of the i-th subpic.

[0168] Figure 8 This is a schematic diagram of a video decoding device 800 (e.g., a video encoder 300 or a video decoder 400) provided according to an embodiment of the present invention. The video decoding device 800 is suitable for implementing the disclosed embodiments described herein. The video decoding device 800 includes: an input port 810 and a receiving unit (Rx) 820 for receiving data; a processor, logic unit, or central processing unit (CPU) 830 for processing data; a transmitting unit (Tx) 840 and an output port 850 for transmitting data; and a memory 860 for storing data. The video decoding device 800 may further include optical-to-electrical (OE) components and electro-optical (EO) components coupled to the input port 810, the receiving unit 820, the transmitting unit 840, and the output port 850 for the input or output of optical or electrical signals.

[0169] Processor 830 is implemented in both hardware and software. Processor 830 can be implemented as one or more CPU chips, cores (e.g., multi-core processors), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and digital signal processors (DSPs). Processor 830 communicates with input port 810, receiving unit 820, transmitting unit 840, output port 850, and memory 860. Processor 830 includes a decoding module 870. Decoding module 870 implements the disclosed embodiments described above. For example, decoding module 870 implements, processes, prepares, or provides various encoding and decoding functions. Therefore, including decoding module 870 provides a substantial improvement to the functionality of video decoding device 800 and enables transitions between different states of video decoding device 800. Alternatively, decoding module 870 can be implemented with instructions stored in memory 860 and executed by processor 830.

[0170] The video decoding device 800 may also include input and / or output (I / O) devices 880 for data communication with the user. I / O devices 880 may include output devices, such as a monitor for displaying video data, a speaker for outputting audio data, etc. I / O devices 880 may also include input devices such as a keyboard, mouse, trackball, etc., and / or corresponding interfaces for interacting with the aforementioned output devices.

[0171] Memory 860 includes one or more disks, one or more tape drives, and one or more solid-state drives, and can be used as an overflow data storage device to store such programs when a program is selected for execution, and to store instructions and data read during program execution. Memory 860 can be volatile and / or non-volatile, and can be read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and / or static random access memory (SRAM).

[0172] Figure 9This is a schematic diagram of one embodiment of the decoding module 900. In one embodiment, the decoding module 900 is implemented in a video decoding device 902 (e.g., a video encoder 300 or a video decoder 400). The video decoding device 902 includes a receiving module 901. The receiving module 901 is used to receive images for encoding or to receive bitstreams for decoding. The video decoding device 902 includes a transmitting module 907 coupled to the receiving module 901. The transmitting module 907 is used to transmit the bitstream to the decoder or to transmit the decoded image to a display module (e.g., one of a plurality of I / O devices 880).

[0173] The video decoding device 902 includes a storage module 903. The storage module 903 is coupled to at least one of a receiving module 901 or a transmitting module 907. The storage module 903 is used to store instructions. The video decoding device 902 also includes a processing module 905. The processing module 905 is coupled to the storage module 903. The processing module 905 is used to execute the instructions stored in the storage module 903 to perform the methods disclosed herein.

[0174] While this invention provides several embodiments, it is understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the invention. The examples of this invention are to be considered illustrative rather than restrictive, and the invention is not limited to the details set forth herein. For example, various elements or components may be combined or integrated into another system, or some features may be omitted or not implemented.

[0175] Furthermore, without departing from the scope of this invention, the techniques, systems, subsystems, and methods described and illustrated as discrete or separate in the various embodiments can be combined or integrated with other systems, components, techniques, or methods. Other variations, substitutions, and modifications can be determined by those skilled in the art and can be made without departing from the spirit and scope of this disclosure.

Claims

1. A method implemented by a decoder, characterized in that, The method includes: The decoder receives a bitstream, wherein the bitstream includes a picture parameter set (PPS), and the PPS includes a single tag for each sub-image; The decoder determines whether the value of the single stripe flag for each sub-image is a first value or a second value, wherein the first value indicates that each sub-image of the PPS includes one and only one rectangular stripe, and the second value indicates that each sub-image of the PPS may include one or more rectangular stripes; When the value of the single stripe flag for each sub-image is the first value, the decoder decodes the one and only one rectangular stripe to obtain a decoded image without parsing the syntax elements indicating the position and size of each sub-image; when the value of the single stripe flag for each sub-image is the second value, the decoder decodes the one or more rectangular stripes to obtain a decoded image.

2. The method according to claim 1, characterized in that, The single-slice flag for each sub-image is represented as pps_single_slice_per_subpic_flag.

3. The method according to claim 1 or 2, characterized in that, The first value is 1, and the second value is 0.

4. The method according to claim 1 or 2, characterized in that, The PPS also includes a bar count flag in the image, and the value of the bar count flag plus 1 indicates the number of bars in each image referencing the PPS.

5. The method according to claim 4, characterized in that, The number of stripes in the image is represented as pps_num_slices_in_pic_minus1.

6. The method according to claim 1, 2, or 5, characterized in that, Each sub-image comprises a rectangular region of one or more stripes within the image.

7. The method according to claim 1, 2, or 5, characterized in that, The method further includes: when the value of the single strip marker of each sub-image is the first value, generating an image based on the one and only one rectangular strip for display; and when the value of the single strip marker of each sub-image is the second value, generating an image based on the one or more rectangular strips for display.

8. A method implemented by an encoder, characterized in that, The method includes: When each sub-image of the reference image parameter set (PPS) includes one and only one rectangular stripe, the encoder sets the single stripe flag for each sub-image to a first value; when each sub-image of the reference PPS may include one or more rectangular stripes, the encoder sets the single stripe flag for each sub-image to a second value. The encoder encodes the PPS into the bitstream, wherein the PPS includes a single tag for each sub-image, without needing to encode the syntax elements indicating the position and size of each sub-image into the bitstream; The encoder stores the bitstream for transmission to the decoder.

9. The method according to claim 8, characterized in that, The single-slice flag for each sub-image is represented as pps_single_slice_per_subpic_flag.

10. The method according to claim 8 or 9, characterized in that, The first value is 1, and the second value is 0.

11. The method according to claim 8 or 9, characterized in that, The PPS also includes a bar count flag in the image, and the value of the bar count flag plus 1 indicates the number of bars in each image referencing the PPS.

12. The method according to claim 11, characterized in that, The number of stripes in the image is represented as pps_num_slices_in_pic_minus1.

13. The method according to claim 8, 9, or 12, characterized in that, Each sub-image comprises a rectangular region of one or more stripes within the image.

14. The method according to claim 8, 9, or 12, characterized in that, The method further includes sending the bitstream to a video decoder.

15. A decoding device, characterized in that, The decoding device includes: A receiver is used to receive a bitstream, wherein the bitstream includes a picture parameter set (PPS), and the PPS includes a single band marker for each sub-image; A memory coupled to the receiver, wherein the memory stores instructions; A processor coupled to the memory, wherein the processor is configured to execute the instructions to cause the decoding device to perform the following operations: Determine whether the value of the single stripe flag for each sub-image is a first value or a second value, wherein the first value indicates that each sub-image of the PPS includes one and only one rectangular stripe, and the second value indicates that each sub-image of the PPS may include one or more rectangular stripes; When the value of the single stripe flag for each sub-image is the first value, the one and only one rectangular stripe is decoded to obtain a decoded image, without parsing the syntax elements indicating the position and size of each sub-image; when the value of the single stripe flag for each sub-image is the second value, the one or more rectangular stripes are decoded to obtain a decoded image.

16. The decoding device according to claim 15, characterized in that, The first value is 1, and the second value is 0.

17. The decoding device according to claim 15 or 16, characterized in that, The PPS also includes a bar count flag in the image, and the value of the bar count flag plus 1 indicates the number of bars in each image referencing the PPS.

18. The decoding device according to claim 17, characterized in that, The number of stripes in the image is represented as pps_num_slices_in_pic_minus1.

19. The decoding device according to claim 15, 16, or 18, characterized in that, The processor is further configured to: generate an image for display based on the one and only one rectangular strip when the value of the single strip marker of each sub-image is the first value; and generate an image for display based on the one or more rectangular strips when the value of the single strip marker of each sub-image is the second value.

20. An encoding device, characterized in that, The encoding device includes: Memory including instructions; A processor coupled to the memory, wherein the processor is configured to execute the instructions to cause the encoding device to perform the following operations: The bitstream is encoded, wherein the bitstream includes a picture parameter set (PPS), and the PPS includes a single tag for each sub-image, without encoding the syntax elements indicating the position and size of each sub-image into the bitstream; When each sub-image of the PPS includes one and only one rectangular stripe, the single stripe flag of each sub-image is set to a first value; when each sub-image of the PPS may include one or more rectangular stripes, the single stripe flag of each sub-image is set to a second value. A transmitter coupled to the processor, wherein the transmitter is used to send the bitstream to the video decoder.

21. The encoding device according to claim 20, characterized in that, The first value is 1, and the second value is 0.

22. The encoding device according to claim 20 or 21, characterized in that, The PPS also includes a bar count flag in the image, and the value of the bar count flag plus 1 indicates the number of bars in each image referencing the PPS.

23. The encoding device according to claim 22, characterized in that, The number of stripes in the image is represented as pps_num_slices_in_pic_minus1.

24. The encoding device according to claim 20, 21, or 23, characterized in that, The memory stores the bitstream before sending it to the decoder.

25. A decoding device, characterized in that, The decoding device includes: A receiver is used to receive images for encoding or to receive bitstreams for decoding. A transmitter coupled to the receiver, wherein the transmitter is used to send the bitstream to the decoder or to send the decoded image to the display; A memory coupled to at least one of the receiver or the transmitter, wherein the memory is used to store instructions; A processor coupled to the memory, wherein the processor is configured to execute the instructions stored in the memory to perform the method according to any one of claims 1 to 7 and any one of claims 8 to 14.

26. The decoding apparatus according to claim 25, characterized in that, The decoding device also includes a display for showing the decoded image.

27. A system, characterized in that, The system includes: Encoder; A decoder communicating with the encoder, wherein the encoder or the decoder includes a decoding device, encoding device or decoding apparatus according to any one of claims 15 to 26.

28. A decoding module, characterized in that, The decoding module includes: The receiving module is used to receive images for encoding or receive bitstreams for decoding; A transmitting module coupled to the receiving module, wherein the transmitting module is used to transmit the bitstream to the decoding module or to the decoding image to the display module; A storage module coupled to at least one of the receiving module or the transmitting module, wherein the storage module is used to store instructions; A processing module coupled to the storage module, wherein the processing module is configured to execute the instructions stored in the storage module to perform the method according to any one of claims 1 to 7 and any one of claims 8 to 14.

Images