Chroma quantization in video coding
Patent Information
- Application Number
- TW113142733
- Authority / Receiving Office
- TW · TW
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2014-08-05
- Filing Date
- 2014-08-25
- Publication Date
- 2026-10-11
- Estimated Expiration
- 2034-08-24
Smart Images

Figure TWG2TB001915234_001 
Figure TWG2TB001915234_002 
Figure TWG2TB001915234_003
Abstract
Description
Technical Field
[0001] This application relates to color quantization, and more particularly to color quantization in video coding. Prior Art
[0002] The next-generation High Efficiency Video Coding (HEVC / H.265) standard, jointly developed by the ITU and ISO MPEG, introduces several new video coding tools aimed at improving video coding efficiency compared to previous video coding standards and technologies, particularly MPEG-2, MPEG-4 Part 2, MPEG-4 AVC / H.264, VC1, and VP8. In its first release, the new standard supports encoding of 8- or 10-bit material in YUV 4:2:0 using three well-defined profiles: Main, Main 10, and Main Still Image. However, work is ongoing to support sample precision (bit depth) higher than 10 bits, as well as different color sampling formats and color spaces, including, in particular, YUV 4:2:2, YUV 4:4:4, and RGB 4:4:4. The encoding of such material is of considerable interest, primarily for professional applications such as cinematic applications, capture, video editing, archiving, medical imaging, etc., but also for consumer applications such as, inter alia, screen content compression and sharing, remote computing, and gaming. Until recently, existing video codecs have almost always given higher priority to the luma component in coding an image. Coding parameters that control the coding characteristics and quality of luma information are primarily provided at lower levels of the video coding hierarchy (such as the tile, coding tree block, coding unit, or even the transform block level in HEVC). Control and tuning parameters for all other components may be primarily provided at higher levels (such as in conjunction with modifications of luma control parameters at the sequence, image, or video level). At the image parameter set or tile level). For example, in MPEG-4 AVC, luma quantization and the corresponding quantization parameter (QP) are controlled by a signaled parameter at the macroblock level. A single offset for each chroma component is provided in the picture parameter set. Each chroma QP offset controls the QP value of the corresponding chroma component relative to the luma QP within each macroblock. However, this relationship is fixed for the entire image. If the quality of either the luma or chroma regions needs to be changed, given this very tight relationship, the other components are also affected. HEVC also provides some additional control by allowing the signaling of separate chroma quantization offsets for each tile. However, while the control provided is still somewhat imprecise, using multiple tiles may not be desirable or functional for some applications. For some applications or content, being able to independently control the quality of some color components is crucial in trying to improve overall quality, compression ratios, and the overall user experience. For example, even within color components, some areas may be characterized by different texture or noise characteristics, and enhancing color edges may be more or less important than enhancing the same information in luminance. Furthermore, for 4:4:4 applications such as shared video displays and remote computing, it may be necessary to encode RGB content, where the red and blue components tend to be more important (and therefore require control) than the chrominance components in the YUV domain. It may also be necessary to encode mixed video content, which is a combination of synthetic content and natural images or video, such as computer graphics or applications. In this scenario, given the different characteristics of natural versus synthetic content and the possibility that the natural content was initially upconverted from 4:2:0 images for display to 4:4:4, the ability to control chrominance quantization parameters can potentially significantly impact encoding performance and subjective quality. What is needed is a method that allows further control of chroma quantization parameters than existing codecs and extends this support for all common color formats (such as YUV, RGV, YCoCg, or YCoCg-R) per component, all common color sampling schemes (such as 4:2:0, 4:2:2, 4:4:4, or 4:4:4:4), and multiple bit depths. This method should allow for a more flexible way to signal and change chroma QP offset information within a coded block without any restrictions on color or sampling formats. Summary of the Invention
[0003] To provide a video coding system that allows for more flexible specification of chroma quantization parameters (QPs), some embodiments of the present invention provide a method for signaling additional chroma QP offset values specific to a quantization group. In some embodiments, each quantization group explicitly specifies its own set of chroma QP offset values. In some embodiments, a table of possible chroma QP offset sets is specified in the header region of a picture or tile, and each quantization group uses an index to select an entry from the table to determine its own set of chroma QP offset values. The chroma QP offset values at the quantization group level are then used to determine the chroma QP values for the blocks or pixel sets within the quantization group. In some embodiments, these quantization group chroma QP values are used in conjunction with the luma QP of the block and chroma QP offset values already specified at higher levels of the video coding hierarchy. Some embodiments associate chroma QP offset specifications with a quantization group (QG) that includes one or more units of video data. In some embodiments, each QG is associated with its own set of chroma QP offset specifications, and in some of these embodiments, the set of chroma QP offset specifications associated with a QG is encoded or embedded within the coding structure of a video data unit in the QG. To further reduce overhead or bit usage, some embodiments specify all available offset values for each component, or a combination of two components, at a higher level (e.g., in a sequence parameter set (SPS), picture parameter set (PPS), or in the current tile header). In some of these embodiments, the higher-level syntax header (SPS / PPS / tile header) lists the different possible offset values in a table, with each entry in the table assigned an index. Then, at the coding unit / quantization group level, some embodiments specify only the index or indices of the desired quantization offset value. These offsets can be independent of, or in addition to, the offset values specified in the PPS or tile header. To reduce bitstream size, some embodiments limit the number of entries in the table to a maximum value. Encoders in different embodiments use different methods for selecting and assigning additional chroma QP offset values specific to a quantization group. Some embodiments perform a pre-analysis step in which the encoder performs a region-level analysis. Some embodiments identify different regions within an image with different types of video content. In some of these embodiments, different regions with different types of video content are assigned different chroma QP offset values or are assigned to different quantization groups. Some embodiments distinguish between graphical content and actual video content. Some embodiments distinguish between 4:4:4 video content originally coded in 4:4:4 format and 4:4:4 video content upsampled from 4:2:0 format. Some embodiments identify video content that may have originally had different bit depths. In addition to the relationship between color components and rate control information, some embodiments use these characteristics of the video content to determine the quantization levels or quantization relationships between all color components. The preceding "Summary of the Invention" is intended to serve as a brief introduction to some embodiments of the present invention. It is not intended to be an introduction or overview of all of the subject matter disclosed in this document. The following "Detailed Description" and the figures referenced in "Detailed Description" further illustrate the embodiments described in "Summary of the Invention" and other embodiments. Therefore, a complete review of "Summary of the Invention," "Detailed Description of the Invention," and "Brief Description of the Figures" is required to understand all of the embodiments described in this document. Simple diagram description
[0004] The novel features of the present invention are set forth in the appended claims. However, for illustrative purposes, several embodiments of the present invention are set forth in the following drawings. [picture] [1] illustrates a hierarchical video coding structure 100 comprising chroma quantization groups, each with its own set of chroma QP offset specifications. [picture] [2a] [To the image] [2c] describes several methods for encoding a set of chroma QP offset values for a chroma quantization group. [picture] [3] Describes an example picture header or picture parameter set (PPS) that specifies the definition of a chrominance quantization group. [picture] [4] illustrates an example coding tree unit that can be in a chroma quantization group. [picture] [5] Describes the provision of additional chroma QP offsets within a quantization group. [picture] [6] illustrate the provision of additional chroma QP offsets within a quantization group using a different approach for specifying chroma QP offset values for two chroma components. [picture] [7] Conceptually describes a process for determining chroma QP values from a video bitstream that allows for additional chroma QP offset provisions. [picture] [8] Describes a hierarchical video coding structure including a table of possible chroma QP offset values at a higher level header. [picture] [9] An example picture header or picture parameter set (PPS) illustrating a table encoding chroma QP offsets. [picture]
[10] A picture header describing each entry of a predictively coded chroma QP offset table. [picture]
[11] describes an example coding tree unit that can be in a quantization group that uses an index to retrieve chroma QP offset values from a table in the picture header. [picture]
[12] describes the use of an index to select a quantization group of entries in a table of chroma QP offset values. [picture]
[13] Conceptually describes a process for determining chroma QP values from a video bitstream using a table of possible chroma QP offset values to implement additional chroma QP offset provisions. [picture]
[14] Pseudocode for a picture header is described including a selection of methods for specifying additional chroma QP offset values. [picture]
[15] describes an implementation of a pseudocode for a transform unit that can specify chroma QP offset values for a quantization group using any of three different methods. [picture]
[16] Describe instance segmentation and classification of images. [picture]
[17] Conceptually illustrates a process 1700 for analyzing different regions of an image and assigning chroma QP offsets accordingly. [picture]
[18] Describe a general video encoder. [picture]
[19] Describe a general video decoder. [picture]
[20] Conceptually illustrates an electronic system used to implement some embodiments of the present invention. Implementation Method
[0005] [Claim of Rights in Previous Applications] [] This application claims the benefit of U.S. Provisional Patent Application No. 61 / 875,664, filed September 9, 2013. U.S. Provisional Patent Application No. 61 / 875,664 is incorporated herein by reference. In the following description, numerous details are set forth for purposes of explanation. However, one skilled in the art will appreciate that the present invention can be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form to avoid obscuring the description of the present invention with unnecessary detail. To provide a video coding system that allows for more flexible specification of chroma quantization parameters (chroma QP), some embodiments of the present invention provide a method for signaling additional chroma QP offset values specific to a quantization group. In some embodiments, each quantization group explicitly specifies its own set of chroma QP offset values. In some embodiments, a table of possible chroma QP offset sets is specified in the header region of a picture or tile, and each quantization group uses an index to select an entry from the table to determine its own set of chroma QP offset values. The chroma QP offset values at the quantization group level are then used to determine the chroma QP values for blocks within the quantization group. In some embodiments, these quantization group chroma QP values are used in conjunction with the luma QP of the block and chroma QP offset values specified at higher levels of the video coding hierarchy. Section I below describes methods for specifying additional chroma QP offset values. Specifically, Section Ia describes methods for explicitly specifying additional chroma QP offset values in a quantization group, while Section Ib describes methods for specifying a table of possible chroma QP offset values. Next, Section II describes various systems and methods for identifying additional chroma QP offset values and assigning them to different regions of an image. Section III describes video encoder and decoder systems that implement some embodiments of the present invention, while Section IV describes computer systems used to implement some embodiments of the present invention. [I.] [Specify additional chromaticity] [QP] [Offset] [] [a.] [Specify additional chroma in quantization group] [QP] [Offset] Some embodiments of the present invention provide a method for specifying chroma quantization parameter offset values (chroma QP offsets) for coding a video sequence. The method associates chroma QP offset specifications with chroma quantization groups, where each chroma QG encompasses one or more video data units (such as coding units in HEVC). In some embodiments, the set of chroma QP offset specifications associated with a chroma quantization group is encoded or embedded within the coding structure of the video data units in the chroma quantization group. In some embodiments, the chroma QP offset specifications for the chroma quantization group are applied in addition to other chroma QP offset specifications specified for the video coding structure encompassing the chroma quantization group at a higher level of the video coding hierarchy, such as a tile or picture. In some embodiments, chroma QP offsets from different levels are used together to determine the chroma quantization parameter (chroma QP) (e.g., by adding the chroma QP offsets to the luma QP value). In some embodiments, in addition to the luma quantization group (luma QG), a chroma quantization group (chroma QG) is also defined. The luma QG also covers one or more units of video data, but the luma QG is used to specify the luma quantization parameter (luma QP). In some embodiments, the chroma QG can overlap or cover one or more luma QGs. In other words, different regions within the chroma QG can belong to different luma QGs and, therefore, have different luma QPs. For some embodiments of the present invention, [picture] [1] A hierarchical video coding structure 100 is illustrated, including chroma quantization groups (chroma QGs), each with its own set of chroma QP offset specifications. Hierarchical video coding structure 100 includes a number of pictures 101 to 103. Among the pictures, picture 102 includes a number of tiles 111 to 113. Tile 112 includes a number of chroma QGs 121 to 123. Picture 102 has a picture parameter set (PPS) 132 including a set of chroma QP offset specifications. Chroma QGs 121 to 123 are associated with chroma QP offset sets 141 to 143, respectively. Hierarchical video coding structure 100 corresponds to a video image sequence. A video image sequence is organized into layers of video data at different levels of the hierarchy, where a video data unit at a higher level encompasses one or more video data units at a lower level. For example, a group of pictures (GOP) is a video data unit at a level higher than a picture in the hierarchy, a picture is a video data unit at a level higher than a tile of the picture in the hierarchy, a tile is a video data unit at a level higher than a coding tree unit in the hierarchy, and so on. In some embodiments, at least some parameters specified for a higher-level video data unit may apply to lower-level video data units encompassed by the higher-level video data unit. For example, in some embodiments, the picture-level chroma QP offset specification (from PPS 132) for picture 102 may apply to all chroma QGs encompassed by picture 102 (e.g., chroma QGs 121-123). In some embodiments, the video coding structure 100 is encoded as a bitstream. This bitstream is based on an established video coding format, such as the HEVC / H.265 standard or the MPEG-4 AVC / H.264 standard. An image under the H.265 standard may include one or more tiles, and each tile may have one or more coding tree units (CTUs). Furthermore, each CTU may be subdivided into a number of blocks. H.265 also allows for the definition of luma quantization groups for quantizing / dequantizing luma components. Each quantization group allows for the derivation of luma quantization parameters specific to the video data units within the quantization group. Once derived, the luma quantization parameters are then used to perform quantization of the luma DCT coefficients. Under the H.265 standard, chroma quantization parameters are derived from the luma quantization parameters based on offset values provided in the image or tile layer of the H.265 compliant bitstream. The video coding architecture 100 provides for signaling or provisioning of chroma QP offsets in addition to the picture / tile level chroma QP offsets allowed by the H.265 standard. Chroma QGs 141-143 are defined ranges within video coding structure 100. Each of QGs 141-143 is associated with its own set of chroma QP offset values 141-143. In some embodiments, a quantization group can be a group of video data units that share the same set of QP values. In some embodiments, the signaling of a chroma QP offset within a QG applies to all video data units within that QG. In some embodiments, QP offsets are applied starting from the first signaled chroma residual information. Any region prior to this signaling is not associated with a signaled chroma QP offset. In other words, a QG can be "segmented." In some embodiments, the QG (chroma or luma) can be defined at different levels or depths of the video coding hierarchy (such as coding tree units, coding units, or transform units), as these units are supported in HEVC. In some embodiments, the definition of the QG for the chroma QP offset inherits the definition of the QG for the luma component when applicable. In some embodiments, the depth, level, and size of the QG (chroma or luma) can be flexibly specified in the bitstream and can vary between pictures. For example, one picture (e.g., 101) can have its chroma QG specified at the top level of the coding quadtree (i.e., the QG is a CTU), while another picture (e.g., 103) can have its chroma QG specified at a lower level of the coding quadtree (e.g., the QG is a coding block of the quadtree). In some embodiments, higher-level parameters (such as the PPS or tile header) specify at which level of the video coding hierarchy the QG of the picture being defined resides. [picture] [1] also shows how the offset specified by the chroma QG is used to calculate the chroma QP value. [picture] [1] In some of those described embodiments, a first set of chroma QP offset parameters is signaled at the PPS 132 (and / or the header of the tile 112). In some embodiments, when a QG specifies its own chroma QP offset value, chroma QP offset values specified at levels above the chroma QG (e.g., the PPS and / or tile header) are ignored. In some embodiments, higher-level chroma QP offset syntax elements and QG-level chroma QP offsets are considered together to reduce overhead. That is, the adaptation of the chroma QP value is performed together with respect to both parameter sets, and the signaled information can be considered a "second-order" offset. In some of these embodiments, the chroma QP value for the current chroma QG is calculated as: QPchroma[i] = QPluma + QPoffset_pps[i] + QPoffset_quant_group[i] (1) (For the first chroma component, i = 0; for the second chroma component, i = 1) where QPluma is the QP for the luma component corresponding to the current chroma quantization group, QPoffset_pps[i] is the QP offset for the i-th chroma component from the current PPS (or tile header), and QPoffset_quant_group[i] is an additional offset signaled at the QG level for that component. QPoffset_pps[i] + QPoffset_quant_group[i] thus constitutes the total chroma QP offset for the i-th chroma component. In some embodiments, the chroma QP value is adapted based on chroma QP offset parameters from multiple levels of the video coding hierarchy. For example, in some embodiments, the calculation of the chroma QP value takes into account the chroma QP offset values from the PPS and from the tile header. In some of these embodiments, the chroma QP value for chroma component i of the current QG is calculated as: QPchroma[i] = QPluma + QPoffset_pps[i] + QPoffset_slice[i] + QPoffset_quant_group[i] (2) Where QPoffset_slice[i] is the QP offset for the i-th chroma component from the current tile header, QPoffset_pps[i] is the QP offset for the i-th chroma component from the current PPS, and the sum QPoffset_pps[i] + QPoffset_slice[i] + QPoffset_quant_group[i] is the total chroma QP offset for the i-th chroma component. Some embodiments optionally allow for one or more additional chroma QP offset values to be specified at other levels of the video coding hierarchy. To encode or decode a particular coded block, some of these embodiments use some or all of the chroma offset QP values specified along the levels of the video coding hierarchy that encompass (or are applicable to) that particular coded block. As mentioned, the chroma QP value for a chroma QG is calculated by adding the chroma QP offset to the luma QP value applicable to the chroma QG. In some embodiments, the luma QP value corresponding to the chroma QG can vary within the chroma QG. This is because the luma QG and chroma QG can overlap, allowing the chroma QG to encompass writing units that fall within different luma QGs and therefore have different luma QP values. Consequently, different writing units within a chroma QG can end up with different chroma QP values (due to the chroma QP offset applicable to the chroma QG being applied to different luma QP values within the chroma QG). exist [picture] In the example of [1], chroma QG 121 has four corresponding luma QP values (QPLuma) because chroma QG 121 overlaps with four different luma QGs 171-174. These four luma QP values for chroma QG 121 produce four possible chroma QP values after applying the chroma QP offsets (QPoffset_pps, QPoffset_slice, QPoffset_quant_group) from PPS offset 132 and QG1 offset 141. A coding unit in chroma QG 121 can therefore have one of these four possible chroma QP values depending on the underlying luma QG in which the coding unit falls. Similarly, chroma QG 123 overlaps with two different luma QGs 181-182, and a coding unit in chroma QG 123 can have one of two possible chroma QP values depending on the underlying luma QG in which the coding unit falls. On the other hand, chroma QG 122 has only one corresponding luma QP value because chroma QG 122 falls entirely within luma QG 175 (or covers the exact same set of CUs as luma QG 175). Therefore, the CUs in chroma QG 122 all have the same chroma QP value after applying the chroma QP offset. Since for most video coding formats (e.g., YCbCr), there are two chroma components for each luma component, in some embodiments, each chroma QP offset of a quantization group is a specified value set including two offset values for calculating the two chroma components. [picture] [2a] [To the image] [2c] Description for encoding [picture] [1] Several methods for encoding offset value sets for QG 141. Some embodiments use only one of these methods for encoding chroma QP offset value sets. Some embodiments select different methods for different QGs based on QG-specific characteristics. [picture] [2a] illustrates the set of chroma QP offset values as two independent offset values 241 and 242. These two values are directly used as chroma QP offsets for component [0] (251) and component [1] (252). Some embodiments choose this method of encoding chroma QP offsets when the two chroma components have little correlation with each other. See below [picture] [5] further describes another example of this method of encoding chroma QP offset values. [picture] [2b] illustrates a set of offset values that are encoded as a single value 243 used by both chroma components (253 and 254) as a chroma QP offset. Some embodiments choose this approach when the two chroma components are very similar in their relationship to the luma component. See below [picture] [6] further describes another example of this method of encoding chroma QP offset values. [picture] [2c] illustrates a set of offset values encoded as an immediate offset value 244 and a delta value 245 based on the immediate offset. In this example, the immediate offset value 244 is applied directly as a chroma QP offset (255) for a first chroma component, and the sum of the delta value 245 and the immediate value 244 is used as a chroma QP offset value (256) for a second chroma component. Some embodiments select this approach when the two chroma components differ in their relationship to the luma component by a small offset that can be cheaply encoded in the bitstream using very few bits. This approach encodes the second chroma QP offset (for component [1]) with respect to the first chroma QP offset. That is, the second chroma QP offset (for component [0]) is now predicted from the first chroma QP offset. In some embodiments, the chroma QP value for the first chroma component is calculated as: QPchroma[0] = QPluma + QPoffset_pps[0] + QPoffset_quant_group[0]. (3) The chroma QP value of the second chroma component is calculated as: QPchroma[1] = QPluma + QPoffset_pps[1] + QPoffset_quant_group[0] + QPoffset_quant_group[1]. (4) (For the first chroma component, i = 0; for the second chroma component, i = 1) As mentioned, some embodiments use chroma QP offset values from multiple different layers of the video coding hierarchy for deriving the final chroma QP value. For some embodiments that use chroma QP offset values from both the PPS and the tile header, the chroma QP value for the first chroma component is calculated as: QPchroma[0] = QPluma + QPoffset_pps[0] + QPoffset_slice[0] + QPoffset_quant_group[0]. (5) The chroma QP value of the second chroma component is calculated as: QPchroma[1] = QPluma + QPoffset_pps[1] + QPoffset_slice[1] + QPoffset_quant_group[0] + QPoffset_quant_group[1]. (6) As mentioned, in some embodiments, the definition of luma and / or chroma quantization groups is specified by parameters in the image level. [picture] [3] An example picture header or PPS 300 that specifies the definition of a chroma quantization group is described. The example picture header 300 is associated with the picture 102 of the video coding structure 100. The picture header 300 is described by a pseudocode modified from the H.265 standard to accommodate additional chroma QP offsets (i.e., adaptation of chroma QP values). In some embodiments, the pseudocode of a video standard (such as the H.265 standard or a modified standard) describes the operations required of a video decoder when processing a bitstream compliant with the video standard. The pseudocode also describes the operations required of a video encoder when generating a bitstream compliant with the video standard. For illustrative purposes, the pseudo code of the image header 300 is shown in FIG. [picture] [3]. Lines 311-313 allow for additional chroma QP offset values and define chroma quantization groups for the chroma QP offsets in picture 102. Specifically, the parameter "additional_chroma_qp_offset" in line 311 informs the decoder that additional chroma QP offsets will be specified in the bitstream. When this variable is not set for a particular picture, the bitstream will not include the bits used to specify the additional chroma QP offset for the picture to avoid wasting bits. The parameter "chroma_qp_offset_max_depth" in line 313 defines the level (and therefore the size or hierarchical depth) of the chroma QG. This parameter can be set equal to the size of the entire coding tree unit (CTU) or down to the size of the smallest coding unit possible in the bitstream. Some embodiments allow the specification and signaling of additional chroma QP offsets to occur at any desired level (e.g., at the CTU level, at the chroma-defined quantization group level, at the coding unit level, at the transform unit level, etc.). In some embodiments, the bitstream uses the parameter "diff_cu_qp_delta_depth" which also defines the quantization group for luma QP, rather than specifically defining quantization groups for chroma QP offsets. In some embodiments, if the parameters for defining a quantization group are not available, the entire image is assigned the same QP for luma, and luma QP adaptation is not allowed. [picture] [4] illustrates an example coding tree unit 400 that may be within a quantization group. The coding tree unit 400 is described using a pseudocode modified from the H.265 pseudocode to accommodate additional chroma QP offsets. The coding tree unit 400 is within a quantization group 122, which is within a tile 112 of a picture 102. The pseudocode for coding tree unit 400 is illustrated with newly added, shaded lines 411 through 413 for determining whether the coding tree unit is part of a QG. Specifically, at line 411, the parameter "additional_chroma_qp_offset" (e.g., from picture header 300) is used to indicate whether additional chroma information is allowed. The parameter "chroma_qp_offset_max_depth" is compared to the variable "log2CbSize," which is used to determine whether coding tree unit 400 is in a quantization group (such as quantization group 122). If so, the pseudocode sets the variable "IsCrCuQpOffsetCoded" to 0 at line 412. For some embodiments, this also initializes the start of the chroma quantization group. [picture] [5] The provision of additional chroma QP offsets within QG 122 is described above by referring to [picture] [3] and [picture] As discussed in [4], the QG 122 is defined by the picture header 300 to include a coding tree unit 400. The additional chroma QP offset is specified in a transform unit 500 of the coding tree unit 400. The transform unit 500 is described by a pseudocode modified from the H.265 pseudocode to accommodate the additional chroma QP offset. The pseudocode for transform unit 500 is illustrated using newly added dimming lines 511-519 for the purpose of specifying additional chroma QP offsets. Specifically, at line 511, the parameter "additional_chroma_qp_offset" indicates whether additional chroma information is allowed. The variable "IsCrCuQpOffsetCoded" indicates whether chroma QP offset information needs to be specified (or received) at transform unit 500. If the chroma QP offset value for QG 122 is already coded, then there is no need to specify a chroma QP offset value. If the QG 122 has not yet coded a set of chroma QP offset values (i.e., if the variable "IsCrCuQpOffsetCoded" is 0), the transform unit 500 defines the chroma QP offset values for the two chroma components at lines 512-517. In this example, the transform unit defines the set of chroma QP offsets as two separate signed values, where each signed value is signaled using a magnitude ("cu_chroma_cmp0_qp_offset_abs" or "cu_chroma_cmp1_qp_offset_abs") and a sign flag ("cu_chroma_cmp0_qp_delta_sign_flag" and "cu_chroma_cmp0_qp_delta_sign_flag"). The pseudocode then sets "IsCrCuQpOffsetCoded" to 1, indicating that chroma QP offset values for this quantization group have been defined. As previously seen [picture] [2a] to [picture] As mentioned in [2b], the quantization groups in different embodiments specify the chroma QP offset values of the two chroma components in different ways. [picture] [6] The specification of additional chroma QP offsets within QG 122 is illustrated using a different approach for specifying chroma QP offset values for two chroma components. [picture] [6] A transform unit 600 is described by pseudocode similar to the pseudocode for transform unit 500, except that lines 611-616 specify only one chroma QP offset value (magnitude + sign) for both chroma components. In some embodiments, since the importance of the two color components may not change in most cases, it is sufficient to specify one chroma QP offset value for both chroma components. Signaling of one or both of these parameters may be used as a header, or may be signaled at a different higher syntax level location such as an SPS, PPS, or tile header. For some embodiments, [picture] [7] A process 700 is conceptually illustrated for determining chroma QP values from a video bitstream that allows for additional chroma QP offset provisions. In some embodiments, the process 700 is performed by a video decoder that uses the process to perform dequantization of chroma components when decoding a video bitstream for display. In some embodiments, the process begins when the decoder has received a video bitstream and begins decoding a particular video picture for display or other purposes. At 705, the processing procedure 700 processes the parameters in the picture header or PPS of a particular video picture. [picture] [3] As discussed, in some embodiments, the picture header or PPS of a video picture includes a flag (e.g., "additional_chroma_qp_offset") for determining whether additional chroma QP offsets are allowed for the video picture. The picture header or PPS also identifies the layer of the video hierarchy in which the chroma quantization group for the additional chroma QP offset is to be defined (e.g., by setting the variable "chroma_qp_offset_max_depth"). Next, the process identifies (at 710) higher-level chroma QP offset values. In some embodiments, these higher-level chroma QP offset values are picture-level offsets encoded in a picture header (or as part of a PPS). In some embodiments, these higher-level chroma QP offset values are tile-level offsets coded in a tile header. Some embodiments identify multiple higher-level chroma QP offset values, including both picture-level and tile-level values. The process then determines (at 720) whether additional chroma QP offsets are allowed or available for this particular video picture and / or tile. If additional chroma QP offsets are not available, the process proceeds to 750. If additional chroma QP offsets are available, the process proceeds to 730. At 730, the process determines whether it has reached the beginning of a chroma quantization group. In some embodiments, the process checks whether it is at a level of the video hierarchy that has been identified as a chroma quantization group. If the process is not at the beginning of a chroma quantization group (e.g., is already inside a chroma quantization group), the process proceeds to 750. If the process is at the beginning of a chroma quantization group, the process proceeds to 740. See above. [picture] [4] Discuss examples of operations 720 and 730. At 740, the process identifies the chroma QP offset value for the chroma quantization group. In some embodiments, as described above with reference to [picture] [5] and [picture] As discussed in [6], the chroma QP offset is explicitly encoded within the chroma quantization group. The process then identifies (at 750) the luma QP value for the chroma quantization group. As mentioned, in some embodiments, the luma QP value is set based on the luma quantization group, which may or may not be the same as the chroma quantization group. Depending on signaling, there may be more or fewer luma quantization groups than chroma quantization groups. In some embodiments, a luma quantization group may contain multiple chroma quantization groups, or vice versa. In some embodiments, the chroma quantization group and the luma quantization group may overlap. Therefore, different coding units in the same chroma quantization group may have different luma QP values based on the luma quantization group. Next, the process calculates (at 760) the chroma QP value. For the decoder, some embodiments calculate the chroma QP value from the identified luma QP value and from each identified chroma QP offset. In some embodiments, this calculation is achieved by adding the luma QP value to all identified chroma QP offsets (from the tile / picture level and from the QG level), as described above in equations (1) to (6). The process then determines (at 770) whether the process has reached the end of the picture (if the higher-level chroma QP offset is for the entire picture) or the end of the tile (if the higher-level chroma QP offset is for the tile). If so, the process 700 ends. If not, the process 700 returns to 730 to process the next quantization group. See above [picture] [1] to [picture] As shown in [6], chroma QP values are predictively coded using offsets from luma QP values. Furthermore, in some embodiments, the offset values themselves are predictively coded by offsetting each other, such as QG level offsets being offsets from tile and / or picture level offsets, and / or the offset value of one chroma component being coded as an offset from another chroma component. In some embodiments, the chroma QP offset is further predicted from the chroma values or offsets of adjacent coding units or blocks, or from the chroma values or offsets of co-located coding units or blocks in adjacent video images, because these chroma QP offsets are most likely sufficiently similar to the coding units or blocks of the current chroma quantization group. Some embodiments do not perform prediction for at least some of the quantization groups. In these cases, the chroma QP values are explicitly coded (not as offsets), so that the QP parameters of the luma component are ignored (because no offset from luma is needed). Some embodiments use a flag to indicate that the chroma QP offset value set from a neighboring quantization group is to be used for the current quantization group. In some embodiments, the neighboring quantization group is spatially located to the left or on top of the current quantization group. For example, some embodiments use a flag "cu_qp_update_signal" to indicate whether the chroma QP offset value from the neighboring quantization group is to be used. In some embodiments, [picture] The "cu_chroma_qp_offset_abs" parameter at line 612 of [6] is previously signaled with the flag "cu_qp_update_signal". If this flag is "1", the current quantization group will provide an update for specifying its own chroma QP offset value. If this flag is "0", the current quantization group will not update its set of chroma QP offset values, but will instead inherit its chroma QP offset value from the last specified set of chroma QP offset values (e.g., from the neighboring quantization group to the left) or another previously specified chroma QP offset value (e.g., from the neighboring quantization group at the top, or another neighboring quantization group). If neighboring chroma QP offset values are not available, some embodiments then use default QP offset values from the PPS and / or tile header. In some embodiments, the flag "cu_qp_update_signal" may present additional possible values for handling different options for specifying chroma QP offset values. For example, in some embodiments, if the flag is "0," the chroma QP offset value from the left neighboring quantization group is used; if the flag is "1," the chroma QP offset value from the top neighboring quantization group is used; and if the flag is "2," the current quantization group will explicitly specify a new set of chroma QP offset values. Otherwise, the current quantization group will use the default QP offset value from the PPS and / or tile header. In some embodiments, the chroma QP offset value inherited by the current quantization group is based on the average of the chroma QP offset values of at least two neighboring quantization groups (e.g., the left neighboring quantization group and the top neighboring quantization group). To further reduce overhead, some embodiments specify that these additional chroma QP offsets can only be within a small range of values, from -x to x. This specification alters the entropy coding process because it changes the expected statistics of the information being signaled (i.e., for context-adaptive binary arithmetic (CABAC) coding, or when using a "maximum-limited" general variable-length coding scheme). In some embodiments, the value of x is used as a header (i.e., a range) for the entire sequence; or the value of x is signaled at a higher syntax level, such as the PPS, SPS, or tile header. Some of these embodiments achieve this by specifying x directly, or by specifying the value log2(x) when x corresponds to a power of 2. Some embodiments specify x by separating the magnitude (i.e., abs(x)) from the sign of x. In CABAC, the value of abs(x) corresponds to the cMax parameter required in the binarization process. [b.] [Additional chromaticity in the specified table] [QP] [Offset] [] To further reduce overhead or bit usage, some embodiments specify all possible offset values for each component, or combined values for two components, at a higher-level syntax level (e.g., in a sequence parameter set (SPS), picture parameter set (PPS), or in the current tile header). In some of these embodiments, the higher-level header (SPS / PPS / tile header) lists the different possible offset values in table form, with each entry in the table assigned an index. Then, at the coding unit / quantization group level, some embodiments simply specify the index or indexes of the desired quantization offset value. These offsets can be independent of the offset values specified in the PPS or tile header, or, conversely, in addition to the offset values specified there. To reduce bitstream size, some embodiments limit the number of entries in the table to a maximum value. [picture] [8] A hierarchical video coding structure 800 is described that includes a table 890 of possible chroma QP offset values encoded at a higher level header. The chroma QG at a lower level of the video coding structure then uses an index to select one of the possible chroma QP offset values from the table 890 used to calculate the chroma QP value. The video coding structure 800 is similar to the video coding structure 100. The video coding structure 800 includes a number of pictures 801 to 803. Among these pictures, picture 802 includes a number of tiles 811 to 812. Tile 811 includes a number of chroma QGs 821 to 823. Picture 802 has a picture parameter set (PPS) 831 that includes a set of chroma QP offset specifications that are applicable to all chroma QGs covered by picture 802. As described above, [picture] [1] and with reference to equations (1) and (2), some embodiments also specify a tile-level chroma QP offset (not shown), and the adaptation of the chroma QP value is based on chroma QP offset parameters from multiple levels of the video coding hierarchy. [picture] Chroma QGs 121 to 123 of [1], each of chroma QGs 821 to 823 selects a chroma QP offset value set from Table 890. Table 890 is an array of multiple entries. Each entry contains a set of chroma QP offsets that can be selected by any of the quantization groups in the picture. In this example, table 890 includes entries 891 through 895, which correspond to chroma QP offset values A, B, C, D, and E, respectively. In some embodiments, each of these entries can be selected by any number of quantization groups or not at all. [picture] In the example of [8], both QGs 822 and 823 select chroma QP offset set A (891), while QG 821 selects chroma QP offset set C (893). In some embodiments, the encoder determines which and how many chroma QP offset sets to include in the table in order to minimize bit rate. Once the chroma QP offset value set has been identified from Table 890, the calculation of the chroma quantization parameters is similar to [picture] [1] is calculated. [picture] In the example of [8], luma QP value 851 (from the four luma QGs of overlapping chroma QG 821) may be applied to chroma QG 821, luma QP value 852 (from the same luma QG) may be applied to chroma QG 822, and luma QP value 853 (from the two luma QGs of overlapping chroma QG 823) may be applied to QG 823. Adding luma QP value 851, chroma QP offset 831 (for picture 802 and / or for tile 811), and chroma QP offset C (taken from table entry 893 of QG 821) produces chroma QP value 861 for QG 821. Adding luma QP value 852, chroma QP offset 831, and chroma QP offset A (taken from table entry 891) produces chroma QP value 862 for QG 822. The chroma QP value 863 for QG 823 is generated by adding the luma QP value 853 , the chroma QP offset 831 , and the chroma QP offset A (taken from table entry 891 ). In some embodiments, a table of chroma QP offsets is encoded in the header area of the encoded video picture. [picture] [9] An example picture header or PPS 900 illustrating coding table 890. Picture header 900 is described by a pseudocode modified from the H.265 standard to accommodate additional chroma QP offsets. The pseudocode for picture header 900 is illustrated with newly added, dimmed rows 911 through 917 for the purpose of defining quantization groups for chroma QP offsets and encoding table 890. Specifically, the parameter "additional_chroma_qp_offset" in row 911 informs the decoder that additional chroma QP offsets will be specified for picture 802, and the parameter "chroma_qp_offset_max_depth" in row 913 defines the level (and therefore size or hierarchical depth) of the QG in picture 802. Rows 914 through 916 of the pseudocode then define the set of chroma QP offset values for each entry (891 through 895) of table 890. As illustrated, each entry in the table is assigned a chroma QP offset value for the first component ("chroma_cmp0_qp_offset[k]") and a chroma QP offset value for the second component ("chroma_cmp1_qp_offset[k]"). Since the two chroma components of each entry in the chroma QP offset table may be correlated, some embodiments use the chroma QP offset value of one chroma component to predict the chroma QP offset value of the other chroma component. In other words, for the kth entry in the table, if chroma_cmp0_qp_offset[k] represents the offset value of the first chroma component and chroma_cmp1_qp_offset[k] represents the offset value of the second chroma component, then the chroma QP offset of the second chroma component of the kth entry is calculated as: chroma_cmp1_qp_offset[k] = chroma_cmp0_qp_offset[k] + delta_chroma_cmp1_qp_offset[k] (7) That is, instead of sending the full offset parameter (ie, chroma_cmp1_qp_offset[k]), some embodiments send a differential parameter (ie, delta_chroma_cmp1_qp_offset[k]) used to calculate the offset of the second chroma QP offset chroma_cmp1_qp_offset[k]. [picture]
[10] illustrates the picture header 1000 encoding each entry (891-895) of the chroma QP offset table 890 as chroma_cmp0_qp_offset[k] and delta_chroma_cmp1_qp_offset[k] (at lines 1015 and 1016) according to equation (7). There are other possible methods for predictively encoding the chroma QP offset table to reduce bit usage. For example, in some embodiments, the chroma QP offset table entries are coded predictively with respect to each other, so that all but one entry is predicted from the others. One example is to encode each successive entry after the first entry in the table as a delta value predicted from the previous entry in the table. In some embodiments, the first entry is an explicit QP. In some embodiments, the first entry is itself an offset, and therefore also a delta. [picture]
[11] An example coding tree unit 1100 is described that may be in a quantization group that uses chroma QP offset values from a table in a picture header. The coding tree unit 1100 is described by a pseudocode modified from the H.265 pseudocode to accommodate the additional chroma QP offset. The coding tree unit 1100 is in a quantization group 821 of a tile 811. The tile 811 is in a picture 802 whose header or PPS includes table 890. Because in some embodiments there is no distinction between coding tree units in a quantization group that explicitly embed chroma QP offset values and coding tree units in a quantization group that use a set of chroma QP offset values from a table, the pseudocode for the coding tree unit 1100 is the same as that for the [picture] The pseudo code of the code tree unit 400 of [4] is the same. The pseudocode for coding tree unit 1100 is illustrated with newly added, shaded rows 1111 through 1113 for determining whether the coding tree unit is part of a QG. Specifically, in row 1111, the parameter "additional_chroma_qp_offset" indicates whether additional chroma information is allowed. The parameter "chroma_qp_offset_max_depth" is compared to the variable "log2CbSize" to determine whether coding tree unit 1100 is within a chroma quantization group. For some embodiments, this also initializes the start of the chroma quantization group. [picture]
[12] illustrates a quantization group that uses an index to select an entry in a table of chroma QP offset values. As discussed above, the quantization group 821 includes a coding tree unit 1100, which in turn includes a transform unit 1200. The transform unit 1200 is described in pseudo code modified from the H.265 pseudo code. [picture]
[12] in order to accommodate the additional chroma QP offset. The pseudocode for transform unit 1200 is illustrated using newly added dimmed rows 1211 through 1214 for the purpose of specifying additional chroma QP offsets. Specifically, at row 1211, the parameter "additional_chroma_qp_offset" indicates whether additional chroma information is allowed. The variable "IsCrCuQpOffsetCoded" indicates whether chroma QP offset information needs to be specified (or received) at transform unit 1200. If the chroma QP offset values for QG 821 have already been coded, no chroma QP offset values need to be specified again. If the chroma QP offset value set for QG 821 has not yet been coded (i.e., if the variable "IsCrCuQpOffsetCoded" is 0), transform unit 1200 specifies the index "cu_chroma_qp_offset_table_index" at row 1212 for selecting an entry from table 890. For this example, the index value is set to select the chroma QP offset value contained in item C (893) of Table 890. For some embodiments, [picture]
[13] A process 1300 is conceptually described for determining a chroma QP value from a video bitstream using a table of possible chroma QP offset values to implement additional chroma QP offset provisions. In some embodiments, the process 1300 is executed by a video decoder that uses the process to perform dequantization of chroma components when decoding a video bitstream for display or other purposes. In some embodiments, the process 1300 begins when the decoder has received a video bitstream and begins decoding a particular video picture. At 1305, processing procedure 1300 processes parameters in a picture header or PPS for a particular video picture. In some embodiments, the picture header includes a flag for determining whether additional chroma QP offsets are allowed for the video picture. The picture header also includes parameters for identifying the layer in the video hierarchy where the quantization group for the additional chroma QP offsets is to be defined. Next, the process identifies (at 1310) higher-level chroma QP offset values. In some embodiments, these higher-level chroma QP offset values are picture-level offsets encoded in a picture header (or as part of a PPS). In some embodiments, these higher-level chroma QP offset values are tile-level offsets encoded in a tile header. Some embodiments identify multiple higher-level chroma QP offset values, including both picture-level and tile-level values. The process then determines (at 1320) whether additional chroma QP offsets are allowed or available for this particular video picture or tile. If additional chroma QP offsets are not available, the process proceeds to 1350. If additional chroma QP offsets are available, the process proceeds to 1325 to retrieve a chroma QP offset table entry from the bitstream. Next, the process determines (at 1330) whether it has reached the beginning of a quantization group. In some embodiments, the process checks whether it is at a level of the video hierarchy that has been identified as a quantization group. If the process is not at the beginning of a quantization group (e.g., already inside a QG), the process proceeds to 1350. If the process is at the beginning of a quantization group, the process proceeds to 1335. At 1335, the process determines whether there are any residual chroma coefficients to be encoded. In some embodiments, if there are no residual chroma coefficients, the offset (or its associated index) is never signaled. Signaling begins when the first non-zero transform coefficient of a chroma block is encountered, and the chroma QP offset becomes valid from that point onward until the end of the chroma quantization group. The chroma QP offset is 0 until the first non-zero transform coefficient of a chroma block is encountered (thus, regions without chroma coefficients will not have any chroma QP offset signaled). If there are residual chroma coefficients to be encoded, the process proceeds to 1340. Otherwise, the process proceeds to 1350. At 1340, the process selects one of the entries in the chroma QP offset table as the set of chroma QP offset values for the quantization group. In some embodiments, this operation requires receiving an index from the bitstream and selecting an entry from the chroma QP offset table using the received index. In some embodiments where the chroma QP offset values in the table are predictively encoded, the process includes recovering the chroma QP offset values (for both chroma components) from the predicted or delta values. The process then identifies (at 1350) the luma QP value for the quantization group. As mentioned, in some embodiments, the luma QP value is defined for the same quantization group as the chroma QP value. In some embodiments, the quantization groups for luma QP and chroma QP are defined independently. Next, the process calculates (at 1360) the chroma QP value. Some embodiments calculate the chroma QP value using the identified luma QP value and using each identified chroma QP offset. In some embodiments, this is achieved by adding the luma QP value to all identified chroma QP offsets (from the tile / picture level and from the QG level), as described above in equations (1) to (6). The process then determines (at 1370) whether the process has reached the end of the picture (if the higher-level chroma QP offset is used for the entire picture) or whether the process has reached the end of the tile (if the higher-level chroma QP offset is used for the tile). If so, the process 1300 ends. If not, the process 1300 returns to 1330 to process the next quantization group. [picture] [5] [picture] [6] and [picture]
[12] Several different methods have been introduced for specifying chroma QP offset values for quantization groups. In some embodiments, quantization groups in different pictures or tiles may use different methods for specifying chroma QP offset values. In some of these embodiments, the encoder selects the best method it deems necessary for the current picture or tile in order to achieve greater flexibility or reduce coding overhead. For some embodiments, [picture]
[14] Pseudocode for a picture header 1400 is illustrated, including selection of a method for specifying additional chroma QP offset values. Picture header 1400 is similar to picture header 900 in that it includes the flags "additional_chroma_qp_offset" (at line 1411) and "chroma_qp_offset_max_depth" (at line 1413). Picture header 1400 also includes entries for chroma QP offset table 890 (at lines 1415-1417). However, unlike picture header 900, picture header 1400 also specifies a method by setting "chroma_qp_offset_method" (at line 1414). [picture]
[15] illustrates a pseudocode implementation for transform unit 1500 that can use any of three different methods to specify the chroma QP offset value for the quantization group. When the method flag "chroma_qp_offset_method" is set to 0 (at line 1512), as in [picture] In
[12] , the transform unit 1500 uses the index "cu_chroma_qp_offset_table_index" to select an entry from table 890 for calculating the chroma QP offset value of QG 821. When the method flag is set to 1 (at line 1514), as in [picture] In [5], transform unit 1500 encodes two independent chroma QP offset values for the two chroma components (using both the magnitude and sign flags). When the method flag is set to 2 (at line 1522), as in [picture] In [6], transform unit 1500 encodes only one chroma QP offset value for two chroma components. [II.] [Assign additional chroma] [QP] [Offset] [] Encoders in different embodiments use different methods for identifying and assigning additional chroma QP offset values. In some embodiments, the encoder analyzes images in a video sequence to identify the most appropriate QP value for quantization (e.g., for best balancing quality and bitrate). Some embodiments analyze various regions of an image to identify quantization groups that can be best encoded with a common set of chroma QP values. Some embodiments perform a pre-analysis step in which the encoder performs a region-level analysis (e.g., for each N×M block, where (for example) N=M=4, or object segmentation-based analysis) to extract intensity (e.g., mean, or brightness for luma and saturation for color), hue, variation / activity / texture characteristics, noise characteristics, and motion characteristics (e.g., motion vectors and / or prediction distortion values) for each color component in that region. Because different types of video content can be combined into the same video stream or even the same video image, some embodiments identify different regions within an image containing different types of video content. In some of these embodiments, different regions containing different types of video content are assigned different chroma QP offset values or assigned to different quantization groups. Some embodiments distinguish between graphical content and actual video content. Some embodiments distinguish between 4:4:4 video content originally coded in 4:4:4 format and 4:4:4 video content upsampled from 4:2:0 format. Some embodiments identify video content that may have originally had different bit depths. In addition to the relationship with color components and rate control information, some embodiments use these characteristics of the video content to determine the quantization levels or quantization relationships between all color components. [picture]
[16] An example segmentation and classification of image 1600 is described. The image is divided into a number of coding tree units, and each coding tree unit is further divided into a number of coding units. The various coding units of image 1600 are divided into four different chroma QP offset groups. In some embodiments, the four different chroma QP offset groups are signaled in the PPS, and the encoder can choose to signal the appropriate group at the quantization group level. In some embodiments, the chroma QP offset is explicitly signaled within each quantization group during encoding. The choice of chroma QP offset value is determined by the relative visual complexity, or texture, between the luma and chroma components. Quantization groups for image 1600 are assigned to different categories 1611 through 1614 based on their texture level. For example, areas that appear monotonous across all color components belong to the first category 1611, monotonous luminance areas with highly textured color components belong to the second category 1612, highly textured luminance areas with monotonous color information belong to the third category 1613, and areas with high texture across all three components belong to the fourth category 1614. In some embodiments, the encoder can generate additional subcategories for each category based on motion and intensity. In some embodiments, additional categories are formed by considering the differences between the two chroma components themselves. These categories can then be used to specify different sets of chroma QP offsets for different categories. For example, in some embodiments, the first category 1611 (all monotone) is assigned a zero or negative QP offset. In this scenario, negative chroma QP offsets allow for improved chroma quality distribution in those regions, as the improved chroma quality can be more noticeable due to the characteristics of all color components. For the second category 1612 (monotonic luma, high-texture color information), larger positive QP offsets can be used. In this case, larger chroma QPs allow for better control of the chroma components, avoiding dominating the luma components, while some texture masking from the chroma information can still be used to ensure good subjective quality. For the third category 1613 (high-texture luma vs. monotone chroma), some embodiments use larger negative chroma QP offsets to help ensure slightly better color quality. Some other embodiments use larger positive chroma QP offsets to better exploit luma texture masking. For the fourth category 1614 (high texture on all three components), some embodiments use zero or positive chroma QP offsets to better exploit luma texture masking and avoid wasting bits for a small improvement in chroma subjective quality. It's important to note that the exact chroma QP offset value assigned to a quantization group depends on the corresponding luma QP value for that quantization group. Different decisions can be made based on subjective or objective quality and bitrate decision mechanisms in other encoders. In some embodiments, if a limited number of chroma QP offsets are permitted or required (in cases where their specification would require a possible increase in bit additions beyond the determination of the region and its desired QP offset set), additional decisions are performed based on the presence of each offset set and its perceived subjective or objective quality impact. Some of these embodiments then use these criteria to "prune" the possible number of chroma QP offset sets signaled within the bitstream, since only the most important offset sets are now signaled. Regions classified as "pruned" within a category can still be assigned offset sets that are close enough to the desired values. Given some subjective or objective weighting process, this assignment represents a compromise whereby chroma QP offsets specified as "suboptimal" for all grouped region types may individually (but given bit rate constraints) provide the best overall / joint performance. For real-time applications where pre-analysis isn't possible, some embodiments use a "front-end decision" of QP offsets. In some embodiments, the decision is based on analysis of data from previously coded images. In some embodiments where chroma QP offsets are explicitly signaled for each quantization group, the decision is made on the fly or dynamically for each actively coded block based on its selected luma quantization parameter, its various spatiotemporal and color characteristics, and coding information from past blocks (i.e., the bitrate, bit relationship between luma and chroma, and the distortion introduced by other similar or neighboring blocks coded earlier). In some embodiments, the luma quantization parameter is derived jointly with the chroma QP offset, given the same type of bitrate and content characteristic relationships and conditions. Some embodiments further refine the selection of a chroma QP offset for the current picture by examining its neighboring pictures. Specifically, some embodiments examine how these neighboring pictures have been or will be coded, and how they temporally relate to the current picture and its regions. For example, if the current picture will be coded as a "critical" picture (e.g., an intra or "periodic" refresh picture), some embodiments encode an additional chroma QP offset to improve the chroma quality of the current picture. Conversely, if the current picture is a disposable or less important picture in the coding hierarchy, some embodiments avoid assigning an additional chroma QP offset in a manner that results in a higher bitrate (or even use no additional chroma QP offset at all). For other types of pictures, some embodiments use a more moderate chroma QP offset variation to achieve a better compromise between bitrate and quality. In some embodiments, different chroma QP offset parameters are specified for different scalability levels (such as for resolution, quality, bit depth, etc.). Some embodiments apply additional chroma QP offsets to 3D / multi-view applications where different chroma QP offsets are assigned to different view angles. For example, chroma QP offsets can be allocated and predicted to account for stereo masking in order to reduce overhead and maximize the subjective quality of such a system. For some embodiments, [picture]
[17] Conceptually illustrates a process 1700 for analyzing different regions of an image and assigning chroma QP offsets accordingly. In some embodiments, the process is performed by a video encoder. The process begins when it receives (at 1710) a video picture. This video picture can be a raw image from an uncompressed video stream or a decoded picture from a compressed video bitstream. The process then identifies (at 1720) a region in the picture that shares common characteristics that make a common set of common chroma QP offset values suitable for coding units in that region. For example, some embodiments identify a region originally coded in 4:2:0 format as one where a higher QP value (and therefore a positive chroma QP offset) can be used to reduce bitrate. Conversely, some embodiments identify a region natively coded in 4:4:4 format as one where a lower QP value (and therefore a lower or negative QP offset value) is required to maintain quality. The process then analyzes (at 1730) the spatiotemporal characteristics of each region. In some embodiments, this analysis includes analysis of the region's texture / variation / activity, format (4:4:4 or 4:2:0, etc.), noise, motion, bit depth, or other characteristics that may affect the relationship between luma and chroma, or between the two chroma components. Next, the processor assigns (at 1740) a chroma QP offset value based on the analysis of the region. In some embodiments, the processor first identifies the chroma QP value appropriate for the region based on the analysis performed in 1730, and then subtracts the selected luma QP value from the identified chroma QP value to obtain the desired total chroma QP offset value. In some embodiments, the processor 1740 also decomposes the total chroma QP offset value into chroma QP offset values at various levels of the video coding hierarchy (e.g., picture, tile, and quantization group). Some embodiments identify the chroma QP offset value for the quantization group by subtracting the chroma QP offset value of a higher level syntax element from the total chroma QP offset value. In some embodiments, as described below, see [picture] As described in
[18] , this operation is performed by a rate controller. The process then fills (at 1750) the chroma QP offset values with the chroma QP offset values identified for the quantization group in the region. The process 1700 also encodes its corresponding index value in the quantization group as discussed above in Section 1b. For some embodiments that explicitly specify chroma QP offset values, the process 1700 encodes the chroma QP offset values within the quantization group itself as discussed above in Section 1a. The process then determines (at 1760) whether there are more regions in the image remaining to be analyzed. If so, the process returns to 1720. Otherwise, the process 1700 ends. [III.] [Video System] [picture]
[18] A general video encoder 1800 (e.g., a HEVC encoder) is described for use in some embodiments of the present invention. The encoder 1800 receives a video stream from a video source 1805 and generates a compressed encoded bit stream 1895 to be stored and / or transmitted. The video encoder includes a transform module 1810, a quantization module 1815, an entropy encoder 1820, an inverse quantization module 1825, an inverse transform module 1830, a deblocking filter 1840, a sample adaptive offset (SAO) filter 1845, a frame buffer 1850, a rate control module 1835, and a prediction module 1890. The prediction module 1890 includes a motion estimation module 1860, a motion compensation module 1865, an intra prediction module 1870, and a mode decision module 1880. In some embodiments, the video encoder 1800 also includes a video display 1855. Quantization module 1815 quantizes transform coefficients (e.g., DCT) from transform module 1810 using quantization parameters. In some embodiments, transform module 1810 can be bypassed entirely (such as in the transform bypass mode supported by HEVC), so that quantization module 1815 receives image values or image prediction error values without transforming. Quantization module 1815 applies different QP values for different regions / blocks of each color component. The QP values used by quantization module 1815 are also encoded into bitstream 1895 as luma QP values and chroma QP offsets. In some embodiments, the QP values used by quantization module 1815 are determined and supplied by rate control module 1835. The rate control module 1835 controls the bit rate of the encoded video bit stream by controlling the QP values used by the quantization module 1815 (and the inverse quantization module 1825). In some embodiments, the rate control module 1835 supplies different QP values to the quantization module 1815 for different quantization groups. To identify the QP values (for luma and chroma) that are most suitable for quantization (e.g., the best balance between quality and bit rate for a given video sequence), in some embodiments, the rate control module 1835 performs the above [chapter] In some embodiments, the rate controller 1835 also uses the analysis described in [II] to arrive at a luma QP value for each luma quantization group and a set of chroma QP offset values for each chroma quantization group. In some embodiments, the rate controller 1835 also uses the analysis for identifying regions and assigning (luma and chroma) quantization groups. In some embodiments, rate controller 1835 decomposes a set of chroma QP offset values into sets of chroma QP offset values at different levels of the video coding hierarchy. In some of these embodiments, at least some sets of chroma QP offset values are further decomposed into prediction values between two chroma components, as shown in equations (1) to (7) above. In some embodiments, a lower-level QP offset value is identified by subtracting a higher-level QP offset value from a total chroma QP offset value. For example, some embodiments calculate a quantization group-level chroma QP offset as: QPoffset_quant_group[i] = QPchroma[i] - QPluma - QPoffset_pps[i] - QPoffset_slice[i] (8) These sets of chroma QP offset values are then provided to the entropy encoder 1820 for encoding into the bitstream 1895. In some embodiments, the rate controller 1835 compiles a table of possible quantization group level offsets and provides the compiled table to the entropy encoder 1820. The entropy encoder module 1820 is a module that performs entropy encoding (e.g., CABAC) on quantized transform coefficients, parameters, and other information and packs them into the bitstream 1895. In some embodiments, the entropy encoder module 1820 receives chroma QP offset values from the rate controller module 1835 and entropy encodes these offset values into the bitstream. In some embodiments, the entropy encoder 1820 is as described above. [chapter] [Ib], the chroma QP offset values are encoded as a table in the PPS area of a picture. In some embodiments, the entropy encoder is as described above. [chapter] Chroma QP offset values are encoded into individual quantization groups as described in [Ia]. Deblocking module 1840 is a loop filter module that improves the visual quality and prediction performance of video codec systems by smoothing out sharp edges that may form between coded blocks. In some embodiments, deblocking module 1840 uses a total chroma QP value calculated from all applicable chroma QP offsets to perform its deblocking operation (i.e., by including quantization group and picture / tile chroma QP offsets). In some other embodiments, deblocking module 1840 only considers some of the specified chroma QP offsets, or only considers chroma QP offsets from the PPS or tile header. Some of these embodiments control the deblocking process based on signaled chroma QP offset values. For example, some embodiments adjust deblocking parameters to compensate for chroma QP offset values. Deblocking can also be performed out-of-loop and as a post-processing step. In some embodiments, some of the total calculated chroma QP value or chroma QP offset is used by other types of in-loop or out-of-loop post-processing such as SAO (e.g., SAO filter 1845), adaptive loop filter (ALF), or noise addition. [picture]
[19] A general video decoder 1900 (e.g., a HEVC decoder) for use in some embodiments of the present invention is described. The decoder 1900 receives a bitstream 1905 and decodes it for display by a display module 1965. The decoder 1900 includes an entropy decoder 1910, an inverse quantization module 1920, an inverse transform module 1930, an intra prediction module 1950, an inter prediction module 1980, a deblocking module 1960, a SAO module 1970, and a frame buffer 1975. Entropy decoding module 1910 performs entropy decoding on the input bitstream and extracts transform coefficients and parameters used by other modules in decoder 1900. Specifically, chroma quantization information included in input bitstream 1905 is extracted by entropy decoder 1910 and passed to inverse quantization module 1920. In some embodiments, the chroma quantization information includes additional chroma QP offset values extracted from a table in the PPS, a tile header, or from the quantization group itself. The deblocking module 1960 of the decoder 1900 performs similar functions to the deblocking module 1840 of the coder 1800. Specifically, the deblocking module 1960 also uses the final chroma QP value calculated from all applicable chroma QP offsets in its deblocking operation (i.e., by including the chroma QP offsets for the quantization group and picture / tile). In some other embodiments, the deblocking module 1960 only considers some of the specified chroma QP offsets, or only considers the chroma QP offsets from the PPS or tile header. In some embodiments, some of the total calculated chroma QP value or chroma QP offsets are used by other types of in-loop or out-of-loop post-processing, such as SAO (e.g., SAO filter 1970), ALF, or noise addition. [IV.] [Electronic System] [] Many of the features and applications described above are implemented as software processes, defined as a set of instructions recorded on a computer-readable storage medium (also referred to as computer-readable media). When these instructions are executed by one or more computing or processing units (e.g., one or more processors, processor cores, or other processing units), the instructions cause the processing unit(s) to perform the actions indicated by the instructions. Examples of computer-readable media include (but are not limited to) CD-ROMs, flash drives, random access memory (RAM) chips, hard drives, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. Computer-readable media does not include carrier waves and electronic signals transmitted wirelessly or via wired connections. In this specification, the term "software" is intended to include firmware residing in read-only memory or applications stored in magnetic storage that can be read into memory for processing by a processor. Furthermore, in some embodiments, multiple software inventions may be implemented as sub-parts of a larger program, leaving distinct software inventions. In some implementations, multiple software inventions may also be implemented as separate programs. Finally, any combination of separate programs that together implement the software inventions described herein falls within the scope of the present invention. In some embodiments, a software program installed to operate on one or more electronic systems defines one or more specific machine implementations that execute and carry out the operations of the software program. [picture]
[20] An electronic system 2000 for implementing some embodiments of the present invention is conceptually illustrated. The electronic system 2000 may be a computer (e.g., a desktop computer, a personal computer, a tablet computer, etc.), a phone, a PDA, or any other type of electronic device. The electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 2000 includes a bus 2005, a processing unit 2010, a graphics processing unit (GPU) 2015, system memory 2020, a network 2025, a read-only memory 2030, a permanent storage device 2035, an input device 2040, and an output device 2045. Bus 2005 generally represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of electronic system 2000. For example, bus 2005 communicatively connects processing unit 2010 with read-only memory 2030, GPU 2015, system memory 2020, and permanent storage 2035. Processing unit 2010 retrieves instructions to be executed and data to be processed from these various memory units in order to execute the processing procedures of the present invention. In various embodiments, the processing unit(s) may be a single processor or a multi-core processor. Some instructions are passed to and executed by GPU 2015. GPU 2015 can offload various computations or supplement the image processing provided by processing unit 2010. Read-only memory (ROM) 2030 stores static data and instructions required by processing unit 2010 and other modules of the electronic system. Persistent storage device 2035, on the other hand, is a read-write memory device. This device is a non-volatile memory unit that stores instructions and data even when electronic system 2000 is turned off. Some embodiments of the present invention utilize a mass storage device (such as a magnetic disk or optical disk and its corresponding drive) as permanent storage device 2035. Other embodiments use removable storage devices (such as floppy disks, flash memory devices, and their corresponding disk drives) as permanent storage devices. Similar to permanent storage device 2035, system memory 2020 is a read-write memory device. However, unlike storage device 2035, system memory 2020 is a volatile read-write memory device, such as random access memory. System memory 2020 stores some of the instructions and data required by the processor during execution. In some embodiments, the processing procedures of the present invention are stored in system memory 2020, permanent storage device 2035, and / or read-only memory 2030. For example, various memory units contain instructions for processing multimedia clips according to some embodiments. Processing unit 2010 retrieves instructions to be executed and data to be processed from these various memory units in order to execute the processing procedures of some embodiments. Bus 2005 also connects to input and output devices 2040 and 2045. Input device 2040 allows a user to communicate information and selection commands to the electronic system. Input device 2040 includes an alphanumeric keyboard and pointing device (also known as a "cursor control"), a camera (e.g., a webcam), a microphone for receiving voice commands, and similar devices. Output device 2045 displays images generated by the electronic system or outputs data in other ways. Output device 2045 includes a printer and display device such as a cathode ray tube (CRT) or liquid crystal display (LCD), as well as speakers or similar audio output devices. Some embodiments include devices that function as both input and output devices, such as a touch screen. Finally, if [picture] As shown in
[20] , bus 2005 also couples electronic system 2000 to network 2025 via a network adapter (not shown). In this way, the computer can be part of a computer network (such as a local area network ("LAN"), a wide area network ("WAN"), or an intranet) or a network such as the Internet. Any or all components of electronic system 2000 may be used in conjunction with the present invention. Some embodiments include electronic components, such as microprocessors, storage, and memory, that store computer program instructions in machine-readable or computer-readable media (alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, compact disc read-only memory (CD-ROM), compact disc recordable (CD-R), compact disc rewritable (CD-RW), read-only video discs (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini SD card, micro SD card, etc.), magnetic and / or solid-state drives, read-only and rewritable Blu-Ray® discs, ultra-high-density optical discs, any other optical or magnetic media, and floppy disks. The computer-readable media may store a computer program executable by at least one processing unit and comprising a set of instructions for performing various operations. Examples of computer programs or computer code include machine code (such as produced by a compiler) and files including high-level code that are executed by a computer, electronic component, or microprocessor using an interpreter. While the above discussion primarily refers to microprocessors or multi-core processors executing software, some embodiments utilize one or more integrated circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some embodiments, these integrated circuits execute instructions stored on the circuits themselves. Additionally, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices. As used in this specification and any solutions herein, the terms "computer," "server," "processor," and "memory" refer to electronic or other technical devices. These terms exclude persons or groups of persons. For the purposes of this specification, the term "display" means displayed on an electronic device. As used in this specification and any solutions herein, the terms "computer-readable medium" and "machine-readable medium" are strictly limited to tangible physical objects that store information in a form readable by a computer. These terms exclude any wireless signals, wired download signals, and any other transient signals. Although the present invention has been described with reference to numerous specific details, one skilled in the art will recognize that the present invention can be embodied in other specific forms without departing from the spirit of the invention. [picture] [7] [picture]
[13] and [picture]
[17] ) The processing procedures are conceptually described. The specific operations of such processing procedures may not be performed in the exact order shown and described. The specific operations may not be performed in a continuous series of operations, and different specific operations may be performed in different embodiments. In addition, the processing procedures may be implemented using several sub-processors or as part of a larger macroprocessor. Therefore, it will be understood by those skilled in the art that the present invention is not limited by the foregoing illustrative details, but is defined by the scope of the appended claims.
[0006] 100: Hierarchical video coding structure 101: Image 102: Image 103: Image 111: Tile 112: Tile 113: Tile 121: Chroma QG 122: Chroma QG 123: Chroma QG 132: Picture Parameter Set (PPS) 141: Chroma QP offset set 142: Chroma QP offset set 143: Chroma QP offset set 171: Brightness QG 172: Brightness QG 173: Brightness QG 174: Brightness QG 175: Brightness QG 181: Brightness QG 182: Brightness QG 241:Offset value 242:Offset value 243: Chroma QP offset 244: Immediate offset value 245:Difference value 251: Quantity[0] 252: Quantity[1] 253: Chroma component 254: Chroma component 255: Chroma QP offset of the first chroma component 256: Chroma QP offset of the second chroma component 300: Image header or PPS 311: Darkening 312:Darkening row 313:Darkening row 400: Write code tree unit 411: Darkening 412: Darkening 413: Darkening 500: Transformation unit 511: Darkening 512:Darkening row 513: Darkening 514:Darkening row 515: Darkening 516: Darkening 517: Darkening 518: Darkening 519: Darkening 600: Transformation unit 611: OK 612: OK 613: OK 614: OK 615: OK 616: OK 700:Handler 800: Hierarchical video coding structure 801: Image 802: Image 803: Image 811: Tile 812: Tile 821: Chroma QG 822: Chroma QG 823: Chroma QG 831: Picture Parameter Set (PPS) / Chroma QP Offset 851: Brightness QP value 852: Brightness QP value 853: Brightness QP value 861: Chroma QP value 862: Chroma QP value 863: Chroma QP value 890: Table of possible chroma QP offset values 891: Chroma QP offset table entry / Chroma QP offset set A 892: Project 893: Chroma QP offset table entry / Chroma QP offset set C 894: Project 895: Project 900: Image header or PPS 911: Darkening 912: Darkening 913: Darkening 914: Darkening 915: Darkening 916: Darkening 917: Darkening 1000: Image header 1011: OK 1012: OK 1013: OK 1014: OK 1015: OK 1016: OK 1017: OK 1100: Write code tree unit 1111:Darkening row 1112:Darkening row 1113:Darkening row 1200: Transformation unit 1211:Darkening row 1212:Darkening row 1213:Darkening row 1214:Darkening row 1300:Handler 1400: Image header 1411: OK 1412: OK 1413: OK 1414: OK 1415: OK 1416: OK 1417: OK 1418: OK 1500: Transformation unit 1511: OK 1512: OK 1513: OK 1514: OK 1515: OK 1516: OK 1517: OK 1518: OK 1519: OK 1520: OK 1521: OK 1522: OK 1523: OK 1524: OK 1525: OK 1526: OK 1527: OK 1528: OK 1600: Image 1611: Category 1 1612: Second Category 1613: Category III 1614: Category 4 1700:Handler 1800: Universal Video Encoder 1805: Video Source 1810: Transformation Module 1815: Quantitative Module 1820: Entropy Encoder 1825: Inverse Quantization Module 1830: Inverse Transformation Module 1835: Rate Control Module 1840: Deblocking filter 1845: Sample Adaptive Offset (SAO) filter 1850:Frame buffer 1855: Video Display 1860: Motion Estimation Module 1865: Motion Compensation Module 1870: In-frame prediction module 1880: Mode Decision Module 1890: Prediction Module 1895: Compressed Encoded Bit Stream 1900: Universal Video Decoder 1905:Bitstream 1910: Entropy Decoder 1920: Inverse Quantization Module 1930: Inverse Transformation Module 1950: In-frame prediction module 1960: Blockchain decomposition module 1965: Display Module 1970: SAO module 1975: Frame Buffer 1980: Inter-frame prediction module 2000: Electronic Systems 2005: Bus 2010: Processing Unit 2015: Graphics Processing Unit (GPU) 2020: System Memory 2025: Internet 2030: Read-only memory 2035: Permanent storage 2040: Input device 2045: Output device
Claims
1. An electronic device comprising: one or more processing units; a memory storing a plurality of instructions, wherein, when the instructions are executed, the memory is configured to cause the one or more processing units to perform a plurality of operations, the operations comprising: receiving, in a bitstream, an encoded video image group, including an encoded first video image having a hierarchical write structure having at least a first write level, a second write level, and a third write level, the encoded first video image including two or more sets of chroma quantization parameter (QP) offset values at the first write level and the second write level, each set of chroma QP offset values providing information about a plurality of chroma QPs of video units encompassed by one level of the hierarchical write structure, the encoded first video image further including one or more chroma quantization groups (QGs) at the third write level; For a specific chroma QG of one or more chroma QGs at the third write-code level, an additional set of chroma QP offsets is identified, the identification comprising: determining a parameter based on information included in the first write-code level or the second write-code level, the parameter specifying the maximum length of a binary string in the bitstream, and using the parameter to identify the additional set of chroma QP offsets; and decoding the encoded first video image by calculating multiple chroma QP values corresponding to the one or more chroma QGs of the encoded first video image, wherein for the specific chroma QG, the decoding comprises: selecting the identified additional set of chroma QP offsets, and using the selected set of chroma QP offsets to calculate the chroma QP values corresponding to the specific chroma QG.
2. The electronic device of claim 1, wherein calculating the chromaticity QP values further includes combining the chromaticity QP offsets in the selected set and one or more luminance QP values corresponding to the particular chromaticity QG.
3. The electronic device of claim 2, wherein at least the particular chromaticity QG is associated with a plurality of luminance QP values, and wherein calculating the chromaticity QP values of the particular chromaticity QG includes calculating a plurality of different chromaticity QP values corresponding to the plurality of luminance QP values.
4. The electronic device of claim 2, wherein at least the particular chromaticity QG is associated with a single luminance QP value, and wherein calculating the chromaticity QP value of the particular chromaticity QG includes calculating a single chromaticity QP value corresponding to the single luminance QP value.
5. The electronic device of claim 1, wherein receiving, in the bitstream, a encoded first video image comprising two or more sets of chroma QP offsets at the first write-code level and the second write-code level, and receiving, in the bitstream, a plurality of first chroma QP offsets corresponding to the first write-code level of the encoded first video image and a plurality of second chroma QP offsets corresponding to the second write-code level of the encoded first video image, the operation further comprising: receiving, in the bitstream, a plurality of third chroma QP offsets corresponding to the third write-code level of the encoded first video image.
6. The electronic device of claim 5, wherein in one of the write levels of the hierarchical write structure, the first write level and the second write level are higher than the third write level, and wherein calculating the chromaticity QP values corresponding to the particular chromaticity QG includes using the first chromaticity QP offset.
7. The electronic device of claim 6, wherein calculating the chroma QP values corresponding to the one or more chroma QGs of the encoded first video image includes using the first chroma QP offsets to calculate the multiple chroma QP values of the one or more chroma QGs associated with the third write-code level of the encoded first video image.
8. The electronic device of claim 6, wherein the first write-code level corresponds to an image level, the second write-code level corresponds to a tile level, and the third write-code level corresponds to one of a write-code tree unit level, a write-code unit level, or a transform unit level, and wherein calculating the chroma QP values corresponding to the specific chroma QG includes using a plurality of chroma QP offsets specified in a set of image parameters (PPS) of the encoded first video image.
9. The electronic device of claim 1, further comprising: receiving in the bitstream one or more indices corresponding to the one or more chroma QGs of the encoded first video image; and for at least one first chroma QG of the one or more chroma QGs, using an index corresponding to the first chroma QG to select a chroma QP offset value from the set of chroma QP offset values.
10. The electronic device of claim 1, wherein the parameter specifying the maximum length of the binary string in the bitstream includes the cMax parameter used in the binary conversion process.
11. The electronic device of claim 10, wherein the encoded first video image is encoded using context-adaptive binary arithmetic (CABAC) write codes.
Citation Information
Patent Citations
Method and apparatus for encoding and decoding depth map of multi-view video data
TW201313034A
Image decoding apparatus and method for handling intra-image predictive decoding with various color spaces and color signal resolutions
US20090190829A1
Methods and devices for data compression using a non-uniform reconstruction space
US20130083845A1
