Chroma quantization in video coding

Abstract

To provide a method for signaling an additional chroma QP offset value specific to a quantization group.SOLUTION: In a method, each quantization group explicitly specifies its own set of chroma QP offset values, or a table of possible sets of chroma QP offset values is specified in a header area of a picture, the quantization group selects an entry from the table using the index, and its own set of chroma QP offset values is determined. Then, using the quantization group-specific saturation QP offset value, in addition to the chroma QP offset values already specified for higher levels of the video coding hierarchy, the chroma QP values are determined for the blocks within the quantization group.SELECTED DRAWING: None

Description

[Technical field]

[0001] Claiming the benefit of prior application This application claims the benefit of U.S. Provisional Patent Application No. 61 / 875,664, filed September 9, 2013. U.S. Provisional Patent Application No. 61 / 875,664 is incorporated herein by reference. [Background technology]

[0002] The next-generation High Efficiency Video Coding (HEVC / H.265) standard, jointly developed by ITU and ISO MPEG, introduces several new video coding tools to increase video coding efficiency over previous video coding standards and technologies, such as MPEG-2, MPEG-4 Part 2, MPEG-4 AVC / H.264, VC1, and VP8, among others. In its first version, the new standard can support the coding of 8- or 10-bit material in YUV4:2:0 with three well-defined profiles: Main, Main10, and Main Still Picture. However, work is still ongoing to support other color sampling formats and color spaces, including YUV4:2:2, YUV4:4:4, and RGB4:4:4, among others, as well as sample precision (bit depth) beyond 10 bits. Coding of such material is of great interest for a variety of professional applications, such as cinema applications, capture, video editing, archiving, and medical imaging, but also for a variety of consumer applications, such as screen content compression and sharing, remote computing, and gaming, among others.

[0003] Until recently, existing video codecs have mostly prioritized coding the luma component of an image. Coding parameters that control the coding characteristics and quality of luma information are provided primarily at lower levels of the video coding hierarchy, e.g., slice, coding tree block, coding unit, or transform block level in HEVC. Control and tuning parameters for all other components can be provided primarily at higher levels, e.g., sequence, picture parameter set, or slice level, in conjunction with modifying luma control parameters. For example, in MPEG-4 AVC, luma quantization and corresponding quantization parameters (QPs) were controlled at the macroblock level using signaled parameters. In the picture parameter set, one offset was provided for each chroma component. Each chroma QP offset controls the QP value of the corresponding chroma component in relation to the luma QP within each macroblock. However, this relation is fixed for the entire picture. If one wanted to change the quality in the luma or chroma area, the relationship was so tight that the other components were also affected. HEVC also offered a little extra control, since it allows separate chroma quantization offset signaling per slice. However, using multiple slices may not be desirable or functional for some applications, and the control offered is still somewhat coarse.

[0004] For some applications and content, being able to control the quality of some color components separately can be very important to improve the overall quality, compression ratio and overall user experience. For example, some areas may feature texture differences or noise characteristics among the color components, while emphasizing color edges may be more or less important than emphasizing that information in luminance. Also, for 4:4:4 applications such as video display sharing and remote computing, it may be desirable to code RGB content, where the importance of red and blue components, and therefore the desired control, tends to be higher than the chroma components in the YUV domain. Also, one may wish to code mixed video content, which is a combination of synthetic content and natural images or videos, such as computer graphics and computer applications. In this scenario, having the ability to control the chroma quantization parameter could have a significant impact on the coding performance and subjective quality, given the difference in characteristics between the natural and synthetic content, and the possibility that the natural content was originally a 4:2:0 image that was upconverted for display in 4:4:4.

[0005] What is needed is a method that allows more control over the chroma quantization parameters compared to existing codecs, and allows extending this support to all common color formats (e.g., YUV, RGV, YCoCg, or YCoCg-R) and all common color sampling schemes (e.g., 4:2:0, 4:2:2, 4:4:4, or 4:4:4:4), as well as various bit depths for each component. Such a method should allow signaling and modification of chroma QP offset information within a coding block in a more flexible manner, without imposing any restrictions on color formats or sampling formats. Summary of the Invention

[0006] To provide a video coding system with more flexibility in specifying chroma quantization parameters (QPs), some embodiments of the present invention provide a method for signaling additional quantization group-specific chroma QP offset values. In some embodiments, each quantization group explicitly specifies its own set of chroma QP offset values. In some embodiments, a table of possible sets of chroma QP offset values ​​is specified in the picture or slice header area, and each quantization group uses an index to select an entry from the table to determine its own set of chroma QP offset values. The quantization group level chroma QP offset values ​​are then used to determine chroma QP values ​​for pixel blocks or pixel sets within the quantization group. In some embodiments, the quantization group chroma QP values ​​are used together with the block's luma QP and chroma QP offset values ​​already specified at higher levels of the video coding hierarchy.

[0007] Some embodiments associate chroma QP offset specifications with quantization groups (QGs) that contain one or more video data units. In some embodiments, each QG is associated with its own set of chroma QP offset specifications, and in some embodiments, the set of chroma QP offset specifications associated with a QG are coded or incorporated into the coding structure of the video data units in the QG.

[0008] To further reduce overhead or bit usage, some embodiments specify all available offset values ​​of individual components or combined values ​​of both components in a higher level, e.g., in the sequence parameter set (SPS), picture parameter set (PPS), or in the current slice header. In some embodiments, the higher level syntax header (SPS / PPS / slice header) lists various possible offset values ​​in a table format, and each entry in the table is assigned an index. Then, at the coding unit / quantization group level, some embodiments specify only the index of the desired quantization offset value. Such offset may be independent of the offset value specified in the PPS or slice header, or conversely, may be added to the offset value specified in the PPS or slice header. To reduce the bitstream size, some embodiments limit the number of entries in the table to a maximum value.

[0009] In another embodiment, the encoder uses another method to select and assign additional quantization group specific chroma QP offset values. In some embodiments, a pre-analysis step is performed in which the encoder performs a region level analysis. In some embodiments, various regions in an image made of various video content types are identified. In some embodiments, various regions with various types of video content are assigned various chroma QP offset values ​​or are placed into various quantization groups. In some embodiments, graphics content is differentiated from actual video content. In some embodiments, 4:4:4 video content originally coded in 4:4:4 format is differentiated from 4:4:4 video content upsampled from 4:2:0 format. In some embodiments, video content that may have originally been at a different bit depth is distinguished. In some embodiments, these characteristics of the video content are used in addition to their relationships across color components and rate control information to determine quantization levels or quantization relationships between all color components.

[0010] The above Summary is intended to serve as a brief introduction to some embodiments of the present invention. It is not intended to introduce or outline all of the inventive subject matter disclosed herein. The following Detailed Description and the Figures referred to in the Detailed Description will provide detailed descriptions of the embodiments described in the Summary and other embodiments. Thus, a complete review of the Summary, Detailed Description, and Figures is required to understand all of the embodiments described herein. [Brief description of the drawings]

[0011] The novel features of the invention are set forth in the appended claims, but for purposes of illustration, certain embodiments of the invention are set forth in the following drawings. [Figure 1] FIG. 1 illustrates a hierarchical video coding structure 100 that includes chroma quantization groups, with each group having its own set of chroma QP offset specifications. [Figure 2a] A diagram showing multiple methods for encoding a set of chroma QP offset values ​​for a chroma quantization group. [Figure 2b] A diagram showing multiple methods for encoding a set of chroma QP offset values ​​for a chroma quantization group. [Figure 2c] A diagram showing multiple methods for encoding a set of chroma QP offset values ​​for a chroma quantization group. [Diagram 3] FIG. 1 illustrates an example picture header or picture parameter set (PPS) specifying chroma quantization group definitions. [Figure 4] FIG. 2 illustrates an example coding tree unit that may be in a chroma quantization group. [Diagram 5] A diagram illustrating the specification of additional chroma QP offsets within a quantization group. [Figure 6] A diagram illustrating the specification of additional chroma QP offsets within a quantization group using another method for specifying chroma QP offset values ​​for two chroma components. [Figure 7] FIG. 1 conceptually illustrates a process for determining chroma QP values ​​from a video bitstream that allows for additional chroma QP offset specifications. [Figure 8] A diagram showing a hierarchical video coding structure that includes a table of possible chroma QP offset values ​​in a high-level header. [Figure 9] FIG. 1 illustrates an example picture header or picture parameter set (PPS) that encodes a table of chroma QP offsets. [Figure 10] FIG. 13 illustrates a picture header that predictively encodes each entry in the chroma QP offset table. [Figure 11] FIG. 13 illustrates an example coding tree unit that may be in a quantization group that uses an index to look up a chroma QP offset value from a table in a picture header. [Figure 12] FIG. 13 illustrates a quantization group that uses an index to select an entry in a table of chroma QP offset values. [Figure 13] FIG. 1 conceptually illustrates a process for determining chroma QP values ​​from a video bitstream that implements additional chroma QP offset specifications using a table of possible chroma QP offset values. [Figure 14] FIG. 13 shows pseudocode for a picture header including options for how to specify additional chroma QP offset values. [Figure 15] FIG. 13 illustrates a pseudocode implementation for a transform unit that can specify chroma QP offset values ​​for a quantization group using one of three different methods. [Figure 16] FIG. 2 illustrates an exemplary segmentation and classification of an image. [Figure 17] FIG. 17 conceptually illustrates a process 1700 for analyzing various regions of an image and assigning chroma QP offsets accordingly. [Figure 18] FIG. 1 illustrates a typical video encoder. [Figure 19] FIG. 1 illustrates a typical video decoder. [Figure 20] FIG. 1 conceptually illustrates an electronic system in which some embodiments of the present invention may be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] In the following description, numerous details are set forth for purposes of explanation. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order not to obscure the description of the present invention with unnecessary detail.

[0013] In order to provide a video coding system with more flexibility in specifying chroma quantization parameters (chroma QPs), some embodiments of the present invention provide a method for signaling additional quantization group specific chroma QP offset values. In some embodiments, each quantization group explicitly specifies its own set of chroma QP offset values. In some embodiments, a table of possible sets of chroma QP offset values ​​is specified in the picture or slice header area, and each quantization group uses an index to select an entry from the table to determine its respective set of chroma QP offset values. The quantization group level chroma QP offset values ​​are then used to determine the chroma QP value for each block in the quantization group. In some embodiments, the quantization group chroma QP value is used together with the block's luma QP and chroma QP offset values ​​already specified at a higher level of the video coding hierarchy.

[0014] Section I below describes how to specify additional chroma QP offset values. Specifically, Section Ia describes how to explicitly specify additional chroma QP offset values ​​within a quantization group, and Section Ib describes how to specify a table of possible chroma QP offset values. Section II then describes various systems and methods for identifying additional chroma QP offset values ​​and assigning them to various regions of an image. Section III describes video encoder and decoder systems that implement some embodiments of the present invention, and Section IV describes computer systems on which some embodiments of the present invention are implemented.

[0015] I. Specifying an additional chroma QP offset a. Specify additional chroma QP offsets within a quantization group Some embodiments of the present invention provide a method for specifying chroma quantization parameter offset values ​​(chroma QP offsets) for encoding a sequence of video. The method associates chroma QP offset designations with chroma quantization groups, each chroma QG encompassing one or more video data units (e.g., coding units in HEVC). In some embodiments, a set of chroma QP offset designations associated with a chroma quantization group is coded or incorporated into the coding structure of the video data units in the chroma quantization group. In some embodiments, the chroma QP offset designation for the chroma quantization group is applied in addition to other chroma QP offset designations specified for a video coding structure located at a higher level of the video coding hierarchy (e.g., a slice or picture) that encompasses the chroma quantization group. In some embodiments, the chroma QP offsets from various levels are applied together (e.g., by adding them to the luma QP value) to determine a chroma quantization parameter (chroma QP).

[0016] In some embodiments, in addition to the luma quantization group (luma QG), a chroma quantization group (chroma QG) is defined. The luma QG also contains one or more video data units, but the luma QG is for specifying a luma quantization parameter (luma QP). In some embodiments, the chroma QG may overlap or contain one or more luma QGs. In other words, different regions in the chroma QG may belong to different luma QGs and therefore have different luma QPs.

[0017] In accordance with some embodiments of the present invention, FIG. 1 illustrates a hierarchical video coding structure 100 in which chroma quantization groups (chroma QGs) are included with each group having its own set of chroma QP offset specifications. The hierarchical video coding structure 100 includes a number of pictures 101-103. Among them, picture 102 includes a number of slices 111-113. Slice 112 includes a number of chroma QGs 121-123. Picture 102 has a picture parameter set (PPS) 132 that includes a set of chroma QP offset specifications. The chroma QGs 121-123 are associated with chroma QP offset sets 141-143, respectively.

[0018] The hierarchical video coding structure 100 corresponds to a sequence of video images. The sequence of video images is made up of layers of video data units at various levels of the hierarchy, with video data units at higher levels encompassing one or more video data units at lower levels. For example, a group of pictures (GOP) is a video data unit at a higher hierarchical level than a picture, a picture is a video data unit at a higher hierarchical level than a slice of a picture, a slice is a video data unit at a higher hierarchical level than a coding tree unit, and so on. In some embodiments, at least some parameters specified for a higher-level video data unit are applicable to lower-level video data units that the higher-level video data unit encompasses. For example, in some embodiments, a picture-level chroma QP offset specification for picture 102 (from PPS 132) is applicable to all chroma QGs (e.g., chroma QGs 121-123) encompassed by picture 102.

[0019] The video coding structure 100 is coded as a bitstream in some embodiments. Such a bitstream is based on established video coding formats, such as the HEVC / H.265 standard or the MPEG-4 AVC / H.264 standard. A picture based on the H.265 standard may include one or more slices, and each slice may have one or more coding tree units (CTUs). Furthermore, each CTU may be subdivided as blocks. H.265 also allows the definition of luma quantization groups for quantizing / dequantizing luma components. Each quantization group allows the derivation of luma quantization parameters specific to the video data units in the quantization group. Once derived, the luma quantization parameters are used to quantize the luma DCT coefficients. Under the H.265 standard, chroma quantization parameters are derived from luma quantization parameters based on offset values ​​provided at the picture layer or slice layer of the H.265 compliant bitstream. The video coding structure 100 provides chroma QP offset signaling or specification in addition to the picture / slice layer chroma QP offsets allowed by the H.265 standard.

[0020] The chroma QGs 141-143 are defined ranges in the video coding structure 100. Each of the QGs 141-143 is associated with its own set of chroma QP offset values ​​141-143. In some embodiments, a quantization group may be a group of video data units that share the same set of QP values. In some embodiments, the signaling for a chroma QP offset in a QG applies to all video data units in the QG. In some embodiments, the QP offset is applied starting from the first signaled chroma residual information and thereafter. Any area preceding the signaling is not associated with the signaled chroma QP offset. In other words, the QG may be "fragmented".

[0021] In some embodiments, the QG (chroma or luma) may be defined at different levels or depths of the video coding hierarchy, such as coding tree units, coding units, or transform units, which are supported in HEVC. In some embodiments, the definition of the QG for the chroma QP offset inherits the definition of the QG for the luma component, if available.

[0022] In some embodiments, the depth, level and size of a QG (chroma or luma) can be flexibly specified in the bitstream and may vary from picture to picture. For example, one picture (e.g., 101) may specify its chroma QG at the top level of the coding quadtree (i.e., the QG is a CTU), and another picture (e.g., 103) may specify its chroma QG at a lower level of the coding quadtree (e.g., the QG is a coding block of the quadtree). In some embodiments, a higher level parameter (e.g., PPS or slice header) specifies at which level of the video coding hierarchy the picture's QG will be defined.

[0023] FIG. 1 also illustrates how the offset specified by the saturation QG is used to calculate the saturation QP value. In some embodiments as shown in FIG. 1, a first set of saturation QP offset parameters are already signaled in the PPS 132 (and / or the header of the slice 112). In some embodiments, the saturation QP offset values ​​specified at a level above the saturation QG (e.g., the PPS and / or the slice header) are ignored if the QG specifies its own saturation QP offset value. In some embodiments, the saturation QP offset syntax element at the higher level and the saturation QP offset at the QG level are considered together to reduce overhead. That is, the adaptation of the saturation QP value is done while relating both sets of parameters simultaneously, and the signaled information may be considered as a "secondary" offset. In some embodiments, the saturation QP value for the current saturation QG is calculated as follows: QPchroma[i]=QPluma+QPoffset_pps[i]+QPoffset_quant_group[i] (1) (i=0 for the first saturation component; i=1 for the second saturation component) where QPluma is the QP of the luma component corresponding to the current chroma quantization group, QPoffset_pps[i] is the QP offset of the i-th chroma component from the current PPS (or slice header), and QPoffset_quant_group[i] is the additional offset signaled at the QG level for that component. QPoffset_pps[i]+QPoffset_quant_group[i] thus constitutes the overall chroma QP offset for the i-th chroma component.

[0024] In some embodiments, the adaptation of the chroma QP value is based on chroma QP offset parameters from multiple levels of the video coding hierarchy. For example, in some embodiments, the calculation of the chroma QP value considers chroma QP offset values ​​from the slice header as well as from the PPS. In some embodiments, the chroma QP value for a chroma component i of the current QG is calculated as follows: QPchroma[i]=QPluma+QPoffset_pps[i] +QPoffset_slice[i]+QPoffset_quant_group[i] (2) where QPoffset_slice[i] is the QP offset of the i-th chroma component from the current slice header, QPoffset_pps[i] is the QP offset of the i-th chroma component from the current PPS, and the sum QPoffset_pps[i]+QPoffset_slice[i]+QPoffset_quant_group[i] is the overall chroma QP offset for the i-th chroma component. In some embodiments, one or more additional chroma QP offset values ​​may be optionally specified at other layers of the video coding hierarchy. To encode or decode a particular coding block, some embodiments use some or all of the chroma offset QP values ​​specified along each level of the video coding hierarchy that contains (or is applicable to) the particular coding block.

[0025] As mentioned above, the chroma QP value of a chroma QG is calculated by adding the chroma QP offset to the luma QP value applicable to the chroma QG. In some embodiments, the luma QP value corresponding to a chroma QG may vary within the chroma QG. This is because, due to the overlap of the luma QG and the chroma QG, the chroma QG may contain coding units that fall into different luma QGs and therefore have different luma QP values. As a result, different coding units within the chroma QG may end up having different chroma QP values ​​(because the chroma QP offset applicable to the chroma QG is applied to different luma QP values ​​within the chroma QG).

[0026] In the example of FIG. 1, chroma QG 121 has four corresponding luma QP values ​​(QP luma) because chroma QG 121 overlaps with four different luma QGs 171-174. These four luma QP values ​​of chroma QG 121 result in four possible chroma QP values ​​after applying the chroma QP offsets (QPoffset_pps, QPoffset_slice, QPoffset_quant_group) from PPS offset 132 and QG1 offset 141. Thus, a coding unit in chroma QG 121 may have one of these four possible chroma QP values ​​depending on the underlying luma QG in which the coding unit falls. Similarly, chroma QG 123 overlaps with two different luma QGs 181-182, and a coding unit in chroma QG 123 may have one of two possible chroma QP values ​​depending on the underlying luma QG in which the coding unit falls.

[0027] On the other hand, chroma QG 122 has only one corresponding luma QP value because chroma QG 122 is entirely within luma QG 175 (or contains the exact same set of coding units as luma QG 175), and therefore all coding units in chroma QG 122 have the same chroma QP value after applying the chroma QP offset.

[0028] Since there are two chroma components for each luma component for most video coding formats (e.g., YCbCr), in some embodiments, each chroma QP offset for a quantization group is a set of values ​​that includes instructions for calculating two offset values ​​for the two chroma components. For some embodiments, Figures 2a-c show multiple methods for encoding a set of offset values ​​for the QG 141 of Figure 1. Some embodiments use only one of these methods to encode the set of chroma QP offset values. Some embodiments select different methods for different QGs based on the specific characteristics of the QGs.

[0029] Figure 2a illustrates a set of chroma QP offset values ​​as two independent offset values ​​241 and 242. These two values ​​are used directly as the chroma QP offsets for component [0] (251) and component [1] (252). In some embodiments, this method of encoding the chroma QP offsets is selected when the two chroma components are largely uncorrelated with each other. Another example of this method of encoding the chroma QP offset values ​​is described in more detail below with reference to Figure 5.

[0030] Figure 2b shows a set of offset values ​​coded as one value 243 used simultaneously as chroma QP offset by both chroma components (253 and 254). In some embodiments, this method is selected when the two chroma components are very similar in their relationship to the luma component. Another example of this method of coding chroma QP offset values ​​is detailed below with reference to Figure 6.

[0031] FIG. 2c illustrates a set of offset values ​​that are coded as a direct offset value 244 and a difference value 245 based on the direct offset. In this example, the direct offset value 244 is directly applied as the chroma QP offset of the first chroma component (255), while the sum of the difference value 245 and the direct value 244 is used as the chroma QP offset value of the second chroma component (256). In some embodiments, this method is selected when the two chroma components differ in relation to the luma component by a small offset that can be coded cheaply in a bitstream of only a few bits. This method codes the second chroma QP offset (for component [1]) in relation to the first chroma QP offset. That is, the second chroma QP offset is then predicted from the first chroma QP offset (for component [0]). In some embodiments, the chroma QP value for the first chroma component is calculated as follows: QPchroma[0]=QPluma+QPoffset_pps[0]+QPoffset_quant_group[0] (3)

[0032] The chroma QP value for the second chroma component is calculated as follows: QPchroma[1]=QPluma+QPoffset_pps[1]+QPoffset_quant_group[0] +QPoffset_quant_group[1] (4) (i=0 for the first saturation component; i=1 for the second saturation component)

[0033] As noted above, in some embodiments, chroma QP offset values ​​from multiple different layers of a video coding hierarchy are used to derive a final chroma QP value. For some embodiments that use chroma QP offset values ​​from both the PPS and the slice header, the chroma QP value for the first chroma component is calculated as follows: QPchroma[0]=QPluma+QPoffset_pps[0]+QPoffset_slice[0] +QPoffset_quant_group[0] (5)

[0034] The chroma QP value for the second chroma component is calculated as follows: QPchroma[1]=QPluma+QPoffset_pps[1]+QPoffset_slice[1] +QPoffset_quant_group[0]+QPoffset_quant_group[1] (6)

[0035] As mentioned above, in some embodiments, the definition of luma quantization group and / or chroma quantization group is specified by a parameter at the picture level. FIG. 3 illustrates an exemplary picture header or PPS 300 that specifies the definition of chroma quantization group. The exemplary picture header 300 is associated with a picture 102 of a video coding structure 100. The picture header 300 is described by pseudocode modified from the H.265 standard to accommodate the additional chroma QP offset (i.e., adaptation of the chroma QP value). In some embodiments, the pseudocode for a video standard (e.g., H.265 standard or modified standard) describes the operations required of a video decoder when processing a bitstream compliant with that video standard. The pseudocode also describes the operations required of a video encoder when generating a bitstream compliant with that video standard.

[0036] For illustrative purposes, pseudocode for picture header 300 is shown in Figure 3 as black lines 311-313. Lines 311-313 allow for additional chroma QP offset values ​​and define chroma quantization groups for chroma QP offsets in picture 102. Specifically, the parameter "additional_chroma_qp_offset" in line 311 informs the decoder that there will be additional chroma QP offset specifications in the bitstream. If this variable is not set for a particular picture, the bitstream will not include bits to specify additional chroma QP offsets for that picture to avoid wasting bits.

[0037] The parameter "chroma_qp_offset_max_depth" on line 313 defines the level (hence the size or hierarchical depth) of the chroma QG. This parameter can be set equal to the size of the entire coding tree unit (CTU) or as small as the size of the smallest possible coding unit in the bitstream. In some embodiments, additional chroma QP offsets can be specified and signaled at any desired level (e.g., at the CTU level, at the chroma-defined quantization group level, at the coding unit level, at the transform unit level, etc.).

[0038] In some embodiments, instead of defining quantization groups specifically for chroma QP offsets, the bitstream uses the parameter "diff_cu_qp_delta_depth", which also defines the quantization group for luma QP. In some embodiments, if a parameter to define the quantization group is not available, the same QP for luma is assigned to the whole picture and no QP adaptation for luma is allowed.

[0039] 4 is a diagram illustrating an example coding tree unit 400 that may be in a quantization group. The coding tree unit 400 is described by pseudocode modified from H.265 pseudocode to accommodate additional chroma QP offsets. The coding tree unit 400 is in a quantization group 122, which is in a slice 112 of a picture 102.

[0040] The pseudocode for coding tree unit 400 is shown in black lines 411-413, which are added to determine if the coding tree unit is part of a QG. Specifically, in line 411, a parameter "additional_chroma_qp_offset" (e.g., from picture header 300) is used to indicate if additional chroma information is allowed. The parameter "chroma_qp_offset_max_depth" is compared to the variable "log2CbSize" to determine if coding tree unit 400 is inside a quantization group (such as quantization group 122). If so, then in line 412, the pseudocode sets the variable "IsCrCuQpOffsetCoded" to 0. In some embodiments, this initializes the start of the chroma quantization group.

[0041] 5 is a diagram illustrating an additional chroma QP offset specification in QG 122. As discussed above with reference to FIGS. 3 and 4, QG 122 is defined by picture header 300 to include coding tree unit 400. The additional chroma QP offset is specified in transform unit 500 of coding tree unit 400. Transform unit 500 is described by pseudocode modified from H.265 pseudocode to accommodate the additional chroma QP offset.

[0042] Pseudocode for transform unit 500 is shown in black lines 511-519, which are added to specify additional chroma QP offsets. Specifically, in line 511, the parameter "additional_chroma_qp_offset" is used to indicate whether additional chroma information is allowed. The variable "IsCrCuQpOffsetCoded" is used to indicate whether chroma QP offset information needs to be specified (or received) in transform unit 500. If the chroma QP offset value for QG122 is already coded, there is no need to specify the chroma QP offset value again.

[0043] If the QG 122 has not yet coded a set of chroma QP offset values ​​(i.e., the variable "IsCrCuQpOffsetCoded" is 0), then in lines 512-517, transform unit 500 specifies chroma QP offset values ​​for the two chroma components. In this case, the transform unit specifies a set of chroma QP offsets as two independent signed values, each of which is signaled with a magnitude ("cu_chroma_cmp0_qp_offset_abs" or "cu_chroma_cmp1_qp_offset_abs") and a sign flag ("cu_chroma_cmp0_qp_delta_sign_flag" and "cu_chroma_cmp0_qp_delta_sign_flag"). The pseudocode then sets the variable "IsCrCuQpOffsetCoded" to 1, indicating that the chroma QP offset values ​​for this quantization group have already been specified.

[0044] As described above with reference to Fig. 2a-b, in another embodiment, the quantization group specifies the chroma QP offset values ​​for the two chroma components in a different way. Fig. 6 illustrates the specification of additional chroma QP offsets in the QG 122 with an alternative way to specify the chroma QP offset values ​​for the two chroma components. Fig. 6 illustrates a transform unit 600 described by a pseudocode similar to that for transform unit 500, with the only difference being that lines 611-616 specify only one chroma QP offset value (magnitude + sign) for both chroma components. In some embodiments, it is sufficient to specify one chroma QP offset value for both chroma components, since in most cases the importance of the two color components will not change. The signaling of one or two such parameters could be pre-determined or signaled at various higher syntax levels, e.g., SPS, PPS or slice header.

[0045] For some embodiments, Figure 7 conceptually illustrates a process 700 for determining chroma QP values ​​from a video bitstream that allows for additional chroma QP offset specifications. In some embodiments, process 700 is performed by a video decoder that uses the process to inverse quantize chroma components when decoding a video bitstream for display. In some embodiments, the process begins when the decoder receives a video bitstream and begins decoding a particular video picture for display or other purposes.

[0046] At 705, process 700 processes parameters in a picture header or PPS of a particular video picture. As described above with reference to Figure 3, in some embodiments, the picture header or PPS of a video picture includes a flag (e.g., "additional_chroma_qp_offset") to determine whether additional chroma QP offsets are allowed for the video picture. The picture header or PPS also identifies the layer of the video hierarchy at which the additional chroma QP offsets are to be defined (e.g., by setting the variable "chroma_qp_offset_max_depth").

[0047] Next, the process identifies (at 710) a higher level chroma QP offset value. In some embodiments, this higher level chroma QP offset value is a picture level offset encoded in the picture header (or as part of the PPS). In some embodiments, this higher level chroma QP offset value is a slice level offset encoded in the slice header. Some embodiments identify multiple higher level chroma QP offset values, including both picture and slice levels.

[0048] The process then determines (at 720) whether additional chroma QP offsets are allowed or available for this particular video picture and / or slice. If additional chroma QP offsets are not available, the process proceeds to 750. If additional chroma QP offsets are available, the process proceeds to 730.

[0049] At 730, the process determines whether the start of a saturation quantization group has been reached. In some embodiments, the process checks whether it is located at a level of the video hierarchy that has been identified as a saturation quantization group. If the process is not located at the start of a saturation quantization group (e.g., already located within a saturation quantization group), the process proceeds to 750. If the process is located at the start of a saturation quantization group, the process proceeds to 740. Examples of operations 720 and 730 are discussed with reference to FIG. 4 above.

[0050] At 740, the process identifies a chroma QP offset value for the chroma quantization group. In some embodiments, the chroma QP offset is explicitly coded within the chroma quantization group, as discussed above with reference to Figures 5 and 6.

[0051] The process then identifies (at 750) luma QP values ​​for the chroma quantization groups. As noted above, in some embodiments, the luma QP values ​​are set according to the luma quantization groups, which may or may not be the same as the chroma quantization groups. Depending on the signaling, there may be more or less luma quantization groups than chroma quantization groups. In some embodiments, a luma quantization group may contain multiple chroma quantization groups, or vice versa. In some embodiments, the chroma quantization groups and luma quantization groups may overlap with each other. Thus, different coding units in the same chroma quantization group may have different luma QPs based on the luma quantization group.

[0052] Next, the process calculates (at 760) the chroma QP value. For a decoder, some embodiments calculate the chroma QP value from the identified luma QP value and the identified individual chroma QP offsets. In some embodiments, this is accomplished by adding all of the identified chroma QP offsets (from slice / picture level and QG level) to the luma QP value, as shown above as equations (1) through (6).

[0053] The process then determines (at 770) whether the end of a picture has been reached (if the higher level chroma QP offset is for the entire picture) or the end of a slice has been reached (if the higher level chroma QP offset is for a slice). If so, process 700 ends. If not, process 700 returns to 730 to process the next quantization group.

[0054] As shown above with reference to Figures 1-6, the chroma QP values ​​are predictively coded by using offsets from the luma QP values. In some embodiments, the offset values ​​themselves are also predictively coded by offsets from each other, e.g., the QG level offset is an offset from a slice and / or picture level offset, and / or the offset value of one chroma component is coded as an offset from another chroma component. In some embodiments, the chroma QP offset is further predicted from chroma values ​​or offsets of neighboring coding units or coding blocks, or from chroma values ​​or offsets of co-located coding units or coding blocks in neighboring video pictures, since they are likely to be sufficiently similar to the coding units or coding blocks of the current chroma quantization group. In some embodiments, no prediction is performed for at least some quantization groups. In such cases, the chroma QP values ​​are explicitly coded (not as offsets) such that the QP parameters of the luma components are ignored (since there is no need to offset from luma).

[0055] In some embodiments, a flag is used to indicate that a set of chroma QP offset values ​​from a neighboring quantization group will be used for the current quantization group. In some embodiments, the neighboring quantization group is spatially located to the left or above the current quantization group. For example, in some embodiments, a flag "cu_qp_update_signal" is used to indicate whether a chroma QP offset value of a neighboring quantization group will be used. In some embodiments, the flag "cu_qp_update_signal" is signaled before the "cu_chroma_qp_offset_abs" parameter in line 612 of FIG. 6.

[0056] If the flag is '1', the current quantization group will provide an update to specify its own chroma QP offset values. If the flag is '0', the current quantization group will not update its own set of chroma QP offset values, but will inherit chroma QP offset values ​​from the last specified set of chroma QP offset values ​​(e.g., from the neighboring quantization group to the left) or from another previously specified chroma QP offset value (e.g., from the neighboring quantization group above or from another neighboring quantization group). If neighboring chroma QP offset values ​​are not available, some embodiments use default QP offset values ​​from the PPS and / or slice header.

[0057] In some embodiments, the flag "cu_qp_update_signal" may take on additional possible values ​​to handle different options for specifying chroma QP offset values. For example, in some embodiments, if the flag is "0", the chroma QP offset values ​​from the left neighboring quantization group are used, if the flag is "1", the chroma QP offset values ​​from the upper neighboring quantization group are used, and if the flag is "2", the current quantization group will explicitly specify a new set of chroma QP offset values. Otherwise, the current quantization group will use default QP offset values ​​from the PPS and / or slice header. In some embodiments, the chroma QP offset values ​​inherited by the current quantization group are based on the average value of at least two (e.g., left and upper) of the neighboring quantization groups.

[0058] To further reduce overhead, some embodiments specify that these additional chroma QP offsets fall only in a small range of values, i.e., from -x to x. Doing so alters the entropy coding process, since it changes the expected statistics of the information that will be signaled (in the case of Context Adaptive Binary Arithmetic (CABAC) coding, or when using a "max-limited" universal variable length coding scheme). In some embodiments, the value (i.e., range) of x is pre-determined for the entire sequence, or signaled at a higher syntax level, such as the PPS, SPS, or slice header. Some embodiments achieve this by specifying x directly, or the value log2(x) if x falls on a power of 2. Some embodiments specify x by separating the magnitude, i.e., abs(x), and the sign of x. In CABAC, the value of abs(x) corresponds to the cMax parameter required in the binarization process.

[0059] b. Specify additional chroma QP offsets in the table To further reduce overhead and bit usage, some embodiments specify all possible offset values ​​of individual components or combined values ​​of both components in a higher level syntax, e.g. in the sequence parameter set (SPS) or picture parameter set (PPS) or in the current slice header. In some embodiments, the higher level header (SPS / PPS / slice header) lists the various possible offset values ​​in a table format, and each entry in the table is assigned an index. Then, at the coding unit / quantization group level, some embodiments specify only the index of the desired quantization offset value. Such offset may be independent of the offset value specified in the PPS or slice header, or conversely, may be added to the offset value specified in the PPS or slice header. To reduce the bitstream size, some embodiments limit the number of entries in the table to a maximum value.

[0060] 8 shows a hierarchical video coding structure 800 including a table 890 of possible chroma QP offset values ​​coded in a high level header. A chroma QG at a lower level of the video coding structure then uses the index to select one of the possible chroma QP offset values ​​from the table 890 to calculate a chroma QP value. The video coding structure 800 is similar to the video coding structure 100. The video coding structure 800 includes a number of pictures 801-803. Among them, the picture 802 includes a number of slices 811-812. The slice 811 includes a number of chroma QGs 821-823. The picture 802 has a picture parameter set (PPS) 831 that includes a set of chroma QP offset specifications applicable to all chroma QGs encompassed by the picture 802. Also, as discussed above with reference to Figure 1 and equations (1) and (2), some embodiments specify a slice-level chroma QP offset (not shown) and specify that the adaptation of chroma QP values ​​is based on chroma QP offset parameters from multiple levels of the video coding hierarchy. Unlike chroma QGs 121-123 of Figure 1, which specify their own set of chroma QP offset values, chroma QGs 821-823 each select a set of chroma QP offset values ​​from table 890.

[0061] Table 890 is an array containing multiple entries. Each entry stores a set of chroma QP offsets that may be selected by any quantization group in the picture. In this example, table 890 contains entries 891-895, which correspond to chroma QP offset sets A, B, C, D, and E, respectively. In some embodiments, each entry may be selected by any number of quantization groups or none at all. In the example of FIG. 8, QG 822 and QG 823 both select QP offset set A (891), and QG 821 selects chroma QP offset set C (893). In some embodiments, the encoder determines which and how many chroma QP offset sets to include in the table to minimize bitrate.

[0062] Once a set of chroma QP offset values ​​is identified from table 890, the calculation of the chroma quantization parameters is similar to that of Figure 1. In the example of Figure 8, luma QP value 851 (from the four luma QGs overlapping chroma QG 821) is applicable to chroma QG 821, luma QP value 852 (from the same luma QG) is applicable to chroma QG 822, and luma QP value 853 (from the two luma QGs overlapping chroma QG 823) is applicable to QG 823. Adding luma QP value 851, chroma QP offset 831 (for picture 802 and / or slice 811), and chroma QP offset C (retrieved from table entry 893 for QG 821) results in chroma QP value 861 for QG 821. Adding the luma QP value 852, the chroma QP offset 831, and the chroma QP offset A (retrieved from table entry 891) results in a chroma QP value 862 for QG 822. Adding the luma QP value 853, the chroma QP offset 831, and the chroma QP offset A (retrieved from table entry 891) results in a chroma QP value 863 for QG 823.

[0063] In some embodiments, a table of chroma QP offsets is coded within the header area of ​​a coded video picture. Figure 9 illustrates an exemplary picture header or PPS 900 that codes the table 890. The picture header 900 is described by pseudocode modified from the H.265 standard to accommodate the additional chroma QP offsets.

[0064] The pseudocode for picture header 900 is shown in black lines 911-917, which are added to define the quantization groups for the chroma QP offsets as well as to encode table 890. Specifically, the parameter "additional_chroma_qp_offset" in line 911 informs the decoder that an additional chroma QP offset specification is planned for picture 802, and the parameter "chroma_qp_offset_max_depth" in line 913 defines the level (and thus the size or hierarchical depth) for the QGs in picture 802. Lines 914-916 of the pseudocode then define the chroma QP offset values ​​for each entry (891-895) of table 890. As shown, each entry in the table is assigned a chroma QP offset value for a first component ("chroma_cmp0_qp_offset[k]") and a chroma QP offset value for a second component ("chroma_cmp1_qp_offset[k]").

[0065] Because the two chroma components for each entry in the chroma QP offset table are likely to be correlated, in some embodiments, the chroma QP offset value of one chroma component is used to predict the chroma QP offset value of the other chroma component. In other words, for the kth entry in the table, if chroma_cmp0_qp_offset[k] represents the offset value of the first chroma component and chroma_cmp1_qp_offset[k] represents the offset value of the second chroma component, then the chroma QP offset for the second chroma component of the kth entry is calculated as follows: chroma_cmp1_qp_offset[k]=chroma_cmp0_qp_offset[k] +delta_chroma_cmp1_qp_offset[k] (7)

[0066] That is, instead of sending the complete offset parameter, i.e., chroma_cmp1_qp_offset[k], in some embodiments, a differential parameter, i.e., delta_chroma_cmp1_qp_offset[k], is sent to calculate the offset for the second chroma QP offset, chroma_cmp1_qp_offset[k]. Figure 10 illustrates a picture header 1000 that encodes each entry (891-895) of chroma QP offset table 890 (at rows 1015 and 1016) as chroma_cmp0_qp_offset[k] and delta_chroma_cmp1_qp_offset[k] according to equation (7).

[0067] There may be other ways to predictively code the chroma QP offset table to reduce bit usage. For example, in some embodiments, the entries of the chroma QP offset table are predictively coded with respect to each other, such that all but one entry is predicted from the other entries. One example is to code each entry following the first entry in the table as a difference value predicted from the previous entry in the table. In some embodiments, the first entry is an explicit QP. In some embodiments, the first entry is itself an offset and thus is itself also a difference.

[0068] 11 illustrates an example coding tree unit 1100 that may be in a quantization group that uses chroma QP offset values ​​from a table in a picture header. The coding tree unit 1100 is described by pseudocode modified from H.265 pseudocode to accommodate the additional chroma QP offset. The coding tree unit 1100 is in a quantization group 821 of a slice 811. The slice 811 is in a picture 802, whose header or PPS includes a table 890. The pseudocode for the coding tree unit 1100 is identical to the pseudocode for the coding tree unit 400 of FIG. 4 because, in some embodiments, there is no difference between a coding tree unit in a quantization group that explicitly incorporates chroma QP offset values ​​and a coding tree unit in a quantization group that uses a set of chroma QP offset values ​​from a table.

[0069] Pseudocode for the coding tree unit 1100 is shown in black lines 1111-1113, which are added to determine if the coding tree unit is part of a QG. Specifically, line 1111 indicates whether additional chroma information is allowed using the parameter "additional_chroma_qp_offset." The parameter "chroma_qp_offset_max_depth" is compared to the variable "log2CbSize" to determine if the coding tree unit 1100 is within a chroma quantization group. For some embodiments, this also initializes the start of the chroma quantization group.

[0070] Figure 12 illustrates a quantization group that uses an index to select an entry in a table of chroma QP offset values. As discussed above, the quantization group 821 includes a coding tree unit 1100, which in turn includes a transform unit 1200. In Figure 12, the transform unit 1200 is described by pseudocode that has been modified from the H.265 pseudocode to accommodate the additional chroma QP offsets.

[0071] Pseudocode for transform unit 1200 is shown in black lines 1211-1214, which are added to specify additional chroma QP offsets. Specifically, line 1211 indicates whether additional chroma information is allowed using the parameter "additional_chroma_qp_offset". The variable "IsCrCuQpOffsetCoded" indicates whether it is necessary to specify (or receive) chroma QP offset information in transform unit 1200. If chroma QP offsets for QG821 are already coded, there is no need to specify the chroma QP offset values ​​again. If QG821 does not already code a set of chroma QP offset values ​​(i.e., the variable "IsCrCuQpOffsetCoded" is 0), transform unit 1200 in line 1212 specifies an index "cu_chroma_qp_offset_table_index" to select an entry from table 890. In this example, the index value is set to select the chroma QP offset value stored in entry C (893) of table 890.

[0072] For some embodiments, Figure 13 conceptually illustrates a process 1300 for determining chroma QP offset values ​​from a video bitstream that implements additional chroma QP offset specifications using a table of possible chroma QP offset values. In some embodiments, process 1300 is performed by a video decoder that uses the process to dequantize chroma components when decoding a video bitstream for display or other purposes. In some embodiments, process 1300 begins when the decoder receives a video bitstream, and in some embodiments, begins decoding a particular video picture.

[0073] At 1305, the process 1300 processes parameters in a picture header or PPS of a particular video picture. In some embodiments, the picture header includes a flag to determine whether additional chroma QP offsets are allowed for the video picture. The picture header also includes parameters to identify a layer of the video hierarchy at which quantization groups for the additional chroma QP offsets are to be defined.

[0074] Next, the process identifies (at 1310) a higher level chroma QP offset value. In some embodiments, this higher level chroma QP offset value is a picture level offset coded in the picture header (or as part of the PPS). In some embodiments, this higher level chroma QP offset value is a slice level offset coded in the slice header. In some embodiments, multiple higher level chroma QP offset values ​​are identified, including both picture and slice levels.

[0075] The process then determines (at 1320) whether additional chroma QP offsets are allowed or available for this particular video picture or slice. If additional chroma QP offsets are not available, the process proceeds to 1350. If additional chroma QP offsets are available, the process proceeds to 1325 to receive a chroma QP offset table entry from the bitstream.

[0076] The process then determines (at 1330) whether it has reached the start of a quantization group. In some embodiments, the process checks whether it is located at a level of the video hierarchy that has been identified as a quantization group. If the process is not located at the start of a quantization group (e.g., already inside a QG), the process proceeds to 1350. If the process is located at the start of a quantization group, the process proceeds to 1335.

[0077] At 1335, the process determines whether there are any residual chroma coefficients to code. In some embodiments, if there are no residual chroma coefficients, then no offsets (or their associated indicators) are signaled at all. Signaling begins when the first non-zero transform coefficient for a chroma block is encountered, and from that point onwards the chroma QP offsets are active until the end of the chroma quantization group. The chroma QP offsets are 0 until the first non-zero transform coefficient of a chroma block is encountered (so regions that have no chroma coefficients will not have chroma QP offset signaling). If there are residual chroma coefficients to code, the process proceeds to 1340. Otherwise, the process proceeds to 1350.

[0078] At 1340, the process selects one of the entries in the chroma QP offset table as storing a set of chroma QP offset values ​​for the quantization group. In some embodiments, this operation involves receiving an indication from the bitstream and selecting an entry from the chroma QP offset table using the received indication. In some embodiments where the chroma QP offset values ​​in the table are predictively encoded, this process includes recovering the chroma QP offset values ​​(for both chroma components) from a prediction or difference value.

[0079] The process then identifies (at 1350) luma QP values ​​for the quantization group. As mentioned above, in some embodiments, the luma QP values ​​are defined for the same quantization group as the quantization group for the chroma QP values. In some embodiments, the quantization group for the luma QP and the quantization group for the chroma QP are defined independently.

[0080] Next, the process calculates (at 1360) a chroma QP value. In some embodiments, the chroma QP value is calculated from the identified luma QP value and from each identified chroma QP offset. In some embodiments, this is accomplished by adding all identified chroma QP offsets (from slice / picture level and QG level) to the luma QP value as shown in equations (1)-(6) above.

[0081] The process then determines (at 1370) whether the end of a picture has been reached (if the higher level chroma QP offset is for the entire picture) or the end of a slice has been reached (if the higher level chroma QP offset is for a slice). If so, the process 1300 ends. If not, the process 1300 returns to 1330 to process the next quantization group.

[0082] 5, 6 and 12 introduce multiple different methods that a quantization group can use to specify a set of chroma QP offset values. In some embodiments, quantization groups in different pictures or slices may use different methods to specify chroma QP offset values. In some embodiments, the encoder selects the best method that it considers necessary for the current picture or slice to increase flexibility and reduce encoding overhead.

[0083] For some embodiments, Figure 14 illustrates pseudocode for a picture header 1400 that includes options for how to specify additional chroma QP offset values. Picture header 1400 is similar to picture header 900 in that it includes a flag "additional_chroma_qp_offset" (at line 1411) and a flag "chroma_qp_offset_max_depth" (at line 1413). Picture header 1400 also includes entries for chroma QP offset table 890 (at lines 1415-1417). However, unlike picture header 900, picture header 1400 also specifies a method by setting "chroma_qp_offset_method" (at line 1414).

[0084] FIG. 15 illustrates a pseudocode implementation for a transform unit 1500 that can specify chroma QP offset values ​​for a quantization group using one of three different methods. If the method flag "chroma_qp_offset_method" is set to 0 (at line 1512), transform unit 1500 uses index "cu_chroma_qp_offset_table_index" to select an entry from table 890 and calculates the chroma QP offset value for QG821 as in FIG. 12. If the method flag is set to 1 (at line 1514), transform unit 1500 encodes two independent chroma QP offset values ​​for the two chroma components (using both magnitude and sign flags) as in FIG. 5. If the method flag is set to 2 (at line 1522), transform unit 1500 encodes only one chroma QP offset value for both chroma components as in FIG. 6.

[0085] II. Assigning additional chroma QP offsets In other embodiments, the encoder identifies and assigns additional chroma QP offset values ​​using other methods. In some embodiments, the encoder analyzes images in a video sequence with the goal of identifying optimal QP values ​​for quantization (e.g., to best balance quality and bitrate). Some embodiments analyze various regions in an image with the goal of identifying quantization groups that can be optimally encoded with a common set of chroma QP values.

[0086] In some embodiments, a pre-analysis step is performed in which the encoder performs a region-level analysis (e.g., per NxM block when N=M=4, or based on object segmentation) to extract, for each color component in the region, the intensity (e.g., average luminance or value, and color saturation), hue, variance / activity / texture characteristics, noise characteristics, motion characteristics (e.g., motion vectors and / or predicted distortion values).

[0087] Since different types of video content may be combined within the same video stream or within the same video image, some embodiments identify different regions within an image of different video content types. In some embodiments, different regions with different types of video content are assigned different chroma QP offset values ​​or assigned to different quantization groups. Some embodiments distinguish graphics content from actual video content. Some embodiments distinguish 4:4:4 video content originally coded in 4:4:4 format from 4:4:4 video content upsampled from 4:2:0 format. Some embodiments distinguish video content that may have originally been at a different bit depth. Some embodiments use these characteristics of the video content in addition to their relationships across color components and rate control information to determine quantization levels or quantization relationships between all color components.

[0088] 16 is a diagram illustrating an example partitioning and classification of an image 1600. The image is partitioned into coding tree units, and each coding tree unit is subdivided into coding units. The various coding units of the image 1600 are partitioned into four different chroma QP offset groups. In some embodiments, the four different chroma QP offset groups are signaled in the PPS, and the encoder can select the appropriate group, which will be signaled at the quantization group level. In some embodiments, the chroma QP offsets are explicitly signaled within each quantization group at the time of encoding.

[0089] The selection of the chroma QP offset value is determined by the relative visual complexity or texture between the luma and chroma components. The quantization groups of the image 1600 are assigned different categories 1611-1614 according to their level of texture. For example, areas that appear monotonous across all color components belong to the first category 1611, areas with high texture color components and monotonous luma belong to the second category 1612, areas with monotonous color information and high texture luma belong to the third category 1613, and areas with high texture across all three components belong to the fourth category 1614. In some embodiments, the encoder may create additional subcategories for each category based on motion and intensity. In some embodiments, additional categorization is done taking into account the difference between the two chroma components themselves.

[0090] These categorizations could then be used to specify different sets of saturation QP offsets for different categories. For example, in some embodiments, the first category 1611 (all monotonic) is assigned a zero or negative QP offset. In this scenario, a negative saturation QP offset allows for better allocation of saturation quality in these regions, since the improved saturation quality may be more pronounced due to the characteristics of all color components. For the second category 1612 (monotone luma and highly textured color information), a larger positive QP offset may be used. In this case, a larger saturation QP allows for better control of the saturation components so that they do not overwhelm the luma components, while some texture masking from the saturation information may still be used to ensure a subjectively better quality. For the third category 1613 (highly textured luma but monotone saturation), some embodiments use a larger negative saturation QP offset to help ensure a slightly better color quality. Some other embodiments use a larger positive chroma QP offset to improve the effect of luma texture masking. For the fourth category 1614 (high texture across all three components), some embodiments use a zero or positive chroma QP offset to improve the effect of luma texture masking and avoid wasting bits on an insignificant improvement in the subjective quality of chroma. Note that the exact chroma QP offset value assigned to a quantization group depends on the corresponding luma QP value for that quantization group. Other decisions could be made depending on the subjective and objective quality, or the bitrate decision mechanism in other encoders.

[0091] In some embodiments, if a limited number of chroma QP offsets are permitted or desired (apart from determining the regions and their desired QP offset sets, given the increased bit overhead that their specification may require), then an additional decision is made based not only on the occurrence of each offset set but also on its perceived subjective or objective quality impact.

[0092] Since only the most significant offset sets are signaled at this time, in some embodiments these criteria are then used to "prune" the number of chroma QP offset sets that are likely to be signaled in the bitstream. Regions that fall into the "prune" category may still be assigned an offset set that is close enough to the desired value. Given some subjective or objective weighting process, this is a compromise that may be made whereby the specified chroma QP offsets are "sub-optimal" for all individual grouped region types, but are likely to yield the best overall / aggregate performance given the bitrate limitations.

[0093] For real-time applications where pre-analysis is not possible, some embodiments use a "pre-fixed (pre-fixed) decision" of the QP offset. In some embodiments, the decision is made based on the analysis of data from previous pictures already coded. In some embodiments where the chroma QP offset is explicitly signaled for every quantization group, the decision is made on the fly or dynamically for every actively coded block based not only on its selected luma quantization parameter and its various spatio-temporal and color characteristics, but also on coding information from past blocks (i.e., what was the bitrate or the relationship between luma and chroma, or what was the introduced distortion of other similar or neighboring blocks coded previously). In some embodiments, the luma quantization parameter is derived together with the chroma QP offset, given the same type of bitrate and content characteristic relationships and conditions.

[0094] Some embodiments further improve the selection of the chroma QP offset for the current picture by examining its neighboring pictures. Specifically, some embodiments examine not only how these neighboring pictures were or will be coded, but also how these neighboring pictures are temporally related to the current picture and its regions. For example, if the current picture is to be coded as a "key" picture (e.g., as an intra or "periodic" refresh picture), some embodiments will code an additional chroma QP offset to improve the chroma quality of the current picture. Conversely, if the current picture is a disposable picture or a picture of less importance in the coding hierarchy, some embodiments will refrain from assigning an additional chroma QP offset in a way that would result in a higher bitrate (or will not use an additional chroma QP offset at all). For other types of pictures, some embodiments use a more gentle change in the chroma QP offset to achieve a better compromise between bitrate and quality.

[0095] In some embodiments, different chroma QP offset parameters are specified for layers of different scalability, e.g., resolution, quality, bit depth, etc. Some embodiments apply additional chroma QP offsets to 3D / multi-view applications where different chroma QP offsets are assigned to different views. For example, stereo masking could be considered to assign and predict chroma QP offsets to reduce overhead and maximize subjective quality of such systems.

[0096] For some embodiments, Figure 17 conceptually illustrates a process 1700 for analyzing various regions of an image and assigning chroma QP offsets accordingly, which in some embodiments is performed by a video encoder.

[0097] The process begins when a video picture is received (at 1710). The video picture may be a raw image in an uncompressed video stream or a decoded picture from a compressed video bitstream. The process then identifies (at 1720) regions within the picture that share common characteristics that allow coding units within the region to share a common set of chroma QP offset values. For example, some embodiments identify regions originally coded in 4:2:0 format as regions where higher QP values ​​(and thus positive chroma QP offsets) may be used to reduce bitrate. Conversely, some embodiments identify regions originally coded in 4:4:4 format as regions where lower QP values ​​(and thus lower or negative QP offset values) are required to maintain quality.

[0098] The process then (at 1730) analyzes the spatio-temporal characteristics of each region. In some embodiments, this analysis includes analyzing the region's texture / variance / activity, format (such as 4:4:4 or 4:2:0), noise, motion, bit depth, or other characteristics that may affect the relationship between luma and chroma or the relationship between the two chroma components.

[0099] The process then assigns (at 1740) a chroma QP offset value based on the analysis of the region. In some embodiments, the process first identifies an appropriate chroma QP value for the region based on the analysis performed at 1730, and then subtracts the selected luma QP value from the identified chroma QP value to determine a desired overall chroma QP offset value. Process 1740 also, in some embodiments, decomposes the overall chroma QP offset value into chroma QP offset values ​​at various levels of the video coding hierarchy (e.g., picture, slice, and quantization group). Some embodiments identify a chroma QP offset value for a quantization group by subtracting a chroma QP offset value for a higher level syntax element from the overall chroma QP offset value. In some embodiments, this operation is performed by a rate controller, as described with reference to FIG. 18 below.

[0100] The process then populates (at 1750) a chroma QP offset table with the identified chroma QP offset values ​​for the quantization groups in the region. The process 1700 also encodes the corresponding index values ​​in the quantization groups, as discussed in section Ib above. In some embodiments that explicitly specify chroma QP offset values, the process 1700 encodes the chroma QP offset values ​​within the quantization groups themselves, as discussed in section Ia above.

[0101] The process then determines (at 1760) whether there are any more regions in the picture that have not yet been analyzed. If so, the process returns to 1720. If not, the process 1700 ends.

[0102] III.Video System 18 illustrates a general video encoder 1800 (e.g., an HEVC encoder) for some embodiments of the present invention. The encoder 1800 receives a video stream from a video source 1805 and produces a compressed and encoded bitstream 1895 that is to be stored and / or transmitted.

[0103] The video encoder includes a transform module 1810, a quantization module 1815, an entropy encoder 1820, an inverse quantization module 1825, an inverse transform module 1830, a deblocking filter 1840, a sample adaptive offset (SAO) filter 1845, a frame buffer 1850, a rate control module 1835, and a prediction module 1890. The prediction module 1890 includes a motion prediction module 1860, a motion compensation module 1865, an intra prediction module 1870, and a mode decision module 1880. The video encoder 1800 also includes a video display 1855 in some embodiments.

[0104] The quantization module 1815 is a module that performs quantization on the transform coefficients (e.g., DCT) from the transform module 1810 using a quantization parameter. In some embodiments, the transform module 1810 may be completely bypassed (e.g., under a transform-bypass mode supported by HEVC), such that the quantization module 1815 receives image values ​​and image prediction error values ​​without transformation. The quantization module 1815 applies different QP values ​​for different regions / blocks for each color component. The QP values ​​used by the quantization module 1815 are also coded into the bitstream 1895 as luma QP values ​​and chroma QP offsets. In some embodiments, the QP values ​​used by the quantization module 1815 are determined and provided by the rate control module 1835.

[0105] The rate control module 1835 controls the bitrate of the encoded video bitstream by controlling the QP values ​​used by the quantization module 1815 (and the inverse quantization module 1825). In some embodiments, the rate control module 1835 provides different QP values ​​to the quantization module 1815 for different quantization groups. To identify the best QP values ​​(for luma and chroma) for quantization (e.g., the best balance between quality and bitrate for a given video sequence), the rate control module 1835 of some embodiments performs at least some of the analysis described in Section II above to arrive at a luma QP value for each luma quantization group and a set of chroma QP offset values ​​for each chroma quantization group. Also, in some embodiments, the rate controller 1835 uses the analysis to identify regions and assign (luma and chroma) quantization groups.

[0106] In some embodiments, the rate controller 1835 decomposes the set of chroma QP offset values ​​into sets of chroma QP offset values ​​at various levels of the video coding hierarchy. In some embodiments, at least some of the set of chroma QP offset values ​​are further decomposed into predictions between two chroma components as shown in equations (1)-(7) above. In some embodiments, the QP offset values ​​of lower levels are identified by subtracting the QP offset values ​​of higher levels from the overall chroma QP offset value. For example, in some embodiments, the chroma QP offsets of the quantization group levels are calculated as follows: QPoffset_quant_group[i]=QPchroma[i]-QPluma -QPoffset_pps[i]-QPoffset_slice[i] (8)

[0107] These sets of chroma QP offset values ​​are then provided to the entropy encoder 1820 to be encoded into the bitstream 1895. In some embodiments, the rate controller 1835 compiles a table of possible quantization group level offsets and provides the compiled table to the entropy encoder 1820.

[0108] The entropy encoder module 1820 performs entropy encoding (e.g., CABAC) on the quantized transform coefficients, parameters, and other information and packages them into the bitstream 1895. In some embodiments, the entropy encoder module 1820 receives chroma QP offset values ​​from the rate controller module 1835 and entropy encodes them into the bitstream. In some embodiments, the entropy encoder 1820 encodes the chroma QP offset values ​​as a table into the PPS area of ​​the picture, as described above in section Ib. In some embodiments, the entropy encoder encodes the chroma QP offset values ​​into individual quantization groups, as described above in section Ia.

[0109] The deblocking module 1840 is a loop filter module that improves the visual quality and prediction performance of a video codec system by smoothing sharp edges formed between coding blocks. In some embodiments, the deblocking module 1840 performs its deblocking operation using an overall chroma QP value calculated from all applicable chroma QP offsets (i.e., by including quantization group and picture / slice chroma QP offsets). In some other embodiments, the deblocking module 1840 considers only some specified chroma QP offsets, or only considers chroma QP offsets from the PPS or slice header. Some embodiments control the deblocking process based on a signal chroma QP offset value. For example, some embodiments adjust deblocking parameters to compensate for the chroma QP offset value. Deblocking may also be performed outside the loop as a post-process. In some embodiments, the calculated overall chroma QP value or some chroma QP offset is used by other types of in-loop or out-of-loop post-processes, such as SAO (e.g., SAO filter 1845), an adaptive loop filter (ALF), or noise addition.

[0110] 19 illustrates a general video decoder 1900 (e.g., an HEVC decoder) for some embodiments of the present invention. The decoder 1900 receives a bitstream 1905, decodes it, and displays it via a display module 1965. The decoder 1900 includes an entropy decoder 1910, an inverse quantization module 1920, an inverse transform module 1930, an intra prediction module 1950, an inter prediction module 1980, a deblocking module 1960, an SAO module 1970, and a frame buffer 1975.

[0111] The entropy decoder module 1910 is a module that performs entropy decoding on the input bitstream to extract transform coefficients and parameters for other modules in the decoder 1900. In particular, chroma quantization information stored in the input bitstream 1905 is extracted by the entropy decoder 1910 and sent to the inverse quantization module 1920. In some embodiments, the chroma quantization information includes an additional chroma QP offset value extracted from the PPS, a table in the slice header, or from the quantization group itself.

[0112] The deblocking module 1960 of the decoder 1900 performs a similar function to the deblocking module 1840 of the encoder 1800. Specifically, the deblocking module 1960 also performs a deblocking operation using a final chroma QP value calculated from all applicable chroma QP offsets (i.e., by including quantization group and picture / slice chroma QP offsets). In some other embodiments, the deblocking module 1960 considers only a portion of the specified chroma QP offsets, or only considers the chroma QP offsets from the PPS or slice header. In some embodiments, the calculated total chroma QP value or a portion of the chroma QP offsets are used by other types of in-loop or out-of-loop post-processes, such as SAO (i.e., SAO filter 1970), ALF, or noise addition.

[0113] IV. Electronic Systems Many of the features and applications described above are implemented as a software process specified as a set of instructions recorded on a computer-readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more computers or processing units (e.g., one or more processors, processor cores, or other processing units), they cause the processing units to perform the operations indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, random access memory (RAM) chips, hard drives, erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), and the like. Computer-readable media does not include electrical signals passing over carrier waves or wireless or wired connections.

[0114] As used herein, the term "software" is intended to include firmware resident in read-only memory and applications stored in magnetic storage devices that can be loaded into memory for processing by a processor. Also, in some embodiments, multiple software inventions may be implemented as subparts of a larger program while remaining separate software inventions. Also, in some embodiments, multiple software inventions may be implemented as individual programs. Finally, any combination of individual programs that together implement a software invention as described herein is within the scope of the invention. In some embodiments, the software program, when installed to operate on one or more electronic systems, defines one or more specific machine implementations that perform the operations of the software program.

[0115] 20 is a conceptual diagram illustrating an electronic system 2000 in which some embodiments of the present invention are implemented. The electronic system 2000 may be a computer (e.g., a desktop computer, a personal computer, a tablet computer, etc.), a phone, a PDA, or any other type of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 2000 includes a bus 2005, a processing unit 2010, a graphics processing unit (GPU) 2015, a system memory 2020, a network 2025, a read-only memory 2030, a permanent storage device 2035, input devices 2040, and output devices 2045.

[0116] The bus 2005 collectively represents all of the system buses, peripheral buses, and chipset buses that communicatively connect the many internal devices of the electronic system 2000. For example, the bus 2005 communicatively connects the processing unit 2010 to the read-only memory 2030, the GPU 2015, the system memory 2020, and the permanent storage device 2035.

[0117] From these various memory units, the processing unit 2010 retrieves instructions to execute and data to process in order to perform the processes of the present invention. The processing unit may be a single processor or, in another embodiment, a multi-core processor. Some instructions are passed to and executed by the GPU 2015. The GPU 2015 may offload various computations or complement the image processing provided by the processing unit 2010.

[0118] The read-only memory (ROM) 2030 stores static data and instructions required by the processing unit 2010 and other modules of the electronic system. The permanent storage device 2035, on the other hand, is a read-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the electronic system 2000 is turned off. In some embodiments of the invention, a mass storage device (such as a magnetic or optical disk and its corresponding disk drive) is used as the permanent storage device 2035.

[0119] In other embodiments, the permanent storage device is a removable storage device (such as a floppy disk, a flash memory device, etc. and its corresponding disk drive). Like the permanent storage device 2035, the system memory 2020 is a readable and writable memory device. However, unlike the permanent storage device 2035, the system memory 2020 is a volatile readable and writable memory, such as a random access memory. The system memory 2020 stores some instructions and data that the processor needs at runtime. In some embodiments, the process of the present invention is stored in the system memory 2020, the permanent storage device 2035, and / or the read-only memory 2030. For example, various memory units contain instructions for processing a multimedia clip according to some embodiments. From these various memory units, the processing unit 2010 retrieves instructions to execute and data to process in order to execute the process of some embodiments.

[0120] The bus 2005 also connects to input devices 2040 and output devices 2045. The input devices 2040 allow a user to communicate information and select commands to the electronic system. The input devices 2040 include alphanumeric keyboards, pointer devices (also called "cursor control devices"), cameras (e.g., webcams), microphones or similar devices for receiving voice commands, etc. The output devices 2045 display images generated by the electronic system or otherwise output data. The output devices 2045 include printers and display devices, such as cathode ray tubes (CRT) or liquid crystal displays (LCD), as well as speakers or similar audio output devices. Some embodiments include devices such as a touch screen that function as both an input device and an output device.

[0121] 20, bus 2005 connects electronic system 2000 to a network 2025 through a network adapter (not shown). In this manner, the computer may be part of a network of computers (such as a local area network ("LAN"), a wide area network ("WAN"), an intranet, or a network of networks, such as the Internet. Any or all of the components of electronic system 2000 may be used in connection with the present invention.

[0122] Some embodiments include electronic components, such as, for example, a microprocessor, storage devices, and memory that store computer program instructions in a machine-readable or computer-readable medium (also referred to as computer-readable storage medium, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, read-only compact disks (CD-ROMs), write-once compact disks (CD-Rs), re-writable compact disks (CD-RWs), read-only digital versatile disks (e.g., DVD-ROMs, dual-layer DVD-ROMs), various write-once / re-writable DVDs (e.g., DVD-RAMs, DVD-RWs, DVD+RWs, etc.), flash memory (e.g., SD cards, mini SD cards, micro SD cards, etc.), magnetic and / or solid state hard drives, read-only and write-once Blu-Ray® disks, Ultra Density Optical Disks, any other optical or magnetic media, floppy disks, etc. The computer readable medium may store a computer program that is executable by at least one processing unit and includes a set of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as that produced by a compiler, and files containing higher level code that are executed by a computer, electronic component, or microprocessor using an interpreter.

[0123] While the above discussion has primarily referred to microprocessors or multi-core processors executing software, some embodiments are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored within the circuitry itself. In addition, some embodiments execute software stored within a programmable logic device (PLD), ROM or RAM device.

[0124] In this specification and in any claims, the terms "computer", "server", "processor", "memory" all refer to electronic devices or other technological devices. These terms exclude a human or group of humans. In this specification, the terms display or display means refer to displaying on an electronic device. In this specification and in any claims, the terms "computer-readable medium", "computer-readable medium" and "machine-readable medium" are entirely limited to tangible physical objects that store information in a form that can be read by a computer. These terms exclude any wireless signals, wired download signals, and any other ephemeral signals.

[0125] Although the present invention has been described with reference to numerous specific details, those skilled in the art will recognize that the present invention may be embodied in other specific forms without departing from the spirit of the present invention. In addition, several figures (including FIG. 7, FIG. 13, and FIG. 17) conceptually illustrate processes. The specific operations of these processes may not be performed in the exact order shown and described. The specific operations may not be performed as a continuous sequence of operations, and different embodiments may perform different specific operations. Furthermore, processes may be implemented using multiple sub-processes or as part of a larger macro process. Thus, those skilled in the art will recognize that the present invention is not limited to the illustrative details set forth above, but rather is defined by the appended claims.

Claims

1. A non-transitory machine-readable medium having instructions stored thereon that, when executed, are configured to cause one or more processing units to perform operations, including: receiving, in a bitstream, a group of coded video pictures including a coded first video picture having a hierarchical coding structure having at least a first coding level, a second coding level, and a third coding level, the coded first video picture including two or more sets of chroma quantization parameter (QP) offset values ​​at the first coding level and the second coding level, each set of chroma QP offset values ​​providing information regarding a chroma QP of a video unit included in a level of the hierarchical coding structure, the coded first video picture further including one or more chroma quantization groups (QGs) at the third coding level; identifying an additional set of chroma QP offsets for a particular chroma QG of the one or more chroma QGs at the third coding level, the identifying including: determining a cMax parameter to be used in a binarization process in Context-Adaptive Binary Arithmetic Coding (CABAC) based on information included in the first coding level or the second coding level; identifying the additional set of chroma QP offsets using the cMax parameter; and decoding the encoded first video picture by calculating a chroma QP value corresponding to the one or more chroma QGs of the encoded first video picture, the decoding comprising: for the particular chroma QG: selecting the identified additional set of chroma QP offsets; calculating the chroma QP value corresponding to the particular chroma QG using the selected set of chroma QP offsets; and A non-transitory machine-readable medium, comprising:

2. 2. The non-transitory machine-readable medium of claim 1, wherein calculating the chroma QP values ​​further comprises combining the chroma QP offsets in the selected set with one or more luma QP values ​​corresponding to the particular chroma QG. Non-transitory machine-readable media.

3. 3. The non-transitory machine-readable medium of claim 2, wherein at least the particular chroma QG is associated with a plurality of luma QP values, and calculating the chroma QP value for the particular chroma QG includes calculating a plurality of different chroma QP values ​​corresponding to the plurality of luma QP values. Non-transitory machine-readable media.

4. 3. The non-transitory machine-readable medium of claim 2, wherein at least the particular chroma QG is associated with one luma QP value, and calculating the chroma QP value for the particular chroma QG includes calculating one chroma QP value corresponding to the one luma QP value. Non-transitory machine-readable media.

5. 2. The non-transitory machine-readable medium of claim 1 , wherein receiving in the bitstream the encoded first video picture including two or more sets of chroma QP offsets at the first encoding level and the second encoding level includes receiving in the bitstream a first chroma QP offset corresponding to the first encoding level of the encoded first video picture and a second chroma QP offset corresponding to the second encoding level of the encoded first video picture, the operations further comprising: receiving, in the bitstream, a third chroma QP offset corresponding to the third encoding level of the encoded first video picture; A non-transitory machine-readable medium, comprising:

6. 6. The non-transitory machine-readable medium of claim 5, wherein the first and second coding levels are higher in a hierarchy of coding levels in the hierarchical coding structure than the third coding level; Calculating the chroma QP value corresponding to the particular chroma QG includes using the first chroma QP offset. Non-transitory machine-readable media.

7. 7. The non-transitory machine-readable medium of claim 6, wherein calculating the chroma QP values ​​corresponding to the one or more chroma QGs of the encoded first video picture includes calculating chroma QP values ​​for the one or more chroma QGs associated with the third encoding level of the encoded first video picture using the first chroma QP offset. Non-transitory machine-readable media.

8. 7. The non-transitory machine-readable medium of claim 6, wherein the first coding level corresponds to a picture level, the second coding level corresponds to a slice level, and the third coding level corresponds to one of a coding tree unit level, a coding unit level, or a transform unit level; calculating the saturation QP value corresponding to the particular saturation QG includes using a saturation QP offset specified in a picture parameter set (PPS) of the encoded first video picture. Non-transitory machine-readable media.

9. 2. The non-transitory machine-readable medium of claim 1, wherein the operations further comprise: receiving, in the bitstream, one or more indices corresponding to the one or more chroma QGs of the encoded first video picture; selecting, for at least a first chroma QP of the one or more chroma QPs, a chroma QP offset value from the set of chroma QP offset values ​​using an index corresponding to the first chroma QP; A non-transitory machine-readable medium, comprising:

Images