Adaptive offset with respect to sample point and cross-component sample point

Abstract

A mechanism for processing video data is disclosed. The mechanism includes determining a parameter of a sample adaptive offset (SAO) filter for a video unit, where the parameter is selected from a set of explicit signaling notifications, a predefined set, or an adaptive set, and where the adaptive set is derived based on parameters of one or more video units encoded and decoded prior to the video unit. Conversion between the visual media data and the bitstream is performed based on the SAO filter.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of International Patent Application No. PCT / CN2022 / 115132 filed on August 26, 2022. The contents of the foregoing patent application are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure relates to the generation, storage and use of digital audio-visual media information in file format. Background Art

[0004] Digital video consumes the largest amount of bandwidth used on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, bandwidth requirements for digital video usage are likely to continue to grow. Summary of the invention

[0005] A first aspect relates to a method for processing video data, comprising: determining parameters of a Sample Adaptive Offset (SAO) filter; and performing conversion between visual media data and a bitstream based on the SAO filter.

[0006] A second aspect relates to an apparatus for processing video data, comprising: a processor; and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform any of the aforementioned aspects.

[0007] A third aspect relates to a non-transitory computer-readable medium, comprising a computer program product for use with a video codec device, the computer program product comprising computer executable instructions stored on the non-transitory computer-readable medium, so that when the computer program product is executed by a processor, the video codec device performs the method of any of the aforementioned aspects.

[0008] A fourth aspect relates to a non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by a video processing device, wherein the method includes: determining parameters of a sample adaptive offset (SAO) filter; and generating the bitstream based on the determination.

[0009] A fifth aspect relates to a method for storing a bitstream of a video, comprising: determining parameters of a sample adaptive offset (SAO) filter; generating the bitstream based on the determination; and storing the bitstream in a non-transitory computer-readable recording medium.

[0010] A sixth aspect relates to a method, device or system described in this patent document.

[0011] For clarity, any of the foregoing embodiments may be combined with one or more of the other foregoing embodiments to create new embodiments within the scope of the present disclosure.

[0012] These and other features will become more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] For a more complete understanding of the present disclosure, reference is now made to the following brief description taken in conjunction with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.

[0014] Figure 1 An example of nominal vertical and horizontal positions of 4:2:2 luma and chroma samples in a picture is shown.

[0015] Figure 2 An example encoder block diagram is shown.

[0016] Figure 3 Examples of 67 intra prediction modes are shown.

[0017] Figure 4 An example of reference samples for wide-angle intra prediction is shown.

[0018] Figure 5 An example problem of discontinuity where the orientation exceeds forty-five degrees (45°) is shown.

[0019] Figure 6 Examples of filter shapes of an adaptive loop filter (ALF), such as chroma: 5×5 diamond and luma: 7×7 diamond, are shown.

[0020] Figure 7 An example of a sub-sampled Laplacian calculation is shown.

[0021] Fig. 8A and Figure 8B The placement of a cross-component ALF (CC-ALF) relative to other loop filters and an example of a diamond filter are shown, respectively.

[0022] Fig. 9A and Fig. 9B An example of modified block classification at a virtual boundary is shown.

[0023] Fig.10 An example of modified ALF filtering for luma components at a virtual boundary is shown.

[0024] Fig.11 Example sample point locations of pi,k and qi,k are shown.

[0025] Fig.12An example of luma mapping with a chroma scaling architecture is shown.

[0026] Fig.13 Examples of four one-dimensional (1-D) 3-pixel patterns for pixel classification in edge offset (EO) are shown.

[0027] Fig.14 An example is shown where four frequency bands are grouped together and indicated by their starting frequency band positions.

[0028] Fig.15 An example of a 25-tap long filter is shown.

[0029] Fig.16 An example is shown where both the Bilateral Filter (BIF) and the Sample Adaptive Offset (SAO) use the samples from the deblocking stage as input. Both generate offsets and these offsets are added to the input samples and clipped.

[0030] Fig.17 The center sample point I is shown C Example naming convention for surrounding sample points.

[0031] Fig.18 Example filtering stages for BIF-Chroma are shown.

[0032] Fig.19 An example of a modified SAO process when the proposed CCSAO is applied is shown.

[0033] Fig. 20 Examples of candidate positions for the CCSAO classifier are shown.

[0034] Fig.21 An example of joint clipping after adding SAO / BIF / CCSAO offsets to the input samples is shown.

[0035] Fig. 22 Exemplary four 1-D directional modes for CCSAO EO sample classification are shown: horizontal (EO Class = 0), vertical (EO Class = 1), 135 degree diagonal, and 45 degree diagonal.

[0036] Fig.23 is a block diagram illustrating an example video processing system.

[0037] Fig.24 is a block diagram of an example video processing device.

[0038] Fig.25 is a flow chart of an example method for video processing.

[0039] Fig.26 is a block diagram illustrating an example video codec system.

[0040] Fig. 27 is a block diagram illustrating an example encoder.

[0041] Fig.28 is a block diagram illustrating an example decoder.

[0042] Fig.29 is a schematic diagram of an example encoder. DETAILED DESCRIPTION

[0043] First, it should be understood that although illustrative implementations of one or more embodiments are provided below, the disclosed systems and / or methods can be implemented using any number of technologies, whether currently known or yet to be developed. The present disclosure should not be limited in any way to the illustrative implementations, drawings, and techniques shown below, including the exemplary designs and implementations shown and described herein, but can be modified within the full scope of the appended claims and their equivalents.

[0044] The section titles used in this document are intended to facilitate understanding, and do not limit the applicability of the techniques and embodiments disclosed in each section to only that section. In addition, the techniques described herein are applicable to other video codec protocols and designs.

[0045] 1. Preliminary Discussion

[0046] This document relates to video codecs. Specifically, it is about sample adaptive offset (SAO) and cross-component SAO (CCSAO), how and / or whether to signal / derive parameters of SAO and CCSAO, and other codec tools in image / video codecs. These concepts can be applied to video codecs such as High Efficiency Video Codec (HEVC), Versatile Video Codec (VVC), or other video codecs.

[0047] 2. Video Codec Standards

[0048] Video codec standards have evolved primarily through the development of ITU-T and International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) standards. The International Telecommunication Union-Telecommunication Standardization Sector (ITU-T) developed H.261 and H.263, ISO / IEC developed Moving Picture Experts Group (MPEG)-1 and MPEG-4 Vision, and the two organizations jointly developed H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Codec (AVC) and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec structure that utilizes temporal prediction plus transform codec. In order to explore future video codec technologies beyond HEVC, the Joint Video Exploration Team (JVET) was jointly established by the Video Codec Experts Group (VCEG) and MPEG. Many methods have been adopted by JVET and incorporated into the reference software called the Joint Exploration Model (JEM). A Joint Video Experts Team (JVET) was formed between the Video Codec Experts Group (VCEG) (Q6 / 16) and MPEG's ISO / IEC JTC1 SC29 / WG11 to develop the VVC standard, with the goal of a 50% bitrate reduction compared to HEVC.

[0049] 2.1 Color Space and Chroma Subsampling

[0050] A color space, also called a color model (or color system), is a mathematical model that describes a range of colors as a tuple of numbers, for example as 3 or 4 values ​​or color components (e.g., RGB). In general, a color space is a specialization of a coordinate system and a subspace. For video compression, the most commonly used color spaces are luminance, blue difference chrominance, and red difference chrominance (YCbCr) and red, green, blue (RGB).

[0051] YCbCr, Y'CbCr or Y Pb / Cb Pr / Cr, also written as YCBCR or Y'CBCR, is a family of color spaces used as part of the color image processing pipeline in video and digital photography systems. Y' is the luminance component, and CB and CR are the blue difference chrominance components and the red difference chrominance components. Y' (with a prime) is different from Y, which is luminance, meaning that the light intensity is encoded non-linearly based on the gamma-corrected RGB primaries.

[0052] Chroma subsampling is the practice of encoding an image at a lower resolution for chroma information than for luminance information by exploiting the fact that the human visual system is less sensitive to color differences than to luminance. 2.1.1 4:4:4

[0054] In 4:4:4, each of the three components Y'CbCr has the same sampling rate. Therefore, there is no chroma subsampling. This scheme is sometimes used in high-end film scanners and film post-production. 2.1.2 4:2:2

[0056] Figure 1 An example of the nominal vertical and horizontal positions of 4:2:2 luma and chroma samples in a picture is shown. In 4:2:2, the two chroma components are sampled at half the sampling rate of luma. The horizontal chroma resolution is halved, while the vertical chroma resolution is unchanged. This reduces the bandwidth of the uncompressed video signal by one third, but there is little visual difference. Figure 1 Examples of nominal vertical and horizontal positions for a 4:2:2 color format are depicted in . 2.1.3 4:2:0

[0058] In 4:2:0, the horizontal sampling is doubled compared to 4:1:1, but since in this scheme the Cb and Cr channels are sampled only on every alternate line, the vertical resolution is halved while the data rate remains the same. Cb and Cr are each subsampled by a factor of 2 in both the horizontal and vertical directions. There are three variations of the 4:2:0 scheme with different horizontal and vertical addressing. In MPEG-2, Cb and Cr are co-located horizontally. Cb and Cr are located between pixels vertically (in the gaps). In JPEG / JFIF, H.261, and MPEG-1, Cb and Cr are located in the gaps, in the middle of alternate luma samples. In 4:2:0DV, Cb and Cr are co-located horizontally. Vertically, Cb and Cr are co-located on alternate lines.

[0059] chroma_format_idc separate_colour_plane_flag Chroma Format SubWidthC SubHeightC 0 0 monochrome 1 1 1 0 4:2:0 2 2 2 0 4:2:2 2 1 3 0 4:4:4 1 1 3 1 4:4:4 1 1

[0060] Table 1. SubWidthC and SubHeightC values ​​derived from chroma_format_idc and separate_colour_plane_flag

[0061] 2.2 Example Encoding and Decoding Process of Video Codec

[0062] Figure 2 An example encoder block diagram is shown. Figure 2An example of an encoder block diagram of VVC is shown, which contains three loop filtering modules: deblocking filter (DF), sample adaptive offset (SAO) and ALF. Unlike DF, which uses a predefined filter, SAO and ALF use the original samples of the current picture to reduce the mean square error between the original samples and the reconstructed samples by adding an offset and applying a finite impulse response (FIR) filter, respectively, where the offset and filter coefficients are signaled by the side information of the codec. ALF is located at the last processing stage of each picture and can be seen as a tool that tries to capture and repair artifacts produced by previous stages.

[0063] 2.3 Intra-mode codec with 67 intra-prediction modes

[0064] Figure 3 An example of 67 intra prediction modes is shown. To capture arbitrary edge directions present in natural video, the number of directional intra modes is extended from 33 as used in HEVC to 65. Additional directional modes are given in Figure 3 , and planar and DC modes remain unchanged. These more dense directional intra prediction modes apply to all block sizes and both luma and chroma intra prediction.

[0065] In HEVC, each intra-coded block has a square shape, and the length of each of the sides of the block is a power of 2. Therefore, no division operation is needed to generate the intra predictor using the DC mode. In VVC, blocks can have a rectangular shape, and in general a division operation must be used for each block. To avoid the division operation for DC prediction, for non-square blocks, only the longer sides are used to calculate the average.

[0066] 2.3.1 Wide-angle intra prediction

[0067] Although 67 modes are defined in VVC, the exact prediction direction for a given intra prediction mode index also depends on the block shape. In some examples, the angular intra prediction direction is defined to be from 45 degrees to -135 degrees in a clockwise direction. In VVC, for non-square blocks, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes. The replaced modes are signaled using the original mode indexes, which are remapped to the indexes of the wide-angle mode after parsing. The total number of intra prediction modes remains unchanged, i.e. 67, and the intra mode encoding and decoding method remains unchanged.

[0068] Figure 4 An example of reference samples for wide-angle intra prediction is shown. To support these prediction directions, Figure 4As shown, a top reference of length 2W+1 and a left reference of length 2H+1 are defined. The number of modes replaced in the wide-angle direction mode depends on the aspect ratio of the block. The replaced intra-frame prediction modes are shown in Table 2.

[0069]

[0070]

[0071] Table 2. Intra prediction modes replaced by wide angle mode

[0072] Figure 5 shows an example problem of discontinuity in the case of orientations exceeding 45°. Figure 5 As shown in FIG. 1 , in the case of wide-angle intra prediction, two vertically adjacent prediction samples can use two non-adjacent reference samples. Therefore, a low-pass reference sample filter and edge smoothing are applied to wide-angle prediction to reduce the increased gap Δp. α If the wide-angle mode represents a non-fractional offset, there are 8 modes in the wide-angle mode that meet this condition, which are [-14, -12, -10, -6, 72, 76, 78, 80]. When predicting a block through these modes, the samples in the reference buffer are directly copied without applying any interpolation. With this modification, the number of samples required for smoothing is reduced. In addition, this is consistent with the design of the non-fractional mode and the wide-angle mode in the general prediction mode set.

[0073] In VVC, 4:2:2 and 4:4:4 as well as 4:2:0 chroma formats are supported. The chroma derivation mode (DM) derivation table for the 4:2:2 chroma format is ported from HEVC, expanding the number of entries from 35 to 67 to keep in line with the extension of intra prediction modes. Since the HEVC specification does not support prediction angles below -135 degrees and above 45 degrees, the luma intra prediction modes ranging from 2 to 5 are mapped to 2. Therefore, the chroma DM derivation table for the 4:2:2 chroma format is updated by replacing the values ​​of some entries of the mapping table to more accurately convert the prediction angles of the chroma blocks.

[0074] 2.4 Inter-frame prediction

[0075] For each inter-predicted codec unit (CU), the motion parameters include motion vectors, reference picture indices, reference picture list usage indices, and additional information required by the new codec features of VVC for sample generation for inter-prediction. Motion parameters can be signaled explicitly or implicitly. When a CU is coded in skip mode, the CU is associated with one prediction unit (PU) and has no significant residual coefficients, no coded motion vector increments and / or reference picture indices. Merge mode is defined as a mode by which motion parameters for the current CU are obtained from neighboring CUs, including spatial and temporal candidates, and additional schedules introduced in VVC. Merge mode can be applied to any inter-predicted CU, not just skip mode. An alternative to Merge mode is to explicitly send motion parameters, where for each CU, motion vectors, corresponding reference picture indices for each reference picture list, reference picture list usage flags, and other useful information are explicitly signaled.

[0076] 2.5 Intra-block copy (IBC)

[0077] Intra-block copying (IBC) is a tool used for screen content coding (SCC) in the HEVC extension. This significantly improves the coding efficiency of screen content materials. Since the IBC mode is implemented as a block-level coding mode, block matching (BM) is performed at the encoder to find the optimal block vector (or motion vector) for each CU. Here, the block vector is used to indicate the displacement from the current block to the reference block that has been reconstructed in the current picture. The luminance block vector of the CU encoded and decoded by IBC is integer precision. The chrominance block vector is also rounded to integer precision. When combined with adaptive motion vector resolution (AMVR), the IBC mode can switch between 1-pel and 4-pixel motion vector precision. The CU encoded and decoded by IBC is regarded as a third prediction mode in addition to the intra prediction mode or the inter prediction mode. The IBC mode is applicable to CUs whose width and height are both less than or equal to 64 luminance samples.

[0078] On the encoder side, hash-based motion estimation is performed for IBC. The encoder performs rate-distortion (RD) checks for blocks with a width or height no greater than 16 luma samples. For non-Merge mode, a block vector search is first performed using a hash-based search. If the hash search does not return a valid candidate, a local search based on block matching is performed.

[0079] In hash-based search, the hash key matching (32-bit cyclic redundancy check (CRC)) between the current block and the reference blocks is extended to all allowed block sizes. The hash key calculation for each position in the current picture is based on 4×4 sub-blocks. For larger-sized current blocks, the hash key is determined to match the hash key of the reference block when all hash keys of all 4×4 sub-blocks match the hash key in the corresponding reference position. If multiple reference blocks are found to match the hash key of the current block, the block vector cost of each matching reference is calculated and the one with the smallest cost is selected.

[0080] In the block matching search, the search range is set to cover both the previous and current codec tree units (CTUs). At the CU level, the IBC mode is signaled with a flag and it can be signaled as the following IBC adaptive motion vector prediction (AMVP) mode or IBC skip / merge mode. IBC skip / merge mode: The Merge candidate index is used to indicate which block vector from the list of neighboring candidate IBC codec blocks is used to predict the current block. The Merge list consists of spatial, history-based motion vector prediction (HMVP) and paired candidates. IBC AMVP mode: Block vector differences are encoded and decoded in the same way as motion vector differences. The block vector prediction method uses two candidates as predictors, one from the left neighboring block and one from the upper neighboring block (if IBC codec is used). When these two neighboring blocks are not available, the default block vector will be used as the predictor. The flag is signaled to indicate the block vector predictor index.

[0081] 2.6 Adaptive Loop Filter

[0082] In VVC, ALF with block-based filter adaptation is applied. For the luma component, one of 25 filters is selected for each 4×4 block based on the direction and activity of the local gradient.

[0083] 2.6.1 Filter shape

[0084] In JEM, up to three diamond filter shapes can be selected for the luminance component (e.g. Figure 6 ). The index is signaled at the picture level to indicate the filter shape used for the luma component. Each square represents a sample, and Ci (i is 0 to 6 (left), 0 to 12 (center), 0 to 20 (right)) represents the coefficient to be applied to that sample. In some examples, a 5×5 diamond shape is always used for the chroma components in a picture. Figure 6 Examples of ALF filter shapes are shown, such as chroma: 5x5 diamond and luma: 7x7 diamond. Figure 6 Example ALF filter shapes are included, including chroma: 5×5 diamond and luma: 7×7 diamond. Using two diamond filter shapes (such as Figure 6 ). The luma component applies a 7×7 diamond shape, and the chroma components apply a 5×5 diamond shape.

[0085] 2.6.2 Block Classification

[0086] For the luma component, each 4×4 block is classified into one of 25 classes. The class index C is based on the quantized value of its directionality D and activity The derivation is as follows:

[0087]

[0088] In order to calculate D and First, use the 1-D Laplacian operator to calculate the horizontal gradient, vertical gradient, and gradients in the two diagonal directions:

[0089]

[0090]

[0091] Where the indices i and j refer to the coordinates of the upper left sample point within the 4×4 block, and R(i,j) indicates the reconstructed sample point at coordinate (i,j).

[0092] Figure 7 An example of sub-sampled Laplacian calculation is shown. In order to reduce the complexity of block classification, sub-sampled 1-D Laplacian calculation is applied. Figure 7 As shown, the gradient calculations in all directions use the same subsampling position. This includes the subsampling positions of the vertical gradient, horizontal gradient, first diagonal gradient, and second diagonal gradient. Figure 7 They are represented as v, h, 01 and 02 respectively.

[0093] Then, the maximum and minimum values ​​of D in the horizontal and vertical directions are set as:

[0094]

[0095] The maximum and minimum values ​​of the gradients in the two diagonal directions are set to:

[0096]

[0097] To derive the value of the directionality D, these values ​​are compared with each other and with two thresholds t1 and t2:

[0098] Step 1. If and is true, D is set to 0.

[0099] Step 2. If Then continue from step 3, otherwise continue from step 4.

[0100] Step 3. If Then D is set to 2; otherwise D is set to 1.

[0101] Step 4. If Then D is set to 4; otherwise D is set to 3.

[0102] The activity value A is calculated as:

[0103]

[0104] A is further quantized to the range 0 to 4, inclusive, and the quantized value is represented as For the chrominance components of the picture, no classification method is applied.

[0105] 2.6.3 Geometric transformation of filter coefficients and limit values

[0106] Before filtering each 4×4 luma block, geometric transformations such as rotations or diagonal and vertical flipping are applied to the filter coefficients f(k,l) and the corresponding filter clipping values ​​c(k,l), depending on the gradient values ​​calculated for that block. This is equivalent to applying these transformations to the samples in the filter support region. The idea is to make different blocks to which the ALF is applied more similar by aligning their directionality.

[0107] Introduce three geometric transformations, including diagonal, vertical flip and rotation:

[0108] Diagonal: f D (k,l)=f(l,k),c D (k,l)=c(l,k), (2-9)

[0109] Flip vertically: f V (k,l)=f(k,Kl-1),c V (k,l)=c(k,Kl-1) (2-10)

[0110] Rotation: f R (k,l)=f(Kl-1,k),c R (k,l)=c(Kl-1,k) (2-11)

[0111] Where K is the size of the filter and 0≤k,l≤K-1 are the coefficient coordinates such that position (0,0) is at the upper left corner and position (K-1,K-1) is at the lower right corner. The transform is applied to the filter coefficients f(k,l) and the clipping values ​​c(k,l) according to the gradient values ​​calculated for the block. The following table summarizes the relationship between the transform and the four gradients in the four directions.

[0112] Gradient Value Transform <![CDATA[g d2 <g d1 And g h <g v ]]> No transformation <![CDATA[g d2 <g d1 And g v <g h ]]> diagonal <![CDATA[g d1 <g d2 And g h <g v ]]> Flip Vertically <![CDATA[g d1 <g d2 And g v <g h ]]> Rotation

[0113] Table 3. Mapping of gradients and transformations computed for a block

[0114] 2.6.4 Filtering process

[0115] At the decoder side, when ALF is enabled for a CTB, each sample R(i,j) within the CU is filtered to obtain the sample value R′(i,j) as shown below:

[0116] R′(i,j)=R(i,j)+((∑ k≠0 ∑ l≠0 f(k,l)×K(R(i+k,j+l)-R(i,j),c(k,l))+64)>>7)

[0117] (2-12)

[0118] Where f(k,l) represents the decoded filter coefficients, K(x,u) is the clipping function, and c(k,l) represents the decoded clipping parameters. The variables k and l are in and where L is the filter length. The clipping function K(x,y)=min(y,max(-y,x)), which corresponds to the function Clip3(-y,y,x). The clipping operation introduces nonlinearity, making the ALF more effective by reducing the impact of adjacent sample values ​​that differ too much from the current sample value.

[0119] 2.6.5 Cross-Component Adaptive Loop Filter

[0120] CC-ALF refines each chroma component using luma sample values ​​by applying an adaptive linear filter to the luma channel and then using the output of this filtering operation for chroma refinement. Fig. 8A The placement of the CC-ALF relative to the other loop filters and an example of a diamond filter are shown. Fig. 8A A system-level diagram of the CC-ALF process relative to the SAO, luma ALF, and chroma ALF processes is provided. Filtering in CC-ALF is achieved by applying a linear diamond filter to the luma channel, as Figure 8B As shown. One filter is used for each chroma channel, and the operation is expressed as:

[0121]

[0122] Where (x, y) is the position of the chrominance component i to be refined, (x Y ,y Y ) is the brightness position based on (x,y), S iis the filter support area in the luminance component, c i (x0, y0) represents the filter coefficients. As shown, the luma filter support is the area co-located with the current chroma sample after taking into account the spatial scaling factor between the luma plane and the chroma plane.

[0123] In the VVC reference software, the CC-ALF filter coefficients are calculated by minimizing the mean squared error of each chroma channel relative to the original chroma content. To achieve this, the VVC Test Model (VTM) algorithm uses a coefficient derivation process similar to that used for chroma ALF. Specifically, a correlation matrix is ​​derived and the coefficients are calculated using a Cholesky decomposition solver in an attempt to minimize the mean squared error metric. When designing the filters, up to eight CC-ALF filters can be designed and transmitted per picture. The resulting filters are then indicated for each of the two chroma channels according to the CTU.

[0124] Additional features of CC-ALF include the following. The design uses a 3×4 diamond shape with 8 taps. The seven filter coefficients are transmitted in an adaptive parameter set (APS). Each transmitted coefficient has a 6-bit dynamic range and is limited to power-of-2 values. The eighth filter coefficient is derived at the decoder so that the sum of the filter coefficients is equal to 0. The APS can be referenced in the slice header. CC-ALF filter selection is controlled at the CTU level for each chroma component. Border padding for horizontal virtual boundaries uses the same memory access pattern as luma ALF.

[0125] As an additional feature, the reference encoder can be configured to enable some basic subjective adjustments via a configuration file. When enabled, VTM attenuates the application of CC-ALF in areas that are encoded with high quantization parameters (QP) and are either close to mid-grey or contain a lot of luma high frequencies. Algorithmically, this is achieved by disabling the application of CC-ALF in a CTU when any of the following conditions are true: the slice QP value minus 1 is less than or equal to the base QP value; the number of chroma samples with local contrast greater than (1<<(bitDepth-2))-1 exceeds the CTU height, where local contrast is the difference between the maximum and minimum luma sample values ​​within the filter support area; or more than one-quarter of the chroma samples are in the range between (1<<(bitDepth–1))-16 and (1<<<(bitDepth–2))+16.

[0126] The motivation for this feature is to ensure, to some extent, that CC-ALF does not amplify artifacts introduced earlier in the decoding path, such as due to the fact that VTM does not explicitly optimize chroma subjective quality. Alternative encoder implementations may not use this feature, or may incorporate alternative strategies appropriate to their coding characteristics.

[0127] 2.6.6 Filter parameter signaling

[0128] The ALF filter parameters are signaled in the APS. In one APS, up to 25 groups of luminance filter coefficients and clipping value indices can be signaled, and up to eight groups of chrominance filter coefficients and clipping value indices can be signaled. In order to reduce the bit overhead, filter coefficients of different classifications of the luminance component can be merged. In the slice header, the index of the APS for the current slice is signaled. The clipping value index decoded from the APS allows the determination of the clipping values ​​using a table of clipping values ​​for both luminance and chrominance components. These clipping values ​​depend on the internal bit depth. More precisely, the clipping values ​​are obtained by the following formula:

[0129] AlfClip = {round(2 B-α*n )for n∈[0..N-1]} (2-14)

[0130] Where B is equal to the internal bit depth, α is a predefined constant value equal to 2.35, and N is equal to 4, the number of clipping values ​​allowed in VVC. AlfClip is then rounded to the nearest value in the power of 2 format.

[0131] In the slice header, up to 7 APS indices can be signaled to specify the luma filter set for the current slice. The filtering process can be further controlled at the CTB level. A flag is always signaled to indicate whether the ALF is applied to the luma CTB. The luma CTB can select a filter set from 16 fixed filter sets and the filter set from the APS. The filter set index is signaled for the luma CTB to indicate which filter set is applied. The 16 fixed filter sets are predefined and hard-coded in both the encoder and the decoder.

[0132] For chroma components, the APS index is signaled in the slice header to indicate the chroma filter set being used by the current slice. At the CTB level, if there is more than one chroma filter set in the APS, the filter index is signaled for each chroma CTB. The filter coefficients are quantized with a norm equal to 128. In order to limit the multiplication complexity, bitstream consistency is applied so that the coefficient values ​​in non-center positions should be in the range of -27 to 27-1, inclusive. The center position coefficients are not signaled in the bitstream and are considered to be equal to 128.

[0133] 2.6.7 Virtual Boundary Filtering Process for Reducing Line Buffer

[0134] Fig. 9A and Fig. 9BAn example of modified block classification at a virtual boundary is shown. In VVC, in order to reduce the line buffer requirements of the ALF, modified block classification and filtering is applied to samples near horizontal CTU boundaries. To this end, Fig. 9A and Fig. 9B As shown, the virtual boundary is defined as a line determined by shifting the horizontal CTU boundary by N samples, where N is equal to 4 for the luma component and N is 2 for the chroma component.

[0135] like Fig. 9A and Fig. 9B As depicted in , a modified block classification is applied to the luminance component. For the 1D Laplacian gradient calculation of the 4×4 block above the virtual boundary (as Fig. 9A ), only the samples above the virtual boundary are used. Similarly, the 1D Laplacian gradient calculation for the 4×4 block below the virtual boundary (as shown in Fig. 9B ), only the samples below the virtual boundary are used. Therefore, by taking into account the reduction in the number of samples used in the 1D Laplacian gradient calculation, the quantization of the activity value A is scaled accordingly.

[0136] Fig.10 An example of modified ALF filtering for the luma component at a virtual boundary is shown. For the filtering process, a symmetric padding operation at the virtual boundary is used for both luma and chroma components. Fig.10 As shown, when the filtered sample is located below the virtual boundary, the adjacent sample located above the virtual boundary is filled in. In addition, the corresponding sample on the other side is also filled symmetrically.

[0137] Unlike the symmetrical padding method used at horizontal CTU boundaries, a simple padding process is applied to slice, slice, and sub-picture boundaries when cross-border filtering is disabled. The simple padding process is also applied to picture boundaries. The padded samples are used for both classification and filtering processes. To compensate for the extreme padding when filtering samples just above or below the virtual boundary, for both luma and chroma, the filter strength is reduced in these cases by increasing the right shift in equation 2-12 by 3.

[0138] 2.7 Deblocking Filter

[0139] Deblocking filtering is an example loop filter in a video codec. In VVC, the deblocking filtering process is applied to CU boundaries, transform subblock boundaries, and predicted subblock boundaries. Prediction subblock boundaries include prediction unit boundaries introduced by subblock-based temporal motion vector prediction (SbTMVP) and affine mode. Transform subblock boundaries include transform unit boundaries introduced by subblock transform (SBT) and intra sub-partitioning (ISP) mode, as well as transforms due to implicit partitioning of large CUs. The processing order of the deblocking filter is defined as first filtering the vertical edges of the entire picture horizontally, and then filtering the horizontal edges vertically. This specific order enables the application of multiple horizontal filtering or vertical filtering processes in parallel threads. The filtering process can also be implemented CTB by CTB with little processing delay.

[0140] Compared to HEVC deblocking, the following modifications are introduced. The filter strength of the deblocking filter depends on the average luma level of the reconstructed samples. Further changes include deblocking tC table extension and adaptation for 10-bit video, 4×4 grid deblocking for luma, stronger deblocking filter for luma, stronger deblocking filter for chroma, deblocking filter at sub-block boundaries, and deblocking decisions that adapt to small motion differences.

[0141] 2.7.1 Luminance Adaptive Deblocking Filter Strength

[0142] As done in HEVC, the filter strength of the deblocking filter in VVC is controlled by variables β and tC, which are derived from the average quantization parameter qPL of two adjacent codec blocks. In VVC, luma-adaptive deblocking can further adjust the filter strength of the deblocking filter based on the average luma level of the reconstructed samples. This additional improvement is used to compensate for nonlinear transfer functions such as the electro-optical transfer function (EOTF) in the linear optical domain.

[0143] Fig.11 Example sample locations of pi,k and qi,k are shown. In this method, the strength of the deblocking filter is controlled by adding an offset to qPL according to the luminance level of the reconstructed sample. The reconstructed luminance level LL is derived as follows:

[0144] LL=((p 0,0 +p 0,3 +q 0,0 +q 0,3 )>>2) / (1< <bitDepth) (2-15)

[0145] Among them, Fig.11 As shown, sample point values ​​pi,k and qi,k are derived, where i=0..3 and k=0 and 3.

[0146] The variable qPL is derived as follows:

[0147] qPL=((QpQ+QpP+1)>>1)+qpOffset (2-16)

[0148] Where QpQ and QpP denote the quantization parameters of the codec unit containing samples q0,0 and p0,0 respectively. The offset qpOffset depends on the transfer function and the reconstructed luma level LL. The mapping function and luma level of qpOffset are signaled in the SPS and should be derived from the transfer characteristics of the content, since the transfer function is different between video formats.

[0149] 2.7.2 TC Table Extension and Adaptation for 10-bit Video Deblocking

[0150] In VVC, the maximum QP changes from 51 to 63, and we want to reflect the corresponding changes in the deblocking table, which derives the value of the deblocking parameter tC based on the block QP. The table is also for 10-bit video, not 8-bit video like HEVC. The following is the updated tC table to accommodate the expansion of the QP range and 10-bit video.

[0151] tC=[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,4,4,4,4,5,5,5,5,7,7,8,9,10,10,11,13,14,15,17,19,21 ,24,25,29,33,36,41,45,51,57,64,71,80,89,100,112,125,141,157,177,198,222,250,280,314,352,395]

[0154] 2.7.3 4×4 Deblocking Grid for Luminance

[0155] HEVC uses an 8×8 deblocking grid for both luma and chroma. In VVC, a 4×4 grid deblocking for luma boundaries is introduced to handle blocking artifacts from rectangular transform shapes. Parallel-friendly luma deblocking is achieved on a 4×4 grid by limiting the number of samples to be deblocked to 1 sample per side of a vertical luma boundary, where the width of a side is 4 or less, or to 1 sample per side of a horizontal luma boundary, where the height of a side is 4 or less.

[0156] 2.7.4 Stronger Deblocking Filter for Luma

[0157] When samples on either side of the boundary belong to a large block, a bilinear filter (e.g., a stronger deblocking filter) is used. Samples are defined as belonging to a large block when the width of the vertical edge is greater than or equal to 32 and when the height of the horizontal edge is greater than or equal to 32. Then, the block boundary samples pi (i=0 to Sp-1) and qi (j=0 to Sq-1) are replaced by linear interpolation as follows:

[0158] p i ′=(f i *Middle s,t +(64-f i )*P s +32)>>6), clipped to p i ±tcPD i (2-17)

[0159] q j ′=(g j *Middle s,t +(64-g j )*Q s +32)>>6), clipped to q j ±tcPD j (2-18)

[0160] Among them, tcPD i and tcPD j The term is the position-dependent clipping described above, and g j 、f i 、Middle s,t , P s and Q s Given as follows:

[0161]

[0162]

[0163]

[0164] Table 4 Derivation of stronger deblocking parameters for luminance

[0165] The stronger luminance filter mentioned above is used only when conditions 1, 2, and 3 are all true. Condition 1 is the "large block condition". This condition detects whether the samples on the P and Q sides belong to large blocks. Conditions 2 and 3 are determined by the following formulas:

[0166] Condition2=(d<β)? TRUE:FALSE

[0167] Condition3 = StrongFilterCondition = (dpq is less than (β>>4), sp+sq is less than (3*β>>5), and Abs(p0-q0) is less than (5*tC+1)>>1)? TRUE:FALSE

[0168] Where d, dpq, sp, and sq are the magnitudes of the gradient calculations used to determine the amount of detail compared to a β-based threshold (QP-related codec noise threshold) to avoid removing detail through filtering. Similar to HEVC, it is also checked whether the magnitude of the gradient across the boundary is less than a tC-based threshold, i.e., a QP-related deblocking strength threshold.

[0169] 2.7.5 Strong Deblocking Filter for Chroma

[0170] The following strong deblocking filter for chroma is defined:

[0171] p2′=(3*p3+2*p2+p1+p0+q0+4)>>3 (2-19)

[0172] p1′=(2*p3+p2+2*p1+p0+q0+q1+4)>>3 (2-20)

[0173] p0′=(p3+p2+p1+2*p0+q0+q1+q2+4)>>3 (2-20)

[0174] The above chroma filter performs deblocking on an 8×8 chroma sample grid. Chroma strong filters are used on both sides of the block boundary. Here, the chroma filter is selected when both sides of the chroma edge are greater than or equal to 8 (in units of chroma samples) and the following three conditions are met. The first decision is the decision for boundary strength and large blocks. The second and third decisions are basically the same as the HEVC luminance decision, which are on / off decisions and strong filter decisions, respectively. In the first decision, for chroma filtering, the boundary strength (bS) is modified as shown in Table 5. The conditions in Table 5 are checked in turn. If a condition is met, the remaining conditions with lower priority are skipped.

[0175]

[0176]

[0177] Table 5 Modified boundary strength

[0178] When bS is equal to 2, or when bS is equal to 1 in case a large block boundary is detected, chroma deblocking is performed. The second and third conditions are basically the same as the HEVC luma strong filter decision.

[0179] 2.7.6 Deblocking Filter at Subblock Boundaries

[0180] The deblocking filtering process is applied to a 4×4 grid for CU boundaries and transform subblock boundaries, and to an 8×8 grid for prediction subblock boundaries. The prediction subblock boundaries include prediction unit boundaries introduced by SbTMVP and affine modes, and the transform subblock boundaries include transform unit boundaries introduced by SBT and ISP modes, as well as transforms due to implicit partitioning of large CUs.

[0181] For SBT and ISP sub-blocks, similar to the logic in the transform unit (TU) in HEVC deblocking filter, the deblocking filter is applied to the TU boundary when there are non-zero coefficients in any transform sub-block across the edge. For SbTMVP and affine prediction sub-blocks, similar to the logic in the prediction unit (PU) in HEVC, the deblocking filter is applied on an 8×8 grid taking into account the difference between the motion vectors of adjacent prediction sub-blocks and the reference picture.

[0182] Transform block boundaries may be deblocked with up to 5 samples on either side of the transform boundary, which is also part of the codec block, where SbTMVP or affine is used to achieve parallel friendly deblocking. Intra-prediction sub-block boundaries that are 4 samples away from the transform block boundary may be deblocked with up to 1 sample on each side, intra-prediction sub-block boundaries that are 8 samples away from the transform block boundary may be deblocked with up to 2 samples on each side of the boundary, and other intra-prediction sub-block boundaries may be deblocked with up to 3 samples on each side of the boundary.

[0183] 2.7.7 Deblocking Decisions for Small Motion Differences

[0184] HEVC enables deblocking of prediction unit boundaries when the difference of at least one motion vector component between blocks on corresponding sides of the boundary is equal to or greater than a threshold of 1 sample. In VTM, a threshold of half a luma sample is introduced to eliminate blocking artifacts originating from boundaries between inter-prediction units with small motion vector differences.

[0185] 2.8 Luminance Mapping with Chroma Scaling (LMCS)

[0186] In VVC, a coding tool called Luma Mapping with Chroma Scaling (LMCS) is added as a new processing block before the loop filter. LMCS has two main components: 1) Loop mapping of the luma component based on an adaptive piecewise linear model; 2) For the chroma components, luma-dependent chroma residual scaling is applied. Fig.12 An example luma mapping with a chroma scaling architecture is shown. Fig.12 The LMCS architecture is shown from the decoder's perspective. Fig.12The shaded blocks 1202 in indicate where processing is applied in the mapped domain; and these include inverse quantization, inverse transform, luma intra prediction, and adding the luma prediction to the luma residual. Fig.12 The unshaded blocks in the indicate where processing is applied in the original (i.e., non-mapped) domain; and these include loop filters such as deblocking, ALF and SAO, motion compensated prediction, chroma intra prediction, adding the chroma prediction to the chroma residual, and storing the decoded picture as a reference picture. Fig.12 The shaded block 1204 in is the new LMCS functional block, which includes the forward and reverse mapping of the luma signal and the luma-dependent chroma scaling process. Like most other tools in VVC, LMCS can be enabled / disabled at the sequence level using the SPS flag.

[0187] 2.8.1 Brightness Mapping Using Piecewise Linear Model

[0188] The loop mapping of the luma component adjusts the dynamic range of the input signal by redistributing codewords within the dynamic range to improve compression efficiency. Luma mapping uses a forward mapping function FwdMap and a corresponding inverse mapping function InvMap. The FwdMap function is signaled using a piecewise linear model with 16 equal segments. The InvMap function does not need to be signaled but is derived from the FwdMap function.

[0189] The luminance mapping model is signaled in the adaptive parameter set (APS) syntax structure with aps_params_type set equal to 1 (LMCS_APS). A maximum of 4 LMCS APSs can be used in a coded video sequence. Only 1 LMCS APS can be used for a picture. The luminance mapping model is signaled using a piecewise linear model. The piecewise linear model divides the dynamic range of the input signal into 16 equal segments, and for each segment, its linear mapping parameters are represented by the number of codewords assigned to the segment. Take 10-bit input as an example. By default, each of the 16 segments will be assigned 64 codewords. The signaled number of codewords is used to calculate the scaling factor and adjust the mapping function of the segment accordingly. At the slice level, the LMCS enable flag is signaled to indicate whether the LMCS will be enabled as shown in the example. Fig.12 The LMCS process described in applies to the current slice. If LMCS is enabled for the current slice, aps_id is signaled in the slice header to identify the APS that carries the luma mapping parameters.

[0190] Each i-th fragment of the FwdMap piecewise linear model, i=0...15, is defined by two input pivot points InputPivot[] and two output (mapped) pivot points MappedPPivot[].

[0191] InputPivot[] and MappedPivot[] are calculated as follows (assuming 10-bit video):

[0192] 1)OrgCW=64

[0193] 2)For i=0:16,InputPivot[i]=i*OrgCW

[0194] 3)For i=0:16,MappedPivot[i] is calculated as follows:

[0195] MappedPivot[0] = 0;

[0196] for(i=0;i<16;i++)

[0197] MappedPivot[i+1]=MappedPivot[i]+SignalledCW[i]

[0198] Among them, SignalledCW[i] is the number of codewords signaled for the i-th segment.

[0199] like Fig.12 As shown, for inter-frame coded blocks, motion compensated prediction is performed in the mapping domain. In other words, the motion compensated prediction block Y is calculated based on the reference signal in the decoded picture buffer (DPB). pred After that, the FwdMap function is applied to map the brightness prediction block in the original domain to the mapping domain, Y′ pred =FwdMap(Y pred ). For intra-frame coded blocks, the FwdMap function is not applied because intra-frame prediction is performed in the mapping domain. r Afterwards, the InvMap function is applied to convert the reconstructed brightness values ​​in the mapped domain back to the reconstructed brightness values ​​in the original domain The InvMap function is applied to luma blocks for both intra and inter coding.

[0200] The luminance mapping process (forward and / or reverse mapping) can be implemented using a lookup table (LUT) or using on-the-fly calculations. If LUTs are used, FwdMapLUT and InvMapLUT can be pre-calculated and pre-stored for use at the tile group level, and forward and reverse mapping can be simply implemented as FwdMap(Y pred )=FwdMapLUT[Y pred ] and InvMap(Y r )=InvMapLUT[Y r]. Alternatively, on-the-fly calculation can be used. Take the forward mapping function FwdMap as an example. In order to find out the fragment to which the brightness sample belongs, the sample value is shifted right by 6 bits (corresponding to 16 equal fragments). Then, the linear model parameters of the fragment are retrieved and applied on-the-fly to calculate the mapped brightness value. Let i be the fragment index, a1, a2 are InputPivot[i] and InputPivot[i+1] respectively, and b1, b2 are MappedPivot[i] and MappedPivot[i+1] respectively. The FwdMap function is calculated as follows:

[0201] FwdMap(Y pred )=((b2-b1) / (a2-a1))*(Y pred -a1)+b1 (3-22)

[0202] The InvMap function can be calculated on the fly in a similar manner. Typically, the fragments in the mapping domain are of unequal size, so the most straightforward inverse mapping process requires a comparison to determine which fragment the current sample value belongs to. Such comparisons increase the complexity of the decoder. For this reason, VVC imposes the following bitstream constraints on the value of the output pivot point MappedPivot[i]. Assume that the range of the mapping domain (for 10-bit video, the range is [0, 1023]) is divided into 32 equal fragments. If MappedPivot[i] is not a multiple of 32, MappedPivot[i+1] and MappedPivot[1] cannot belong to the same fragment among the 32 equally sized fragments, that is, MappedPivot[i+1]>>(BitDepthY-5) should not be equal to MappedPivot[i]>>(BitDepthY-5). Due to such bitstream constraints, the InvMap function can also be implemented using a simple right shift of 5 bits (corresponding to 32 equally sized fragments) to find out which fragment a sample value belongs to.

[0203] 2.8.2 Scaling of Chroma Residuals Related to Luma

[0204] Chroma residual scaling is designed to compensate for the interaction between luma signals and their corresponding chroma signals. Whether chroma residual scaling is enabled is also signaled at the slice level. If luma mapping is enabled, an additional flag is signaled to indicate whether luma-dependent chroma residual scaling is enabled. When luma mapping is not used, luma-dependent chroma residual scaling is disabled. In addition, luma-dependent chroma residual scaling is always disabled for chroma blocks with area less than or equal to 4.

[0205] The chroma residual scaling depends on the average of the neighboring luma samples reconstructed on the top and / or left of the current VPDU. If the current CU is Inter 128x128, Inter 128x64, and Inter 64x128, the chroma residual scaling factor derived for the CU associated with the first VPDU is used for all chroma transform blocks in that CU. Denote avgYr as the average of the neighboring luma samples reconstructed (see Fig.12 ). C ScaleInv The value of is calculated as follows:

[0206] 1) Based on the InvMap function, find the index Y of the piecewise linear model to which avgYr belongs Idx .

[0207] 2) C ScaleInv =cScaleInv[Y Idx ], where cScaleInv[] is a pre-computed 16-segment LUT based on the value of SignalledCW[i] and the offset value for the chroma residual scaling process signaled in APS.

[0208] Unlike brightness mapping performed based on sample points, C ScaleInv is a constant value used for the entire chrominance block. ScaleInv , the application of chroma residual scaling is as follows:

[0209] Encoder side: C ResScale =C Res *C Scale =C Res / C ScaleInv

[0210] Decoder side: C Res =C ResScale / C Scale =C ResScale *C ScaleInv

[0211] 2.8.3 Encoder-side LMCS parameter estimation

[0212] A non-normative reference implementation is provided in the VTM encoder for estimating the LMCS model parameters. Because the VTM anchor handles standard dynamic range (SDR), high dynamic range (HDR) perceptual quantizer (PQ), and HDR hybrid log-gamma (HLG) differently, the reference algorithm in VTM11 is designed differently for SDR, HDR PQ, and HDR HLG sequences. For SDR and HDR HLG sequences, the encoder algorithm is based on local luminance variance and optimized for the peak signal-to-noise ratio (PSNR) metric. For HDR PQ sequences, the encoder algorithm is based on luminance values ​​and optimized for the weighted PSNR (wPSNR) metric.

[0213] 2.8.3.1 LMCS parameter estimation for SDR

[0214] The basic idea of ​​the VTM11 reference implementation for SDR is to assign segments with more codewords to those dynamic range segments with variance below the average, and to assign segments with fewer codewords to those dynamic range segments with variance above the average. In this way, smooth areas of the picture will be encoded and decoded with more codewords than the average, and vice versa. For SDR, VTM11 provides a reference algorithm and configurable LMCS parameters for user adjustment.

[0215] For the SDR test sequence, the reference algorithm performs the following signal analysis:

[0216] 1) Assuming a 10-bit intra codec bit depth, collect and analyze statistics for the input video. If the intra codec bit depth is not 10 bits, the bit depth is first normalized to 10 bits.

[0217] 2) Divide the dynamic range of [0, 1023] into 16 equal segments.

[0218] 3) For each brightness sample position in the image, use winSize xwinSize centered at the current position The local spatial variance of the luma sample value is calculated by the neighborhood. The specific segment (out of 32 segments) to which the current luma sample value belongs is denoted as p. Therefore, the local variance is associated with the pth segment.

[0219] 4) For each of the 16 segments, calculate the average local spatial variance (bin_var)

[0220] 5) For all valid segments, each segment is assigned an equal number of codewords; where binCW[i] is the number of codewords assigned to the i-th segment, totalCW is the total number of allowed codewords, and startIdx and endIdx are the indices of the first and last valid segment, respectively.

[0221] 6) adjusting the allocation of codewords so that more codewords are allocated to segments with lower average local spatial variance and fewer codewords are allocated to segments with higher average local variance;

[0222] If normVar<1.0, then

[0223]

[0224] Otherwise, if normVar>1.0, then

[0225]

[0226] where normVar = binVar[i] / normVar, delta1 = round(10*hist), delta2 = round(20*hist), where binVar[i] is the average local spatial variance of the luminance values ​​in the i-th segment; meanVar is the average of the average local spatial variances across all valid segments; and hist is the percentage of samples in the i-th segment to the total number of samples, clipped to the range of [0,0.4] to avoid aggressive codeword allocation.

[0227] 7) If the total number of allocated codewords exceeds the maximum number of codewords allowed, the total number of codewords is reduced by an equal amount starting from the first fragment.

[0228] 8) Based on the relative histogram and average local spatial variance of the luminance signal before and after reshaping, make adaptive decisions to set the LMCS stripe type, high bitrate, and chroma adjustment adaptation parameters. Stripe type adaptation refers to enabling LMCS for the following stripe type combinations: intra and inter; only pictures with temporal ID 0; or only inter. High bitrate adaptation refers to adjusting the number of codewords assigned to fragments with QP values ​​less than or equal to 22. Chroma adjustment adaptation refers to disabling or enabling chroma residual scaling. All of these adaptive decisions are based on a series of threshold comparisons.

[0229] 9) Signal the SignalledCW[i] value in the LMCS APS.

[0230] When LMCS is applied, the sum of squared errors (SSE) is used for the luma of intra (I) slices, and weighted SSE is used for the luma of inter (P or B) slices. The weight w_lmcs(k) is derived based on the codeword assignment of the kth fragment in the piecewise linear model as follows.

[0231] w_lmcs[k]=(SignalledCW[k] / OrgCW)^2 (2-25)

[0232] SSE is often used for chroma mode decisions.

[0233] Different considerations are given to different codec configurations in terms of picture-level decision whether to enable LMCS. For the random access (RA) test condition, picture analysis is performed for each intra random access point (IRAP) picture as explained above. Then, if the average local spatial variance of the original picture is large, or if the average local variance of the mapped picture is larger compared to the original picture, LMCS is disabled for the intra picture. For other inter-codec pictures within the same IRAP period, if LMCS is enabled for the IRAP picture, then LMCS is enabled only for the picture with the temporal layer ID equal to 0. Otherwise, if LMCS is disabled for the IRAP picture, then LMCS is enabled for all inter-codec pictures.

[0234] For all intra (AI) and low delay (LD) test conditions, LMCS is enabled for all pictures. For AI, LMCS parameter estimation is performed for all codec pictures and the model parameters are updated in the LMCS APS for all codec pictures. For LD, LMCS parameters are estimated every one second and the model parameters are updated in the LMCS APS for these picture instances.

[0235] 2.8.3.2 LMCS parameter estimation for HDR

[0236] In the JVET HDR Common Test Conditions (CTC), two types of HDR sequences are included: PQ and HLG. These two types of sequences are processed differently in the VTM reference encoder. For the PQ sequence, the VTM reference encoder applies brightness-based QP adaptation and allows the QP value to vary in the spatial domain. For the HLG sequence, static quantization is used. Accordingly, LMCS is also applied differently for these two types of sequences. For PQ, LMCS is applied using the default LMCS mapping function calculated as described below. For HLG, an LMCS parameter estimation algorithm similar to that of SDR is applied.

[0237] The VTM reference encoder uses wPSNR instead of regular PSNR as the objective quality metric in HDR PQ CTC. The default HDR PQ LMCS curve is calculated to match the delta QP (dQP) function to maximize the wPSNR metric.

[0238] Luma-based QP adaptation derives a local dQP value for each CTU based on the average value of the luminance samples:

[0239] dQP(Y)=max(-3,min(6,0.015*Y-1.5-6)) (2-26)

[0240] Where Y is the average brightness value, Y∈[0,maxY], and for 10-bit video, maxY=1023. The weight (W_SSE) used in the wPSNR calculation is derived based on the dQP value:

[0241] W_SSE(Y)=2^(dQP(Y) / 3) (2-27)

[0242] The default LMCS curve is calculated based on the brightness sample value, as shown below:

[0243] 1) Calculate the slope of the reshaped curve: slope[Y]=sqrt(W_SSE(Y))=2^(dQP(Y) / 6).

[0244] 2) If the signal is in the narrow range (also called the standard range), set slope[Y] = 0 for Y∈[0,64) or Y∈(940,1023].

[0245] 3) Calculate F[Y] by integrating slope[Y], F[Y+1]=F[Y]+slope[Y], Y=0…maxY-1

[0246] 4) Calculate FwdLUT[Y] by normalizing F[Y] to [0maxY], FwdLUT[Y]=clip3(0,maxY,round(F[Y]*maxY / F[maxY])).

[0247] 5) Calculate the number of codewords SignalledCW[i] of 16 equal segments, i=0...15, as shown below:

[0248] SignaledCW

[15] =FwdLUT

[1023] -FwdLUT

[960] ;

[0249] for(i=14;i>=0;i--)

[0250] SignaledCW[i]=FwdLUT[(i+1)*OrgCW]-FwdLUT[i*OrgCW];

[0251] In terms of rate-distortion optimization mode decision at the encoder, when LMCS is applied, for intra (I) slices, SSE is used for luma and weighted SSE is used for chroma as distortion metrics. For inter (P or B) slices, weighted SSE is used for both luma and chroma. LMCS is applied to all slices.

[0252] 2.9 Sample Adaptive Offset (SAO)

[0253] Sample adaptive offset (SAO) is applied to the reconstructed signal after the deblocking filter by using the offset specified by the encoder for each CTB. The HEVC Test Model (HM) encoder first makes a decision whether to apply the SAO process to the current slice. If SAO is applied to the slice, each CTB is classified into one of the five SAO types as shown in Table 6. The concept of SAO is to classify pixels into multiple categories and reduce distortion by adding offsets to the pixels of each category. The SAO operation includes edge offset (EO) for pixel classification using edge attributes in SAO types 1 to 4 and band offset (BO) for pixel classification using pixel intensity in SAO category 5. Each applicable CTB has SAO parameters, which include sao_merge_left_flag, sao_merge_up_flag, SAO type, and four offsets. If sao_merge_left_flag is equal to 1, the current CTB will reuse the SAO type and offset of the left CTB.

[0254] If sao_merge_up_flag is equal to 1, the current CTB will reuse the SAO type and offset of the upper CTB.

[0255] SAO Type The type of sample adaptive offset to use Number of categories 0 none 0 1 1-D 0 degree mode edge offset 4 2 1-D 90 degree pattern edge offset 4 3 1-D 135 degree pattern edge offset 4 4 1-D 45 degree pattern edge offset 4 5 Band offset 4

[0256] Table 6 Specifications of SAO types

[0257] 2.9.1 Operations of each SAO type

[0258] Fig.13 EO shows an example of four 1-D 3-pixel patterns for pixel classification. Edge offset uses four 1-D 3-pixel patterns for classification of the current pixel p by considering the edge direction information d, as Fig.13 From left to right: 0 degrees, 90 degrees, 135 degrees and 45 degrees.

[0259] According to Table 7, each CTB is classified into one of five categories.

[0260] category condition meaning 0 None of the following Basic monotony 1 p<2 adjacent Local Minimum 2 p<1 adjacent&&p==1 adjacent edge 3 p>1 adjacent&&p==1 adjacent edge 4 p>2 adjacent Local maximum

[0261] Table 7 EO pixel classification rules

[0262] Fig.14An example is shown where four bands are grouped together and indicated by their starting band positions. Band Offset (BO) classifies all pixels in a CTB region into 32 uniform bands by using the five most significant bits of the pixel value as the band index. In other words, the pixel intensity range is divided into 32 equal segments from zero to the maximum intensity value (e.g., 255 for 8-bit pixels). Four adjacent bands are grouped together and each group is indicated by its leftmost position, as shown in FIG. Fig.14 The encoder searches all positions to obtain the group with the maximum distortion reduction by compensating the offset of each frequency band.

[0263] 2.10 Adaptive Loop Filter in ECM

[0264] 2.10.1 ALF Simplification Removal

[0265] ALF gradient subsampling and ALF virtual boundary processing are removed. The block size for classification is reduced from 4×4 to 2×2. The filter size for both luma and chroma for which ALF coefficients are signaled is increased to 9×9.

[0266] 2.10.2 ALF with fixed filter

[0267] To filter the luminance samples, three different classifiers (C0, C1 and C2) and three different sets of filters (F0, F1 and F2) are used. Sets F0 and F1 contain fixed filters whose coefficients are trained for classifiers C0 and C1. The filter coefficients in F2 are signaled. Which filter from the set Fi is used for a given sample is determined by the class C assigned to that sample using classifier Ci. i decided.

[0268] 2.10.3 Filtering

[0269] First, two 13×13 diamond fixed filters F0 and F1 are applied to derive two intermediate samples R0(x,y) and R1(x,y). Then, F2 is applied to R0(x,y), R1(x,y) and adjacent samples to derive the filtered samples as:

[0270]

[0271] where f i,j is the clipping difference between the adjacent sample point and the current sample point R(x,y), and a i YesR ii20 The limit difference between (x, y) and the current sample point. Filter coefficient c i ,i=0,…21 is notified by signaling.

[0272] 2.10.4 Classification

[0273] Based on the directionality D i and activities Assign class C to each 2×2 block i :

[0274]

[0275] Among them, M D,i Indicates directionality D i The total number of .

[0276] As in VVC, the 1-D Laplacian operator is used to calculate the values ​​of horizontal, vertical and two diagonal gradients for each sample point. The sum of the sample gradients within the 4×4 window covering the target 2×2 block is used for classifier C0, and the sum of the sample gradients within the 12×12 window is used for classifiers C1 and C2. The sum of the horizontal, vertical and two diagonal gradients is expressed as and Directionality D i It is determined by comparing with a set of thresholds.

[0277]

[0278] As in VVC, the directionality D2 is derived using thresholds 2 and 4.5. For D0 and D1, the horizontal / vertical edge strength is first calculated and diagonal edge strength Use threshold Th = [1.25, 1.5, 2, 3, 4.5, 8]. If The edge strength is 0; otherwise, is to make The largest integer. If The edge strength is 0; otherwise, is to make The maximum integer of . That is, when horizontal / vertical edges dominate, D is derived by using Table 8(a) i ; Otherwise, the diagonal edges dominate, and D is derived by using Table 8(b) i .

[0279]

[0280]

[0281] Table 8 and To D i Mapping

[0282] To obtain Vertical and horizontal gradient A i The sum is mapped to the range from 0 to n, where n is equal to 4, and for and n is equal to 15. In ALF_APS, a maximum of 4 luma filter sets are signaled, and each set can have a maximum of 25 filters.

[0283] 2.10.5 Alternative 2x2 ALF Classifier

[0284] Classification in ALF is extended with additional alternative classifiers. For the signaled luma filter set, a flag is signaled to indicate whether the alternative classifier is applied. The geometric transformation is not applied to the alternative band classifier. When the band-based classifier is applied, the sum of the sample values ​​of the 2×2 luma block is first calculated. Then, the class index is calculated as follows,

[0285] class_index = (sum of sample values ​​* 25) >> (sample bit depth + 2) (2-31)

[0286] 2.10.6 CCALF with long tap filter

[0287] Fig.15 An example of a 25-tap long filter is shown. The CCALF process uses a linear filter to filter the luma sample values ​​and generate residual corrections for the chroma samples. Fig.15 As shown, a 25-tap large filter is used in the CCALF process. For a given slice, the encoder can collect statistics for the slice, analyze it, and notify up to 16 filters through APS signaling.

[0288] 2.11 Bilateral Filter

[0289] Fig.16 An example is shown where both BIF and SAO use samples from the deblocking stage as input. Both create offsets, and these offsets are added to the input samples and clipped. Fig.16 As shown, this filter is performed in the SAO loop filter stage. Both BIF and SAO use samples from deblocking as input. Each filter creates an offset for each sample, and these offsets are added to the input samples, then clipped, and then passed to the ALF.

[0290] Both BIF and SAO use samples from the deblocking stage as input. They create offsets that are added to the input samples and clipped. Specifically, the output samples I OUT It is obtained by:

[0291] I OUT =clip3(I C +ΔI BIF +ΔI SAO ), (2-32)

[0292] Among them I C is the input sample from deblocking, ΔI BIF is the offset from the bilateral filter, and ΔI SAO is an offset from SAO. The implementation provides the encoder with the possibility to enable or disable filtering at CTU and slice level. The encoder makes the decision by evaluating the RDO cost.

[0293] Fig.17 The center sample point I is shown C Example naming convention for samples around . For a filtered CTU, the filtering process is as follows. At picture boundaries, when samples are not available, the bilateral filter uses expansion (sample repetition) to fill the unavailable samples. For virtual boundaries, the behavior is the same as SAO, that is, no filtering is performed. When crossing horizontal CTU boundaries, the bilateral filter can access the same samples as SAO. For example, if Fig.17 The center sample point I C Located in the top row of CTU, like SAO, read I from the CTU above NW ,I A and I NE , but I AA is filled, so no additional row buffer is needed.

[0294] according to Fig.17 Indicates the center sample point I C The sample points around the circle are shown in Figure 1, where A, B, L, and R represent above, below, left, and right, respectively, and NW, NE, SW, and SE represent northwest (i.e., upper left), northeast (i.e., upper right), southwest (i.e., lower left), and southeast (i.e., lower right), respectively. Similarly, AA represents the upper part of the upper circle, BB represents the lower part of the lower circle, LL represents the left part of the left, and RR represents the right part of the right. This diamond shape is similar to not using I AA ,、I BB ,I LL or I RR The square filter supports different

[0295] Each surrounding sample point I A ,I R etc. will contribute corresponding correction values etc. These calculations are as follows: From the right sample point I R Starting with the contribution of , calculate the difference:

[0296] ΔI R =(|I R -I C |+4)>>3, (2-33)

[0297] Where |·| represents the absolute value. For non-10-bit data, the following formula is used instead: ΔI R =(|I R -I C |+2 n-6 )>>(n-7), where n=8 for 8-bit data, and so on. The resulting value is now clipped to be less than 16:

[0298] i R =min(15,ΔI R ) (2-34)

[0299] The correction value is now calculated as follows:

[0300]

[0301] Among them, LUT ROW [] is an array of 16 values ​​determined by the value of qpb = clip(0,25,QP+bilateral_filter_qp_offset-17):

[0302] {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,}, if qpb=0

[0303] {0,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,}, if qpb=1

[0304] {0,2,2,2,1,1,0,1,0,0,0,0,0,0,0,0,}, if qpb=2

[0305] {0,2,2,2,2,1,1,1,1,1,1,1,0,1,1,-1,}, if qpb=3

[0306] {0,3,3,3,2,2,1,2,1,1,1,1,0,1,1,-1,}, if qpb=4

[0307] {0,4,4,4,3,2,1,2,1,1,1,1,0,1,1,-1,}, if qpb=5

[0308] {0,5,5,5,4,3,2,2,2,2,2,1,0,1,1,-1,}, if qpb=6

[0309] {0,6,7,7,5,3,3,3,3,2,2,1,1,1,1,-1,}, if qpb=7

[0310] {0,6,8,8,5,4,3,3,3,3,3,2,1,2,2,-2,}, if qpb=8

[0311] {0,7,10,10,6,4,4,4,4,3,3,2,2,2,2,-2,}, if qpb=9

[0312] {0,8,11,11,7,5,5,4,5,4,4,2,2,2,2,-2,}, if qpb=10

[0313] {0,8,12,13,10,8,8,6,6,6,5,3,3,3,3,-2,}, if qpb=11

[0314] {0,8,13,14,13,12,11,8,8,7,7,5,5,4,4,-2,}, if qpb=12

[0315] {0,9,14,16,16,15,14,11,9,9,8,6,6,5,6,-3,}, if qpb=13

[0316] {0,9,15,17,19,19,17,13,11,10,10,8,8,6,7,-3,}, if qpb=14

[0317] {0,9,16,19,22,22,20,15,12,12,11,9,9,7,8,-3,}, if qpb=15

[0318] {0,10,17,21,24,25,24,20,18,17,15,12,11,9,9,-3,}, if qpb=16

[0319] {0,10,18,23,26,28,28,25,23,22,18,14,13,11,11,-3,}, if qpb=17

[0320] {0,11,19,24,29,30,32,30,29,26,22,17,15,13,12,-3,}, if qpb=18

[0321] {0,11,20,26,31,33,36,35,34,31,25,19,17,15,14,-3,}, if qpb=19

[0322] {0,12,21,28,33,36,40,40,40,36,29,22,19,17,15,-3,}, if qpb=20

[0323] {0,13,21,29,34,37,41,41,41,38,32,23,20,17,15,-3,}, if qpb=21

[0324] {0,14,22,30,35,38,42,42,42,39,34,24,20,17,15,-3,}, if qpb=22

[0325] {0,15,22,31,35,39,42,42,43,41,37,25,21,17,15,-3,}, if qpb=23

[0326] {0,16,23,32,36,40,43,43,44,42,39,26,21,17,15,-3,}, if qpb=24

[0327] {0,17,23,33,37,41,44,44,45,44,42,27,22,17,15,-3,}, if qpb=25

[0328] These values ​​can be stored using six bits per entry, resulting in 26*16*6 / 8 = 312 bytes, or 300 bytes excluding the first row of all zeros. ΔIL , μ ΔIA and μ ΔIB The correction value of I is calculated in the same way L ,

[0329] I A and I B For the diagonal sample point I NW ,I NE ,I SE ,I SW And the sample point I of the two-step distance AA ,I BB ,I RR and I LL , the calculation also follows equations (2-33) and (2-34), but uses values ​​shifted by 1. Using the diagonal sample points I SE As an example:

[0330]

[0331] And the other diagonal samples and the samples with two steps distance are calculated in the same way. The correction values ​​are added together:

[0332]

[0333] It should be noted that Same as before Likewise, Equal to the sample point above Similar symmetries can be found for the diagonal and two-step correction values. This means that in hardware implementation, the six values ​​are calculated and is sufficient, and the remaining six values ​​can be obtained from the previously calculated values.

[0334] Now m sum The value is multiplied by c = 1, 2, or 3, which can be accomplished using a single adder and a logic AND gate as follows:

[0335] c v =k1&(m sum <<1)+k2&m sum , (2-38)

[0336] Where & represents logical AND, k1 is the most significant bit of multiplier c, and k2 is the least significant bit. The value to be multiplied is obtained using the minimum block dimension D=min(width, height), as shown in Table 9:

[0337] Block Type D≤4 4<D<16 D≥16 Intra 3 2 1 Interframe 2 2 1

[0338] Table 9 obtains the c parameter according to the minimum size of the block D=min(width, height).

[0339] Finally, calculate the bilateral filter offset ΔI BIF . For full strength filtering:

[0340] ΔI BIF =(c v +16)>>5, (2-39)

[0341] And for half-intensity filtering:

[0342] ΔI BIF =(c v +32)>>6. (2-40)

[0343] The general formula for N-bit data is as follows:

[0344]

[0345] Wherein bilateal_filter_strength can be 0 or 1 and is signaled in the picture parameter set (PPS).

[0346] 2.12 Chroma Bilateral Loop Filter

[0347] Fig.18 An example of the filtering stage of BIF-chroma is shown. As with BIF-luminance, the example BIF-chroma is also performed in parallel with the SAO and CCSAO processes, as shown in Fig.18 As shown. BIF-Chroma, CCSAO and SAO use the same chroma samples produced by the deblocking filter as input and generate three offsets in parallel for each chroma sample. These three offsets are then added to the input chroma sample to obtain the sum, which is then clipped to form the final output chroma sample value. The proposed BIF-Chroma provides an on / off control mechanism at the CTU level and the slice level.

[0348] The filtering process for BIF-chroma is similar to that for BIF-luminance. For chroma samples, a 5×5 diamond filter is used to generate the filtering offset. The difference between the center sample and each surrounding sample is first calculated. The coefficient of each reference sample is extracted from a predefined lookup table directly based on the calculated difference. The coefficients for the chroma components are retrained, which is different from the coefficients in BIF-luminance. In the BIF-luminance design, the block-level filter strength parameter c is determined based on the luminance TU size and CU mode. In the BIF-chroma design, when dual-tree splitting is enabled for the current slice, the parameters of the chroma components are determined based on the chroma TU size and mode, and when dual-tree splitting is disabled, the parameters of the chroma components are determined based on the corresponding luminance TU size and mode.

[0349] 2.13 Cross-Component Sample Adaptive Offset (CCSAO)

[0350] Fig.19 An example of a modified SAO process when the proposed CCSAO is applied is shown. For example, CCSAO is used to refine the reconstructed chroma samples. Similar to SAO, CCSAO classifies the reconstructed samples into different categories, derives an offset for each category, and adds the offset to the reconstructed samples of the category. However, unlike SAO, which only uses a single luminance / chrominance component of the current sample as input, CCSAO utilizes all three components to classify the current sample into different categories. To facilitate parallel processing, the output samples from the deblocking filter are used as input to CCSAO. Fig.19 Diagram showing the decoding workflow when CCSAO is applied.

[0351] Fig. 20An example of candidate positions for a CCSAO classifier 500 is shown. For example, co-located and adjacent luminance (brightness) component Y 502 is classified using co-located chrominance (color) component U 504, co-located chrominance (color) component Y 506 and / or adjacent pixels / samples 508. In CCSAO, only BO is used to improve the quality of the reconstructed samples. For a given luminance / chrominance sample, three candidate samples are selected to classify the given sample into different categories: one co-located Y sample, one co-located U sample, and one co-located V sample. The sample values ​​of the three selected samples are then classified into three different frequency bands {band Y ,band U ,band V}, and the joint index i represents the category of a given sample. An offset is signaled and added to the reconstructed samples belonging to that category, and the formula can be expressed as:

[0352]

[0353] where {Y col ,U col ,V col} are three selected co-located sample points used to classify the current sample point; {N Y ,N U ,N V} are respectively applied to {Y col ,U col ,V col}The number of equally divided frequency bands of the full range; BD is the internal codec bit depth; C rec and C′ rec are the reconstructed samples before and after applying CCSAO; σ CCSAO [i] is the value of the CCSAO offset applied to the i-th BO category. The co-located luma samples can be selected from 9 candidate positions, while the co-located chroma sample positions are fixed, such as Fig. 20 As depicted in .

[0354] Similar to SAO, different classifiers can be applied to different local areas to further enhance the overall image quality. The parameters of each classifier (i.e., Y col The position of N Y 、N U 、N V and offset) are signaled at the picture level, and the classifier to be used is explicitly signaled and switched at the CTB level. For each classifier, {N Y ,N U ,N VThe maximum value of} is set to {16, 4, 4} and the offset is clamped to the range [-15, 15]. At most 4 classifiers are used per frame.

[0355] Fig.21 An example joint clipping after adding SAO / BIF / CCSAO offsets to the input samples is shown. The SAO, BIF, and CCSAO offsets are calculated in parallel and added to the reconstructed chroma samples and clipped jointly, as shown in Fig.21 shown.

[0356] Fig. 22 An example of four 1-D directional patterns for CCSAO EO sample classification is shown: horizontal (EO class = 0), vertical (EO class = 1), 135° diagonal, and 45° diagonal. Similar to the edge classifier of SAO in VVC, the edge-based classifier of CCSAO also uses four 1-D directional patterns for sample classification: horizontal, vertical, 135° diagonal, and 45° diagonal, as shown in Fig. 22 shown.

[0357] For each 1-D mode, each sample is classified based on the sample difference between the luma sample value labeled "c" and its two adjacent luma samples labeled "a" and "b" along the selected 1-D mode.

[0358] Similar to SAO, the encoder can use rate-distortion optimization (RDO) to decide the best 1-D directional mode and signal this additional information in each classifier / set. Both sample differences "ac" and "bc" are compared with a predefined threshold (Th) to derive the final "class_idx" information.

[0359] The encoder selects the best "Th" value from a predefined threshold array based on RDO and signals the index of the "Th" array.

[0360] In addition, another difference between the CCSAO edge-based classifier and the SAO edge classifier in VVC is that in the former, the chroma samples use co-located luma samples to derive edge information (samples "a", "c" and "b" are co-located luma samples), while in the latter, the chroma samples use their own neighboring samples to derive edge information.

[0361] The formula for the edge-based classifier process is as follows:

[0362] Ea=(ac<0)? (ac<(-Th)?0:1):(ac<(Th)?2:3) (2-43)

[0363] Eb=(bc<0)? (bc<(-Th)?0:1):(bc<(Th)?2:3) (2-44)

[0364] class_idx=i b *16+Ea*4+Eb (2-45)

[0365] C′ Rec =Clip1(C rec +σ CCSAO [class_idx]) (2-46)

[0366] The variable “i” in equation (3) B ” is derived as follows:

[0367] i B =(cur·N cur )>>BD(or)i B =(col1·N col1 )>>BD(or)i B =(col2·N col2 )>>BD,

[0368] (2-47)

[0369] The sample "cur" is the sample currently being processed, and col1 and col2 are co-located samples. When processing luminance samples, col1 and col2 are co-located C b and C r When processing chromaticity (C b ) sample point, col1 and col2 are co-located Y and C r Similarly, when processing chrominance (C r ) sample point, col1 and col2 are co-located Y and C b Sample point.

[0370] Based on RDO, the encoder signals one of the samples "cur", "col1", "col2" used in deriving the frequency band information.

[0371] 3. Technical problems solved by the disclosed technical solutions

[0372] In the example design of SAO, parameters (e.g., classification and offset) are signaled at the CTU level, and the overhead of signaling these parameters may limit the encoding and decoding performance of SAO. In the example design of CCSAO, parameters (e.g., classification and offset) are signaled at the picture level, and the overhead of signaling these parameters may limit the encoding and decoding performance of CCSAO. In the example design of CCSAO, the high-level syntax (HLS) control flags are signaled in the 4:0:0 format, which is redundant.

[0373] 5. Technical solutions and implementation examples

[0374] In order to solve the above problems, the method summarized as follows is disclosed. The embodiments should be regarded as examples to explain the general concept and should not be interpreted narrowly. In addition, these embodiments can be applied alone or in combination in any way. In these embodiments, the terms of sample adaptive offset (SAO) and cross-component sample adaptive offset (CCSAO) are not limited to the specific terms defined in the example standards or technologies. Any variant of the technology is also applicable.

[0375] Example 1

[0376] In one example, the parameters of SAO and / or CCSAO may be derived from a predefined set, a fixed set, and / or an adaptive set. In one example, the parameters of SAO and / or CCSAO may refer to parameters used to derive classification and / or offsets of different categories. In one example, an adaptive set may refer to parameters in the set being adaptively derived and explicitly signaled in the bitstream.

[0377] Example 2

[0378] In one example, which sets are used may be signaled in the bitstream.In one example, a syntax element may be used to indicate which set is used for a video unit.

[0379] Example 3

[0380] In one example, the number or maximum number of predefined / fixed sets and / or the number or maximum number of adaptive sets may be predefined or signaled in the bitstream. In one example, the number or maximum number of predefined / fixed sets and / or the number or maximum number of adaptive sets may depend on codec information.

[0381] Example 4

[0382] In one example, the classification mechanism and / or how the offsets are used in the defined / fixed set may be the same as in the adaptive set. In one example, the classification mechanism may refer to the number of categories and / or the derivation of different categories. In one example, the number of categories and / or the maximum number of categories in the predefined / fixed set may be the same as in the adaptive set. In one example, the classification mechanism and / or how the offsets are used in the defined / fixed set may be different from that in the adaptive set. In one example, the number of categories and / or the maximum number of categories in the predefined / fixed set may be different from that in the adaptive set.

[0383] Example 5

[0384] In one example, the predefined / fixed set and the adaptive set may be treated the same. In one example, multiple offsets may be derived from one or more predefined / fixed sets and / or adaptive sets. In one example, multiple offsets may be fused and applied to a sample. In one example, one of the multiple offsets may be selected and applied to a sample.

[0385] Example 6

[0386] In one example, an adaptive set of parameters may be signaled at a sequence level, a picture group level, a picture level, a slice level, and / or a slice group level, such as in a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), a decoder capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, and / or a slice group header. In one example, an adaptive set of parameters for SAO may be signaled in an APS or a picture header. In one example, an adaptive set of parameters for CCSAO may be signaled in an APS. In one example, when an adaptive set of parameters for SAO and / or CCSAO is signaled in an APS, a syntax element indicating which APS the adaptive set is from may be signaled in the bitstream for a video unit. In one example, the syntax element may refer to an APS identifier (id). In one example, a syntax element (such as aps_params_type) in the first APS may be signaled to indicate whether an adaptive set of parameters for SAO and / or CCSAO is signaled in the first APS. For example, if the syntax element is equal to a specific value, the parameters for SAO and / or CCSAO are signaled in the first APS. In one example, the parameters for SAO and / or CCSAO may be encoded and decoded in a predictive manner. In one example, a first parameter may be predicted by a second parameter signaled in the same video unit (such as an APS). In one example, a first parameter may be predicted by a second parameter signaled in a different video unit (such as an APS).

[0387] Example 7

[0388] In one example, the determination of whether and / or how to use CCSAO may depend on whether SAO is used. In one example, whether CCSAO is enabled for a component may depend on whether SAO is enabled for the component. In one example, the component may refer to a luma component. In one example, when luma SAO is disabled, CCSAO is disabled for the luma component. In one example, the component may refer to one or more chroma components. In one example, when chroma SAO is disabled for a blue difference chroma (Cb) or red difference chroma (Cr) component, CCSAO is disabled for the Cb or Cr component. In one example, CCSAO may be disabled when SAO is disabled. In one example, when the SPS flag of SAO is disabled, the SPS flag of CCSAO is not signaled and is inferred to be false.

[0389] Example 8

[0390] In one example, the determination of whether and / or how to use CCSAO may depend on the color format. In one example, when the color format is 4:0:0, CCSAO may be disabled. In one example, when the color format is 4:0:0, syntax elements related to CCSAO may not be signaled. In one example, the syntax elements may refer to syntax elements in an SPS, and / or a picture header and / or a slice header. In one example, the syntax elements may be inferred to be a predefined value (such as 0) indicating that CCSAO is not applied.

[0391] Example 9

[0392] In one example, whether and / or how CCSAO is used may depend on the color format. In one example, the color format may refer to 4:2:0, 4:2:2, or 4:4:4. In one example, the determination of the classification and / or offset may be different for different color formats. In one example, the number of categories, and / or the minimum and / or maximum offset values, and / or the thresholds for determining the classification may be different for different color formats. In one example, the chroma samples may be used for edge classifiers for 4:2:2 or 4:4:4 color formats. In one example, the candidate positions of the co-located luminance samples may be different for different color formats.

[0393] Example 10

[0394] In the above examples, a video unit may refer to a sequence, a picture, a sub-picture, a slice, a codec tree unit (CTU), a CTU row, a CTU group, a codec unit (CU), a prediction unit (PU), a transform unit (TU), a codec tree block (CTB), a codec block (CB), a prediction block (PB), a transform block (TB), and / or any other region containing more than one luma or chroma sample / pixel.

[0395] Example 11

[0396] Whether and / or how to apply the above disclosed methods can be signaled in the bitstream. In one example, they can be signaled at the sequence level / picture group level / picture level / slice level / slice group level, such as in the sequence header, picture header, SPS, VPS, DPS, DCI, PPS, APS, slice header, and slice group header. In one example, they can be signaled at PB, TB, CB, PU, ​​TU, CU, VPDU, CTU, CTU row, slice, slice, sub-picture, other types of regions containing more than one sample or pixel.

[0397] Example 12

[0398] Whether and / or how to apply the above disclosed method may depend on codec information, such as block size, color format, single / dual tree partitioning, color component, slice / picture type.

[0399] 5. Examples

[0400] 5.1 Example 1

[0401] Adaptive Parameter Set RBSP Syntax

[0402]

[0403]

[0404]

[0405]

[0406] 5.2 Example 2 Adaptive Parameter Set RBSP Syntax

[0407]

[0408]

[0409]

[0410] 5.3 Example 3 Sequence Parameter Set RBSP Syntax

[0411]

[0412] Fig.23 4000 is a block diagram illustrating an example video processing system 4000 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 4000. System 4000 may include an input 4002 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8-bit or 10-bit multi-component pixel values), or in a compressed format or an encoded format. Input 4002 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, a passive optical network (PON), and the like, and wireless interfaces such as Wi-Fi or cellular interfaces.

[0413] System 4000 may include a codec component 4004, which may implement various codecs or coding methods described in this document. Codec component 4004 may reduce the average bit rate of the video from the input 4002 of codec component 4004 to the output to generate a codec representation of the video. Therefore, codec technology is sometimes referred to as video compression or video transcoding technology. The output of codec component 4004 may be stored or may be transmitted via a communication connection as represented by component 4006. The stored or transmitted bitstream (or codec) representation of the video received at input 4002 may be used by component 4008 to generate pixel values ​​or displayable video, which is sent to display interface 4010. The process of generating a user-viewable video from a bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "codec" operations or tools, it should be understood that coding tools or operations are used at the encoder, and the corresponding decoding tools or operations of the reverse coding results will be performed by the decoder.

[0414] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High-Definition Multimedia Interface (HDMI) or DisplayPort, etc. Examples of storage interfaces include Serial Advanced Technology Attachment (SATA), Peripheral Component Interconnect (PCI), Integrated Drive Electronics (IDE) interface, etc. The techniques described in this document may be embodied in various electronic devices, such as mobile phones, laptop computers, smart phones, or other devices capable of performing digital data processing and / or video display.

[0415] Fig.244100. The apparatus 4100 may be used to implement one or more methods described herein. The apparatus 4100 may be embodied in a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The apparatus 4100 may include one or more processors 4102, one or more memories 4104, and a video processing circuit 4106. The processor(s) 4102 may be configured to implement one or more methods described in this document. The memory(s) 4104 may be used to store data and code for implementing the methods and techniques described herein. The video processing circuit 4106 may be used to implement some of the techniques described in this document in hardware circuits. In some embodiments, the video processing circuit 4106 may be at least partially included in a processor 4102 (e.g., a graphics coprocessor).

[0416] Fig.25 4200. The method 4200 includes determining parameters of a sample adaptive offset (SAO) filter at step 4202. At step 4204, conversion between visual media data and a bitstream is performed based on the SAO filter. The conversion of step 4204 may include encoding at an encoder or decoding at a decoder, depending on the example.

[0417] It should be noted that the method 4200 may be implemented in an apparatus for processing video data, the apparatus comprising a processor and a non-transitory memory having instructions thereon, such as the video encoder 4400, the video decoder 4500, and / or the encoder 4600. In such a case, the instructions, when executed by the processor, cause the processor to perform the method 4200. In addition, the method 4200 may be performed by a non-transitory computer-readable medium, the medium comprising a computer program product for use by a video codec device. The computer program product comprises computer executable instructions stored on a non-transitory computer-readable medium, such that when executed by the processor, the video codec device performs the method 4200.

[0418] Fig.26 43 is a block diagram illustrating an example video codec system 4300 that can utilize the techniques of the present disclosure. The video codec system 4300 may include a source device 4310 and a target device 4320. The source device 4310 generates encoded video data, and the source device 4310 may be referred to as a video encoding device. The target device 4320 may decode the encoded video data generated by the source device 4310, and the target device 4320 may be referred to as a video decoding device.

[0419] Source device 4310 may include a video source 4312, a video encoder 4314, and an input / output (I / O) interface 4316. Video source 4312 may include a source such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of these sources. Video data may include one or more pictures. Video encoder 4314 encodes video data from video source 4312 to generate a bitstream. The bitstream may include a series of bits that form a codec representation of video data. The bitstream may include a codec picture and associated data. The codec picture is a codec representation of a picture. Associated data may include sequence parameter sets, picture parameter sets, and other syntax structures. I / O interface 4316 may include a modulator / demodulator (modem) and / or a transmitter. The encoded video data may be directly transmitted to target device 4320 via network 4330 via I / O interface 4316. The encoded video data may also be stored on storage medium / server 4340 for access by target device 4320.

[0420] Target device 4320 may include an I / O interface 4326, a video decoder 4324, and a display device 4322. The I / O interface 4326 may include a receiver and / or a modem. The I / O interface 4326 may obtain encoded video data from source device 4310 or storage medium / server 4340. The video decoder 4324 may decode the encoded video data. The display device 4322 may display the decoded video data to a user. The display device 4322 may be integrated with the target device 4320, or may be external to the target device 4320, and may be configured to be connected to an external display device via an interface.

[0421] The video encoder 4314 and the video decoder 4324 may operate according to a video compression standard, such as the HEVC standard, the VVC standard, and other current and / or later standards.

[0422] Fig. 27 is a block diagram showing an example of a video encoder 4400, which may be Fig.26 Video encoder 4314 in system 4300 shown. Video encoder 4400 can be configured to perform any or all of the techniques of the present disclosure. Video encoder 4400 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video encoder 4400. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0423] The functional components of the video encoder 4400 may include a segmentation unit 4401; a prediction unit 4402, which may include a mode selection unit 4403, a motion estimation unit 4404, a motion compensation unit 4405 and an intra-frame prediction unit 4406; a residual generation unit 4407; a transform processing unit 4408; a quantization unit 4409; an inverse quantization unit 4410; an inverse transform unit 4411; a reconstruction unit 4412; a buffer area 4413 and an entropy coding unit 4414.

[0424] In other examples, the video encoder 4400 may include more, fewer, or different functional components. In one example, the prediction unit 4402 may include an intra-block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, in which at least one reference picture is a picture in which the current video block is located.

[0425] Furthermore, some components, such as the motion estimation unit 4404 and the motion compensation unit 4405 , may be highly integrated, but are represented separately in the example of the video encoder 4400 for purposes of explanation.

[0426] The segmentation unit 4401 may segment the picture into one or more video blocks. The video encoder 4400 and the video decoder 4500 may support various video block sizes.

[0427] The mode selection unit 4403 may, for example, select one of the coding modes (e.g., intra or inter) based on the error result, and provide the resulting intra or inter coded block to the residual generation unit 4407 to generate residual block data, and to the reconstruction unit 4412 to reconstruct the coded block for use as a reference picture. In some examples, the mode selection unit 4403 may select a combined intra and inter prediction (CIIP) mode, where the prediction is based on an inter prediction signal and an intra prediction signal. In the case of inter prediction, the mode selection unit 4403 may also select a resolution of the motion vector for the block (e.g., sub-pixel or integer pixel precision).

[0428] In order to perform inter-frame prediction on the current video block, the motion estimation unit 4404 may generate motion information for the current video block by comparing one or more reference frames from the buffer 4413 with the current video block. The motion compensation unit 4405 may determine a predicted video block for the current video block based on the motion information and decoded samples of pictures from the buffer 4413 other than the picture associated with the current video block.

[0429] The motion estimation unit 4404 and the motion compensation unit 4405 may perform different operations on the current video block, for example, depending on whether the current video block is in an I slice, a P slice, or a B slice.

[0430] In some examples, the motion estimation unit 4404 may perform unidirectional prediction on the current video block, and the motion estimation unit 4404 may search for a reference video block of the current video block in the reference picture of list 0 or list 1. The motion estimation unit 4404 may then generate a reference index indicating a reference picture containing the reference video block in list 0 or list 1, and a motion vector indicating a spatial displacement between the current video block and the reference video block. The motion estimation unit 4404 may output the reference index, the prediction direction indicator, and the motion vector as motion information of the current video block. The motion compensation unit 4405 may generate a predicted video block of the current block based on the reference video block indicated by the motion information of the current video block.

[0431] In other examples, the motion estimation unit 4404 may perform bidirectional prediction on the current video block, and the motion estimation unit 4404 may search for a reference video block of the current video block in the reference pictures in list 0, and may also search for another reference video block of the current video block in the reference pictures in list 1. Then, the motion estimation unit 4404 may generate a reference index and a motion vector, the reference index indicating the reference pictures containing the reference video blocks in list 0 and list 1, and the motion vector indicating the spatial displacement between the reference video block and the current video block. The motion estimation unit 4404 may output the reference index and the motion vector of the current video block as the motion information of the current video block. The motion compensation unit 4405 may generate a predicted video block of the current video block based on the reference video block indicated by the motion information of the current video block.

[0432] In some examples, motion estimation unit 4404 may output a complete set of motion information for use in a decoding process by a decoder. In some examples, motion estimation unit 4404 may not output a complete set of motion information for the current video. Instead, motion estimation unit 4404 may reference motion information of another video block to signal motion information of the current video block. For example, motion estimation unit 4404 may determine that the motion information of the current video block is sufficiently similar to the motion information of an adjacent video block.

[0433] In one example, the motion estimation unit 4404 may indicate a value in a syntax structure associated with the current video block that indicates to the video decoder 4500 that the current video block has the same motion information as another video block.

[0434] In another example, the motion estimation unit 4404 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 4500 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0435] As discussed above, the video encoder 4400 may predictively signal motion vectors.Two examples of predictive signaling techniques that may be implemented by the video encoder 4400 include Advanced Motion Vector Prediction (AMVP) and Merge mode signaling.

[0436] The intra prediction unit 4406 may perform intra prediction on the current video block. When the intra prediction unit 4406 performs intra prediction on the current video block, the intra prediction unit 4406 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include the predicted video block and various syntax elements.

[0437] The residual generation unit 4407 may generate residual data for the current video block by subtracting the prediction video block(s) of the current video block from the current video block. The residual data of the current video block may include residual video blocks corresponding to different sample components of the samples in the current video block.

[0438] In other examples, there may be no residual data for the current video block of the current video block, for example, in skip mode, the residual generation unit 4407 may not perform a subtraction operation.

[0439] The transform processing unit 4408 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0440] After the transform processing unit 4408 generates a transform coefficient video block associated with the current video block, the quantization unit 4409 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0441] The inverse quantization unit 4410 and the inverse transform unit 4411 may apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 4412 may add the reconstructed residual video block to the corresponding samples of one or more predicted video blocks generated by the prediction unit 4402 to generate a reconstructed video block associated with the current block for storage in the buffer 4413.

[0442] After the reconstruction unit 4412 reconstructs the video block, a loop filtering operation may be performed to reduce video block artifacts in the video block.

[0443] The entropy coding unit 4414 may receive data from other functional components of the video encoder 4400. When the entropy coding unit 4414 receives the data, the entropy coding unit 4424 may perform one or more entropy coding operations to generate entropy coded data and output a bitstream including the entropy coded data.

[0444] Fig.28 is a block diagram showing an example of a video decoder 4500, which may be Fig.26 Video decoder 4324 in the system 4300 shown. Video decoder 4500 can be configured to perform any or all of the techniques of the present disclosure. In the example shown, video decoder 4500 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video decoder 4500. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0445] In the example shown, the video decoder 4500 includes an entropy decoding unit 4501, a motion compensation unit 4502, an intra prediction unit 4503, an inverse quantization unit 4504, an inverse transform unit 4505, a reconstruction unit 4506, and a buffer 4507. In some examples, the video decoder 4500 may perform a decoding process that is generally substantially the inverse of the encoding process described with reference to the video encoder 4400.

[0446] The entropy decoding unit 4501 may retrieve an encoded bitstream. The encoded bitstream may include entropy-encoded video data (e.g., encoded blocks of video data). The entropy decoding unit 4501 may decode the entropy-encoded video data, and from the entropy-decoded video data, the motion compensation unit 4502 may determine motion information including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 4502 may determine such information, for example, by performing AMVP and Merge modes.

[0447] The motion compensation unit 4502 may generate a motion compensated block, possibly performing interpolation based on an interpolation filter. An identifier of the interpolation filter used with sub-pixel precision may be included in the syntax element.

[0448] The motion compensation unit 4502 may calculate interpolation of sub-integer pixels of a reference block using an interpolation filter used by the video encoder 4400 during encoding of the video block. The motion compensation unit 4502 may determine the interpolation filter used by the video encoder 4400 according to received syntax information and use the interpolation filter to generate a prediction block.

[0449] The motion compensation unit 4502 can use some syntax information to determine the size of the blocks used to encode (multiple) frames and / or (multiple) slices of the encoded video sequence, partitioning information describing how each macroblock of the pictures of the encoded video sequence is partitioned, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-frame codec block, and other information used to decode the encoded video sequence.

[0450] The intra prediction unit 4503 may form a prediction block from spatially adjacent blocks using, for example, an intra prediction mode received in the bitstream. The inverse quantization unit 4504 inversely quantizes, i.e., dequantizes, the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 4501. The inverse transform unit 4505 applies an inverse transform.

[0451] The reconstruction unit 4506 may add the residual block to the corresponding prediction block generated by the motion compensation unit 4502 or the intra prediction unit 4503 to form a decoded block. If necessary, a deblocking filter may be applied to the decoded block to remove blocking artifacts. The decoded video block is then stored in a buffer 4507 to provide a reference block for subsequent motion compensation / intra prediction, and also to generate a decoded video for presentation on a display device.

[0452] Fig.29 is a schematic diagram of an example encoder 4600. The encoder 4600 is suitable for implementing VVC technology. The encoder 4600 includes three loop filters, namely a deblocking filter (DF) 4602, a sample adaptive offset (SAO) 4604, and an adaptive loop filter (ALF) 4606. Unlike the DF 4602 that uses a predefined filter, the SAO 4604 and the ALF 4606 use the original samples of the current picture to reduce the mean square error between the original samples and the reconstructed samples by adding an offset and applying a finite impulse response (FIR) filter, respectively, where the offset and filter coefficients are signaled using auxiliary information of the codec. The ALF 4606 is located at the last processing stage of each picture and can be regarded as a tool that attempts to capture and repair artifacts produced by previous stages.

[0453] The encoder 4600 also includes an intra prediction component 4608 and a motion estimation / compensation (ME / MC) component 4610 configured to receive an input video. The intra prediction component 4608 is configured to perform intra prediction, and the ME / MC component 4610 is configured to perform inter prediction using a reference picture obtained from a reference picture buffer 4612. The residual block from the inter prediction or intra prediction is fed to a transform (T) component 4614 and a quantization (Q) component 4616 to generate quantized residual transform coefficients, which are fed to an entropy coding component 4618. The entropy coding component 4618 entropy codes and decodes the prediction result and the quantized transform coefficients and sends them to a video decoder (not shown). The quantization component output from the quantization component 4616 can be fed to an inverse quantization (IQ) component 4620, an inverse transform component 4622, and a reconstruction (REC) component 4624. The REC component 4624 is capable of outputting images to the DF 4602 , SAO 4604 , and ALF 4606 for filtering before the images are stored in the reference picture buffer 4612 .

[0454] A list of some example preferred solutions is provided next.

[0455] The following solutions illustrate examples of the techniques discussed herein.

[0456] 1. A method for processing video data, comprising: determining parameters of a sample adaptive offset (SAO) filter; and performing conversion between visual media data and a bitstream based on the SAO filter.

[0457] 2. The method of solution 1, wherein the SAO filter comprises a cross-component SAO (CCSAO) filter or is associated with a CCSAO filter.

[0458] 3. A method according to any of solutions 1-2, wherein parameters for the SAO filter are predefined, adaptively derived or contained in the bitstream.

[0459] 4. A method according to any of solutions 1-3, wherein parameters for the SAO filter are used for classified derivation or class offset.

[0460] 5. A method according to any one of solutions 1-4, wherein a syntax element is used to indicate which set is used for a video unit in a bitstream.

[0461] 6. A method according to any one of solutions 1-5, wherein the number of predefined sets of parameters, the maximum number of predefined sets of parameters, the number of adaptive sets of parameters or the maximum number of adaptive sets of parameters are included in the bitstream.

[0462] 7. A method according to any of solutions 1-6, wherein multiple offsets are derived from one or more sets of parameters, and wherein the multiple offsets are fused or selected and applied to the samples.

[0463] 8. A method according to any one of solutions 1-7, wherein the set of parameters is included in an adaptive parameter set (APS) or a picture header.

[0464] 9. The method according to any of solutions 1-8, wherein a syntax element in the first APS indicates whether an adaptive set of parameters of the SAO filter is included in the first APS.

[0465] 10. A method according to any one of solutions 1-9, wherein the first parameter of the SAO filter is predicted by the second parameter of the SAO filter.

[0466] 11. The method according to any of solutions 1-10, wherein the use of the CCSAO filter depends on the use of the SAO filter.

[0467] 12. A method according to any one of solutions 1-11, wherein when an SAO filter is enabled for a component, a CCSAO filter is enabled for the component.

[0468] 13. A method according to any one of solutions 1-12, wherein when the SAO filter is disabled for a component, the CCSAO filter is disabled for the component.

[0469] 14. The method according to any of solutions 1-13, wherein the use of the CCSAO filter depends on the color format.

[0470] 15. A method according to any of solutions 1-14, wherein the CCSAO filter is disabled when using a 4:0:0 color format.

[0471] 16. A method according to any of solutions 1-15, wherein syntax elements of the CCSAO filter are inferred as predefined values.

[0472] 17. A method according to any of solutions 1-16, wherein the determination of the classification or the determination of the offset is different for different color formats.

[0473] 18. A method according to any one of solutions 1-17, wherein the number of categories, the minimum offset value, the maximum offset value, or the threshold value for determining the classification is different for different color formats.

[0474] 19. The method according to any of the solutions 1-18, wherein the chroma samples are used in the edge classifier in 4:2:2 or 4:4:4 color format.

[0475] 20. The method according to any one of solutions 1-19, wherein the candidate positions of the co-located brightness samples are different for different color formats.

[0476] 21. An apparatus for processing video data, comprising: a processor; and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method of any one of solutions 1-20.

[0477] 22. A non-transitory computer-readable medium, comprising a computer program product for use by a video codec device, the computer program product comprising computer executable instructions stored on the non-transitory computer-readable medium, so that when executed by a processor, the video codec device performs the method described in any one of solutions 1-20.

[0478] 23. A non-transitory computer-readable recording medium storing a bitstream of a video, the bitstream of the video being generated by a method performed by a video processing device, wherein the method comprises: determining parameters of a sample adaptive offset (SAO) filter; and generating the bitstream based on the determination.

[0479] 24. A method of storing a bitstream of a video, comprising: determining parameters of a sample adaptive offset (SAO) filter; generating a bitstream based on the determination; and storing the bitstream in a non-transitory computer-readable recording medium.

[0480] 25. A method, apparatus or system as described in this document.

[0481] In the solution described herein, an encoder may comply with the format rules by generating a codec representation according to the format rules. In the solution described herein, a decoder may parse the syntax elements in the codec representation using the format rules to generate decoded video, knowing the presence and absence of syntax elements according to the format rules.

[0482] In this document, the term "video processing" may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during the conversion process from a pixel representation of a video to a corresponding bitstream representation, or vice versa. As defined by the syntax, the bitstream representation of the current video block may correspond, for example, to bits that are co-located or distributed at different locations in the bitstream. For example, a macroblock may be encoded based on a transformed and coded error residual value, and the macroblock may also be encoded using bits in a header and other fields in the bitstream. In addition, during conversion, the decoder may parse the bitstream based on the determination described in the above solution with the knowledge that certain fields are present or absent. Similarly, the encoder may determine whether to include certain syntax fields and generate the codec representation accordingly by including or excluding the syntax fields in the codec representation.

[0483] The disclosed and other solutions, examples, embodiments, modules and functional operations described in this document may be implemented in digital electronic circuits, or in computer software, firmware or hardware (including the structures disclosed in this document and their equivalents), or in a combination of one or more of them. The disclosed and other embodiments may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium, for execution by a data processing device or for controlling the operation of the data processing device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of substances that affect a machine-readable propagation signal, or a combination of one or more of them. The term "data processing device" includes all devices, equipment and machines for processing data, including, for example, a programmable processor, a computer or multiple processors or computers. In addition to hardware, a device may also include code that creates an execution environment for the computer program in question, for example, code that constitutes a processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagation signal is an artificially generated signal, for example, a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.

[0484] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language (including compiled or interpreted languages) and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that contains other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing portions of one or more modules, subroutines, or code). A computer program may be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communications network.

[0485] The processes and logic flows described in this document may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and apparatus may also be implemented as, special purpose logic circuitry, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC).

[0486] Processors suitable for executing computer programs include, for example, both general-purpose and special-purpose microprocessors, and any one or more processors of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data or be operably coupled to these mass storage devices to receive data from them or to transmit data to them, or both. However, a computer does not necessarily require such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-transitory memory, media, and memory devices, for example, including semiconductor storage devices, such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory devices; disks, such as internal hard disks or removable disks; magneto-optical disks; and compact disk read-only memory (CD ROM) and digital versatile disk read-only memory (DVD-ROM) disks. The processor and memory can be supplemented by or incorporated into a dedicated logic circuit.

[0487] Although the present disclosure contains many details, these details should not be interpreted as limitations on any subject matter or the scope of what may be claimed, but rather as descriptions of features that are peculiar to particular embodiments of particular technologies. Certain features described in the present disclosure in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination. In addition, although the above features may be described as functioning in certain combinations, or even initially claimed as such, in some cases one or more features in the combination may be removed from the claimed combination, and the claimed combination may point to a sub-combination or a variant of a sub-combination.

[0488] Similarly, although operations are depicted in a particular order in the drawings, this should not be understood as requiring that the operations be performed in the particular order or sequence shown, or that all of the operations shown be performed, in order to achieve the desired results. In addition, the separation of various system components in the embodiments described in this disclosure should not be understood as requiring such separation in all embodiments.

[0489] Only a few implementations and examples are described, and other implementations, enhancements, and variations may be made based on what is described and shown in this disclosure.

[0490] A first component is directly coupled to a second component when there is no intermediate component between the first component and the second component other than a line, a trace, or another medium. A first component is indirectly coupled to a second component when there is an intermediate component between the first component and the second component other than a line, a trace, or another medium. The term "coupled" and its variations include both direct coupling and indirect coupling. Unless otherwise specified, the use of the term "approximately" is intended to include a range of ±10% of the subsequent value.

[0491] Although several embodiments have been provided in the present disclosure, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. The present examples should be considered illustrative rather than restrictive, and are not intended to be limited to the details given herein. For example, various elements or components may be combined or integrated in another system, or certain features may be omitted or not implemented.

[0492] In addition, without departing from the scope of the present disclosure, the techniques, systems, subsystems and methods described and shown as discrete or separate in various embodiments can be combined or integrated with other systems, modules, techniques or methods. Other items shown or discussed as coupled can be directly connected, or can be indirectly coupled or communicated by electrical, mechanical or other means through some interface, device or intermediate component. Other examples of changes, substitutions and alterations can be determined by those skilled in the art, and these changes, substitutions and alterations can be made without departing from the spirit and scope disclosed herein.

Claims

1. A method for processing video data, comprising: determining parameters of a sample adaptive offset (SAO) filter for a video unit, wherein the parameters are selected from an explicitly signaled set, a predefined set, or an adaptive set, wherein the adaptive set is derived based on parameters of one or more video units encoded and decoded before the video unit; and Conversion between visual media data and a bitstream is performed based on the SAO filter.

2. The method according to claim 1, wherein: The SAO filter includes or is associated with a cross-component SAO (CCSAO) filter.

3. The method according to any one of claims 1 to 2, wherein: The parameters of the SAO or CCSAO filter are derived from the predefined set, or the adaptive set, or the explicitly signaled set.

4. The method according to any one of claims 1 to 3, wherein: The parameters of the SAO or CCSAO filter are a classification mechanism for samples, or offsets for sample classes.

5. The method according to any one of claims 1 to 4, wherein: Syntax elements signaled in the bitstream are used to indicate which set is used to derive the parameters of the SAO or CCSAO filter for a video unit.

6. The method according to any one of claims 1 to 5, wherein: The classification mechanism, the number of categories, the maximum number of categories, or the mechanism of using the offset in the predefined set is the same as that in the adaptive set.

7. The method according to any one of claims 1 to 6, wherein: The classification mechanism, the number of categories, the maximum number of categories, or the mechanism of using the offset in the predefined set is different from that in the adaptive set.

8. The method according to any one of claims 1 to 7, wherein: The number of predefined sets of parameters, the maximum number of predefined sets of parameters, the number of adaptive sets of parameters or the maximum number of adaptive sets of parameters is included in the bitstream, is predefined, or is derived depending on codec information.

9. The method according to any one of claims 1 to 8, wherein: Multiple offsets are derived from one or more sets of parameters, and wherein the multiple offsets are blended or selected and applied to the samples.

10. The method according to any one of claims 1 to 9, wherein: The adaptation set including the parameters of the SAO or CCSAO filter is included in a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), a decoder capability information (DCI), a picture parameter set (PPS) and an adaptation parameter set (APS), a slice header or a slice group header.

11. The method according to any one of claims 1 to 10, wherein: The syntax element indicates which APS the adaptation set belongs to when multiple APSs are used, or the syntax element in a first APS indicates whether the adaptation set of the SAO or CCSAO filter is included in the first APS.

12. The method according to any one of claims 1 to 11, wherein: The first parameters of the SAO or CCSAO filter are predicted by second parameters of the SAO or CCSAO filter.

13. The method according to any one of claims 1 to 12, wherein: Whether to use the CCSAO filter depends on whether to use the SAO filter.

14. The method according to any one of claims 1 to 13, wherein: When the SAO filter is enabled for a component, the CCSAO filter is enabled for the component.

15. The method according to any one of claims 1 to 14, wherein: When a syntax element in an SPS, a picture header, or a slice header indicates that the SAO filter is enabled for a component, the CCSAO filter is enabled for the component.

16. The method according to any one of claims 1 to 15, wherein: When the SAO filter is disabled for a component, the CCSAO filter is disabled for the component.

17. The method according to any one of claims 1 to 16, wherein: When a syntax element in an SPS, a picture header, or a slice header indicates that the SAO filter is disabled for a component, the CCSAO filter is disabled for the component.

18. The method according to any one of claims 1 to 17, wherein: The CCSAO filter used depends on the color format.

19. The method according to any one of claims 1 to 18, wherein: When using the 4:0:0 color format, the CCSAO filter is disabled.

20. The method according to any one of claims 1 to 19, wherein: The syntax elements for the CCSAO filter are not signaled and are inferred as predefined values.

21. The method according to any one of claims 1 to 20, wherein: The determination of the classification or the determination of the offset is different for different color formats.

22. The method according to any one of claims 1 to 21, wherein: The number of categories, the minimum offset value, the maximum offset value, or the threshold value for determining the classification is different for different color formats.

23. The method according to any one of claims 1 to 22, wherein: The chroma samples used in the edge classifier are different for 4:2:2 or 4:4:4 color formats.

24. The method according to any one of claims 1 to 23, wherein: The candidate positions of the co-located luminance samples are different for different color formats.

25. The method according to any one of claims 1 to 24, wherein: The converting includes encoding the visual media data into the bitstream.

26. The method according to any one of claims 1 to 24, wherein: The converting includes decoding the visual media data from the bitstream.

27. An apparatus for processing video data, comprising: processor; and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method of any one of claims 1-26.

28. A non-transitory computer-readable medium, comprising a computer program product for use by a video codec device, the computer program product comprising computer executable instructions stored on the non-transitory computer-readable medium, so that when the computer program product is executed by a processor, the video codec device performs the method as described in any one of claims 1-26.

29. A non-transitory computer-readable recording medium storing a bit stream of a video generated by a method performed by a video processing device, wherein the method comprises: Determining parameters of a sample adaptive offset (SAO) filter; as well as The bitstream is generated based on the determination.

30. A method for storing a bitstream of a video, comprising: Determining parameters of a sample adaptive offset (SAO) filter; generating the bitstream based on the determination; as well as The bit stream is stored in a non-transitory computer-readable recording medium.

Images