Adaptive cross-component sample offset filtering parameters

Abstract

The present disclosure describes a set of advanced video coding techniques, and in particular relates to cross component sample offset (CCSO) filtering of reconstructed samples. For example, CCSO filter parameters may be allowed to vary between different filtering units. The filtering unit may be a filtering block in the reconstructed frame or a filtering region comprising a plurality of spatially adjacent filtering blocks. A corresponding permission option for a combination of CCSO filter parameters may be specified. Each filtering unit may be selected from the allowed combination options. The selected combination options may be signaled or derived in some other way in the bitstream. Permitting such CCSO filter parameter variations within a frame may provide a filter quality gain that exceeds the overhead cost in signaling.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This PCT international application is based on and claims the priority of U.S. non-provisional patent application No. 18 / 466,603 filed on September 13, 2023, which is based on and claims the priority of U.S. provisional application No. 63 / 447,308 filed on February 21, 2023, and the U.S. non-provisional patent application and the U.S. provisional application are incorporated herein by reference in their entirety. Technical Field

[0002] The present disclosure generally describes a set of advanced video coding techniques, and particularly relates to cross-component sample offset filtering. Background Art

[0003] Uncompressed digital video may include a series of pictures and may specify bitrate requirements for transmission bandwidth in storage, data processing, and streaming applications. One purpose of video encoding and decoding may be to reduce redundancy in the uncompressed input video signal through various compression techniques. Summary of the invention

[0004] The present disclosure describes a set of advanced video coding techniques, and in particular relates to cross-component sample offset (CCSO) filtering of reconstructed samples. For example, CCSO filter parameters may be allowed to vary between different filter units. A filter unit may be a filter block in a reconstructed frame or a filter region comprising a plurality of spatially adjacent filter blocks. Corresponding allowed options for CCSO filter parameter combinations may be specified. Each filter unit may select from the allowed options. The selected option may be signaled in the bitstream or derived in some other manner. Allowing such CCSO filter parameter variations within a frame may provide a filtering quality gain that exceeds the overhead cost in signaling.

[0005] In some example implementations, a method for in-loop filtering of a video bitstream is disclosed. The method may include: reconstructing a frame from the video bitstream to generate reconstructed samples of at least a first color component and a second color component; for a filter unit at a level lower than the level of the reconstructed frame, selecting a combination of cross-component sample offset (CCSO) filter parameters from a plurality of candidate combinations of CCSO filter parameters for use in the filter unit in the reconstructed frame; determining a CCSO filter based on the selected combination of CCSO filter parameters; and applying the CCSO filter to the reconstructed samples of the first color component in the filter unit to generate an offset to be applied to the reconstructed samples of the second color component in the filter unit.

[0006] In the above example implementation, each of the multiple candidate combinations of CCSO filter parameters may include: a filter shape parameter indicating the number and tap positions of the CCSO filter taps; a quantization step size used to discretize the cross-component taps relative to sample increments; and a number of bands indicating the number of sample amplitude bands within the allowed sample value range.

[0007] In any of the example implementations of the above implementations, the plurality of candidate combinations of CCSO filter parameters include four candidate combinations.

[0008] In any of the example implementations above, the four candidate combinations differ in both filter shape parameters and the number of bands.

[0009] In any of the example implementations in the above implementations, the four candidate combinations share the same filter shape parameters.

[0010] In any of the above exemplary implementations, the four candidate combinations share the same quantization step size.

[0011] In any of the example implementations of the above implementations, a combination of CCSO filter parameters is selected from a plurality of candidate combinations of CCSO filter parameters according to syntax elements explicitly signaled in a video bitstream.

[0012] In any of the example implementations above, the CCSO filter includes a CCSO lookup table for determining a cross-component sample offset for a filtering unit.

[0013] In any of the example implementations of the above implementations, each candidate combination of the plurality of candidate combinations of CCSO filter parameters is associated with at least one CCSO lookup table.

[0014] In any of the example implementations above, wherein a CCSO lookup table for each candidate combination of multiple candidate combinations of CCSO filter parameters is signaled in the video bitstream.

[0015] In any of the example implementations above, the multiple candidate combinations of CCSO filter parameters are the same in all filtering units in the reconstructed frame.

[0016] In any of the example implementations of the above implementations, the plurality of candidate combinations of CCSO filter parameters differ between at least two different filtering units in the reconstructed frame.

[0017] In any of the example implementations above, a plurality of candidate combinations of CCSO filtering parameters are associated with a combination set identifier.

[0018] In any of the example implementations above, wherein a combination set identifier for the filtering unit to select a CCSO filter parameter combination is signaled in the video bitstream.

[0019] In any of the exemplary implementations of the above implementations, the filtering unit includes a filtering block or a filtering area, and the filtering area includes a plurality of spatially adjacent filtering blocks.

[0020] In any of the example implementations above, values ​​of CCSO filter parameters for multiple candidate combinations of CCSO filter parameters are signaled in a sequence, frame, slice, or tile header in a high-level syntax.

[0021] In some implementations, a video encoding or decoding apparatus is disclosed. The apparatus may include a circuit system configured to implement any of the above methods.

[0022] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by a computer for video decoding and / or encoding, cause the computer to perform a method for video decoding and / or encoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings, in which:

[0024] Figure 1 A schematic diagram showing a simplified block diagram of a communication system (100) according to an example implementation.

[0025] Figure 2 A schematic diagram showing a simplified block diagram of a communication system (200) according to an example implementation.

[0026] Figure 3 A schematic diagram showing a simplified block diagram of a video decoder according to an example implementation.

[0027] Figure 4 A schematic diagram showing a simplified block diagram of a video encoder according to an example implementation.

[0028] Figure 5 A block diagram of a video encoder according to another example embodiment is shown.

[0029] Figure 6 A block diagram of a video decoder according to another example embodiment is shown.

[0030] Figure 7 An exemplary adaptive loop filter according to an embodiment of the present disclosure is shown.

[0031] FIG. 8A to FIG. 8D Examples of sub-sampling positions respectively used to calculate gradients in a vertical direction, a horizontal direction, and two diagonal directions according to an embodiment of the present disclosure are shown.

[0032] Fig. 8E An example approach to determining block directionality based on various gradients for use by an Adaptive Loop Filter (ALF) is shown.

[0033] Fig.9A and Fig. 9B Modified block classification at virtual boundaries is shown according to an example implementation of the present disclosure.

[0034] FIG. 10A to FIG. 10F An exemplary adaptive loop filter with padding operations at corresponding virtual boundaries according to an embodiment of the present disclosure is shown.

[0035] Fig.11 An example of maximum coding unit aligned picture quadtree partitioning according to an embodiment of the present disclosure is shown.

[0036] Fig.12 An example embodiment according to the present disclosure is shown. Fig.11 The corresponding quadtree partitioning pattern.

[0037] Fig.13 Cross-component filters for generating chroma components according to an example implementation of the present disclosure are shown.

[0038] Fig.14 An example of a cross-component ALF filter according to an embodiment of the present disclosure is shown.

[0039] Fig.15 Exemplary positions of chroma samples relative to luma samples are shown in accordance with an embodiment of the present disclosure.

[0040] Fig.16 An example of performing a direction search for a block according to an embodiment of the present disclosure is shown.

[0041] Fig.17 An example of subspace projection according to an embodiment of the present disclosure is shown.

[0042] Fig.18An example location of cross-component sample offset (CCSO) filtering in the loop filter pipeline is shown.

[0043] Fig.19 An example of a filter support region in a CCSO filter according to an embodiment of the present disclosure is shown.

[0044] Fig. 20 An example implementation of a 3-tap CCSO filter shape is shown in accordance with an embodiment of the present disclosure.

[0045] Fig.21 A flow chart outlining a process (2100) according to an implementation of the present disclosure is shown.

[0046] Fig. 22 A schematic diagram of a computer system according to an example implementation is shown. DETAILED DESCRIPTION

[0047] Throughout the specification and claims, terms may have nuanced meanings that are suggested or implied from the context beyond the explicitly stated meaning. The phrases "in one embodiment / implementation" or "in some embodiments / implementations" as used herein do not necessarily refer to the same embodiment / implementation, and the phrases "in another embodiment / implementation" or "in other embodiments" as used herein do not necessarily refer to different embodiments. For example, it is intended that the claimed subject matter includes combinations of all or part of the exemplary embodiments / implementations.

[0048] Typically, terms can be understood at least in part from usage in context. For example, terms such as "and", "or" or "and / or" used in this document can include various context-dependent meanings. Typically, "or", if used in an association list such as A, B, or C, is intended to mean: A, B, and C, used herein in an inclusive sense; and A, B, or C, used herein in an exclusive sense. In addition, the terms "one or more", "at least one", "one", "an", or "the" used in this document can be used in a singular sense or a plural sense, depending at least in part on the context. In addition, the term "based on" or "determined by..." can be understood as not necessarily intended to convey an exclusive set of factors, and can alternatively allow for the presence of additional factors that are not necessarily explicitly described, which also depends at least in part on the context.

[0049] Figure 1A simplified block diagram of a communication system (100) according to an embodiment of the present disclosure is shown. The communication system (100) includes a plurality of terminal devices, such as terminal devices 110, 120, 130, and 140, which can communicate with each other via, for example, a network (150). Figure 1 In an example, a first pair of terminal devices (110) and (120) can perform unidirectional transmission of data. For example, the terminal device (110) can encode video data in the form of one or more encoded bit streams (e.g., a video picture stream captured by the terminal device (110)) for transmission via the network (150). The terminal device (120) can receive the encoded video data from the network (150), decode the encoded video data to restore the video picture, and display the video picture based on the restored video data. The unidirectional data transmission can be implemented in a media service application, etc.

[0050] In another example, the second pair of terminal devices (130) and (140) can perform bidirectional transmission of encoded video data, for example, during a video conferencing application. For the bidirectional transmission of data, in the example, each of the terminal devices (130) and (140) can encode video data (e.g., a video picture stream captured by the terminal device) for transmission to the other terminal device (130) and (140), and can also receive the encoded video data from the other terminal device (130) and (140) to restore and display the video picture.

[0051] exist Figure 1 In the example of, the terminal device can be implemented as a server, a personal computer and a smart phone, but the applicability of the basic principles of the present disclosure may not be limited to this. The embodiments of the present disclosure can be implemented in a desktop computer, a laptop computer, a tablet computer, a media player, a wearable computer, a dedicated video conferencing equipment, etc. The network (150) represents any number or type of network that transmits encoded video data between terminal devices, including, for example, wired (wired) and / or wireless communication networks. The communication network (150) can exchange data in a circuit switching channel, a packet switching channel and / or other types of channels. Representative networks include telecommunication networks, local area networks, wide area networks and / or the Internet.

[0052] As examples of applications of the disclosed subject matter, Figure 2 The placement of the video encoder and the video decoder in a video streaming environment is shown. The disclosed subject matter can be equally applicable to other video applications, including, for example, video conferencing, digital TV broadcasting, gaming, virtual reality, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0053] like Figure 2As shown, a video streaming system may include a video capture subsystem (213), which may include a video source (201) such as a digital camera for creating an uncompressed video picture or image stream (202). In an example, the video picture stream (202) includes samples recorded by the digital camera of the video source 201. The video picture stream (202) is depicted as a thick line to emphasize the high amount of data when compared to the encoded video data (204) (or encoded video bitstream), and the video picture stream (202) may be processed by an electronic device (220) coupled to the video source (201) including a video encoder (203). The video encoder (203) may include hardware, software, or a combination thereof to implement or implement aspects of the disclosed subject matter as described in more detail below. The encoded video data (204) (or encoded video bitstream (204)) is depicted as a thin line to emphasize the lower data volume when compared to the uncompressed video picture stream (202), and the encoded video data (204) can be stored on the streaming server (205) for future use or directly to a downstream video device (not shown). One or more streaming client subsystems such as Figure 2 The client subsystems (206) and (208) in the streaming server (205) can access the streaming server (205) to retrieve copies (207) and (209) of the encoded video data (204). The client subsystem (206) can include, for example, a video decoder (210) in the electronic device (230). The video decoder (210) decodes the incoming copy (207) of the encoded video data and creates an outgoing video picture stream (211) that is uncompressed and can be presented on a display (212) (e.g., a display screen) or other presentation device (not shown).

[0054] Figure 3 A block diagram of a video decoder (310) of an electronic device (330) according to any of the following embodiments of the present disclosure is shown. The electronic device (330) may include a receiver (331) (e.g., receiving circuitry). The video decoder (310) may be used instead of Figure 2 A video decoder (210) in an example of FIG.

[0055] like Figure 3As shown, a receiver (331) can receive one or more coded video sequences from a channel (301). In order to combat network jitter and / or handle playback timing, a buffer memory (315) can be set between the receiver (331) and an entropy decoder / parser (320) (hereinafter referred to as "parser (320)"). The parser (320) can reconstruct symbols (321) based on the coded video sequence. The categories of these symbols include information for managing the operation of the video decoder (310) and potential information for controlling a rendering device such as a display (312) (e.g., a display screen). The parser (320) can parse / entropy decode the coded video sequence. The parser (320) can extract a subgroup parameter set for at least one of the subgroups of pixels in the video decoder from the coded video sequence. The subgroup can include a group of pictures (Group of Picture, GOP), a picture, a tile, a slice, a macroblock, a coding unit (Coding Unit, CU), a block, a transform unit (Transform Unit, TU), a prediction unit (Prediction Unit, PU), etc. The parser (320) may also extract information such as transform coefficients (e.g., Fourier transform coefficients), quantizer parameter values, motion vectors, etc. from the coded video sequence. The reconstruction of the symbols (321) may involve a number of different processing or functional units. The units involved and how these units are involved may be controlled by subgroup control information parsed by the parser (320) from the coded video sequence.

[0056] The first unit may include a sealer / inverse transform unit (351). The sealer / inverse transform unit (351) may receive quantized transform coefficients as (one or more) symbols (321) and control information from the parser (320), including information indicating which type of inverse transform to use, block size, quantization factors / parameters, quantization scaling matrix, etc. The sealer / inverse transform unit (351) may output blocks including sample values ​​that may be input into an aggregator (355).

[0057] In some cases, the output samples of the scaler / inverse transform (351) may belong to an intra-coded block, i.e., a block that does not use predictive information from a previously reconstructed picture but may use predictive information from a previously reconstructed portion of the current picture. Such predictive information may be provided by an intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) may use surrounding block information that has already been reconstructed and stored in a current picture buffer (358) to generate a block of the same size and shape as the block being reconstructed. For example, the current picture buffer (358) buffers a partially reconstructed current picture and / or a fully reconstructed current picture. In some implementations, the aggregator (355) may add the predictive information that has been generated by the intra-prediction unit (352) to the output sample information provided by the scaler / inverse transform unit (351) on a per-sample basis.

[0058] In other cases, the output samples of the scaler / inverse transform unit (351) may belong to an inter-coded and possibly motion-compensated block. In such a case, the motion compensated prediction unit (353) may access the reference picture memory (357) based on the motion vector to obtain samples for inter-picture prediction. After motion compensation of the obtained reference samples according to the symbols (321) belonging to the block, these samples may be added to the output of the scaler / inverse transform unit (351) by the aggregator (355) (the output of unit 351 may be referred to as residual samples or residual signal) to generate output sample information.

[0059] The output samples of the aggregator (355) may be subjected to various loop filtering techniques in a loop filter unit (356) which may include several types of loop filters. The output of the loop filter unit (356) may be a sample stream which may be output to a rendering device (312) and stored in a reference picture memory (357) for future inter-picture prediction.

[0060] Figure 4 A block diagram of a video encoder (403) according to an example implementation of the present disclosure is shown. The video encoder (403) may be included in an electronic device (420). The electronic device (420) may also include a transmitter (440) (e.g., a transmission circuit system). The video encoder (403) may be used instead of Figure 4 A video encoder (403) in an example.

[0061] The video encoder (403) may receive video samples from a video source (401). According to some example embodiments, the video encoder (403) may encode and compress pictures of a source video sequence into an encoded video sequence (443) in real time or under any other time constraints required by the application. Implementing a suitable encoding speed constitutes a function of the controller (450). In some embodiments, the controller (450) may be functionally coupled to and control other functional units as described below. Parameters set by the controller (450) may include parameters related to rate control (picture skipping, quantizer, lambda value of rate-distortion optimization techniques, etc.), picture size, picture group (GOP) layout, maximum motion vector search range, etc.

[0062] In some example embodiments, the video encoder (403) may be configured to operate in a decoding loop. The decoding loop may include a source encoder (430) and a (local) decoder (433) embedded in the video encoder (403). Even if the embedded decoder 433 is processed by the encoded video stream encoded by the source encoder 430 without entropy coding, the decoder (433) reconstructs the symbols to create sample data in a manner similar to that of the (remote) decoder to create sample data (because in the video compression techniques considered in the disclosed subject matter, any compression between the symbols and the encoded video bitstream in entropy coding can be lossless). At this point, it can be observed that any decoder technology other than parsing / entropy decoding that may only exist in the decoder may also necessarily need to exist in the corresponding encoder in substantially the same functional form. For this reason, the disclosed subject matter may sometimes focus on the decoder operation associated with the decoding portion of the encoder. Since the encoder technology is reciprocal to the decoder technology that has been fully described, the description of the encoder technology can be simplified. A more detailed description of the encoder is provided below only in certain areas or aspects.

[0063] During operation in some example implementations, the source encoder (430) may perform motion compensated predictive encoding that predictively encodes an input picture with reference to one or more previously encoded pictures from a video sequence designated as "reference pictures."

[0064] The local video decoder (433) can decode the encoded video data of the picture that can be designated as the reference picture. The local video decoder (433) replicates the decoding process that can be performed by the video decoder on the reference picture, and can cause the reconstructed reference picture to be stored in the reference picture cache (434). In this way, the video encoder (403) can store a copy of the reconstructed reference picture locally, which has common content (absent transmission errors) with the reconstructed reference picture to be obtained by the far-end (remote) video decoder.

[0065] The predictor (435) can perform prediction search for the encoding engine (432). That is, for a new picture to be encoded, the predictor (435) can search the reference picture memory (434) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference picture motion vectors, block shapes, etc., that can be used as appropriate prediction references for the new picture.

[0066] The controller (450) can manage encoding operations of the source encoder (430), including, for example, setting parameters and subgroup parameters for encoding video data.

[0067] The outputs of all the above mentioned functional units may be subjected to entropy encoding in an entropy encoder (445). The transmitter (440) may buffer the encoded video sequence(s) created by the entropy encoder (445) in preparation for transmission via a communication channel (460), which may be a hardware / software link to a storage device where the encoded video data will be stored. The transmitter (440) may combine the encoded video data from the video encoder (403) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0068] The controller (450) may manage the operation of the video encoder (403). During encoding, the controller (450) may assign a certain encoding picture type to each encoding picture, which may affect the encoding techniques that may be applied to the corresponding picture. For example, a picture may generally be assigned to one of the following picture types: intra picture (I picture), predictive picture (P picture), bidirectional predictive picture (B picture), multi-predictive picture. As described in further detail below, a source picture may generally be spatially subdivided into a plurality of sample encoding blocks.

[0069] Figure 5 A diagram of a video encoder (503) according to another example implementation of the present disclosure is shown. The video encoder (503) is configured to receive a processed block (e.g., a predicted block) of sample values ​​within a current video picture in a sequence of video pictures and encode the processed block into an encoded picture that is part of an encoded video sequence. The example video encoder (503) may be used instead of Figure 4 The video encoder (403) in the example.

[0070] For example, the video encoder (503) receives a matrix of sample values ​​for a processing block. The video encoder (503) then uses, for example, rate-distortion optimization (RDO) to determine whether to best encode the processing block using intra mode, inter mode, or bi-prediction mode.

[0071] exist Figure 5 In the example of FIG. 5 , the video encoder ( 503 ) includes: Figure 5 An inter-frame encoder (530), an intra-frame encoder (522), a residual calculator (523), a switch (526), ​​a residual encoder (524), a general controller (521) and an entropy encoder (525) coupled together are shown in the example arrangement.

[0072] The inter-frame encoder (530) is configured to receive samples of a current block (e.g., a processing block), compare the block with one or more reference blocks in a reference picture (e.g., blocks in a previous picture and a subsequent picture in display order), generate inter-frame prediction information (e.g., based on a description of redundant information, motion vectors, merge mode information of an inter-frame coding technique), and calculate an inter-frame prediction result (e.g., a prediction block) based on the inter-frame prediction information using any suitable technique.

[0073] The intra encoder (522) is configured to receive samples of a current block (e.g., a processing block), compare the block with already encoded blocks in the same picture, and generate quantized coefficients after transformation, and in some cases also generate intra predictive information (e.g., intra prediction direction information according to one or more intra coding techniques).

[0074] The general controller (521) may be configured to determine general control data, and control other components of the video encoder (503) based on the general control data, such as determining a prediction mode for a block and providing a control signal to a switch (526) based on the prediction mode.

[0075] The residual calculator (523) may be configured to calculate the difference (residual data) between the received block and the prediction result of the block selected from the intra encoder (522) or the inter encoder (530). The residual encoder (524) may be configured to encode the residual data to generate a transform coefficient. The transform coefficient is then subjected to a quantization process to obtain a quantized transform coefficient. In various example embodiments, the video encoder (503) further includes a residual decoder (528). The residual decoder (528) is configured to perform an inverse transform and generate decoded residual data. The entropy encoder (525) may be configured to format the bitstream to include the coded block and perform entropy encoding.

[0076] Figure 6 A diagram of an example video decoder (610) according to another embodiment of the present disclosure is shown. The video decoder (610) is configured to receive an encoded picture as part of an encoded video sequence and decode the encoded picture to generate a reconstructed picture. In an example, the video decoder (610) may be used instead of Figure 4A video decoder (410) in an example of FIG.

[0077] exist Figure 6 In the example of FIG. 6 , the video decoder ( 610 ) includes Figure 6 An entropy decoder (671), an inter-frame decoder (680), a residual decoder (673), a reconstruction module (674), and an intra-frame decoder (672) coupled together are shown in the example arrangement of.

[0078] The entropy decoder (671) may be configured to reconstruct certain symbols representing the syntax elements constituting the coded picture according to the coded picture. The inter-frame decoder (680) may be configured to receive inter-frame prediction information and generate an inter-frame prediction result based on the inter-frame prediction information. The intra-frame decoder (672) may be configured to receive intra-frame prediction information and generate a prediction result based on the intra-frame prediction information. The residual decoder (673) may be configured to perform inverse quantization to extract dequantized transform coefficients, and process the dequantized transform coefficients to convert the residual from the frequency domain to the spatial domain. The reconstruction module (674) may be configured to combine the residual output by the residual decoder (673) with the prediction result (output by the inter-frame prediction module or the intra-frame prediction module, as the case may be) in the spatial domain to form a reconstructed block, which forms a part of the reconstructed picture as part of the reconstructed video.

[0079] Note that the video encoders (203), (403) and (503) and the video decoders (210), (310) and (610) may be implemented using any suitable technology. In some example embodiments, the video encoders (203), (403) and (503) and the video decoders (210), (310) and (610) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (203), (403) and (503) and the video decoders (210), (310) and (610) may be implemented using one or more processors executing software instructions. And (810) may be implemented using one or more processors executing software instructions.

[0080] In some example implementations, loop filters may be included in the encoder and decoder to reduce coding artifacts and improve the quality of the decoded picture. For example, loop filter 356 may be included as Figure 3 For another example, the loop filter may be Figure 4The embedded decoder unit 433 in the encoder 420 of the video block is a part of the decoder 420. These filters are called loop filters because they are included in the decoding loop for the video block in the decoder or encoder. Each loop filter can be associated with one or more filter parameters. Such filter parameters can be predefined or can be derived by the encoder during the encoding process. These filter parameters (when derived by the encoder) or their indexes (when predefined) can be included in the final bit stream in coded form. Then, the decoder can parse these filter parameters from the bit stream during decoding and perform loop filtering based on the parsed filter parameters.

[0081] Various loop filters can be used to reduce coding artifacts and improve decoded video quality in different aspects. Such loop filters may include, but are not limited to, one or more deblocking filters, adaptive loop filters (AdaptiveLoop Filter, ALF), cross-component adaptive loop filters (Cross-Component Adaptive LoopFilter, CC-ALF), Constrained Directional Enhancement Filter (Constrained Directional Enhancement Filter, CDEF), Sample Adaptive Offset (Sample Adaptive Offset, SAO) filter, Cross-Component Sample Offset (Cross-Component Sample Offset, CCSO) filter and Local Sample Offset (Local Sample Offset, LSO) filter. These filters may or may not be interdependent. They may be arranged in any suitable order compatible with their interdependencies (if any) in the decoding loop of the decoder or encoder. These various loop filters are described in more detail in the following disclosure.

[0082] The encoder / decoder may apply an adaptive loop filter (ALF) with block-based filter adaptation to reduce artifacts. ALF is adaptive in the sense that the filter coefficients / parameters or their indices are signaled in the bitstream and can be designed based on the image content and the distortion of the reconstructed picture. ALF may be applied to reduce the distortion introduced by the encoding process and improve the quality of the reconstructed image.

[0083] For the luma component, one of a plurality of filters (e.g., 25 filters) may be selected for a luma block (e.g., a 4×4 luma block), for example, based on the direction and activity of the local gradient. The filter coefficients of these filters may be derived by the encoder during the encoding process and signaled to the decoder in the bitstream.

[0084] The ALF may have any suitable shape and size. Figure 7 In the example of ALF (710), the ALF may have a diamond shape, such as a 5×5 diamond shape for ALF (710) and a 7×7 diamond shape for ALF (711). In ALF (710), thirteen (13) elements may be used for the filtering process and form a diamond shape. For these 13 elements, seven values ​​(e.g., C0 to C6) may be used and arranged in the example manner shown. In ALF (711), twenty-five (25) elements may be used for the filtering process and form a diamond shape. Thirteen (13) values ​​(e.g., C0 to C12) may be used for 25 elements in the example manner shown.

[0085] Reference Figure 7 In some examples, an ALF filter having one of the two diamond shapes (710) to (711) may be selected to process a luma block or a chroma block. For example, a 5×5 diamond shape filter (710) may be applied to a chroma component (e.g., a chroma block, a chroma CB), and a 7×7 diamond shape filter (711) may be applied to a luma component (e.g., a luma block, a luma CB). Other suitable shapes and sizes may be used in the ALF. For example, a 9×9 diamond shape filter may be used.

[0086] The filter coefficient at the position indicated by the value (e.g., C0 to C6 in (710) or C0 to C12 in (711)) may be non-zero. In addition, when the ALF includes a clipping function, the clipping value at the position may be non-zero. The clipping function may be used to limit the upper limit of the filter value in the luminance block or the chrominance block.

[0087] In some implementations, a particular ALF to be applied to a particular block having a luma component may be based on a classification of the luma block. For block classification of luma components, a 4×4 block (or luma block, luma CB) may be classified or categorized into one of a plurality of (e.g., 25) categories corresponding to, for example, 25 different ALFs (e.g., 25 ALFs out of 7×7 ALFs with different filter coefficients). The classification index C may be based on a quantized value of a directionality parameter D and an activity value A. Use formula (1) to derive .

[0088] In order to calculate the directivity parameter D and the quantization value The gradients gv, gv in the vertical direction, gv in the horizontal direction, and gv in the two diagonal directions (e.g., d1 and d2) can be calculated using a 1-dimensional Laplace as follows: h , g d1 and g d2 . Where the indices i and j refer to the coordinates of the top left sample within the 4×4 block, and R(k,l) indicates the reconstructed sample at coordinate (k,1). The directions (eg, d1 and d2) refer to the 2 diagonal directions.

[0089] In order to reduce the complexity of the above block classification, a subsampled 1-D Laplacian calculation may be applied. FIG. 8A to FIG. 8D The calculation methods for the vertical direction ( Fig. 8A ), horizontal direction( Figure 8B ) and two diagonal directions d1( Figure 8C ) and d2( Fig.8D )’s gradient g v , g h , g d1 and g d2 An example of subsampling positions for . Fig. 8A The label "V" shows the method used to calculate the vertical gradient g. v The sub-sampling position of Figure 8B The label "H" shows the calculation method for the horizontal gradient g. h The sub-sampling position of Figure 8C The label "D1" shows the calculation method for the diagonal gradient g of d1. d1 The sub-sampling position of Fig.8D The label "D2" shows the calculation method used to calculate the diagonal gradient g of d2. d2 The subsampling position. Fig. 8A and Figure 8B It is shown that the same sub-sampling position can be used for gradient calculations in different directions. In some other implementations, different sub-sampling schemes can be used for all directions. In some other implementations, different sub-sampling schemes can be used for different directions.

[0090] Horizontal and vertical gradients g v and g h The maximum value of and minimum value Can be set to: The gradient g in the two diagonal directions d1 and g d2 The maximum value of and minimum value Can be set to: The directivity parameter D can be derived based on the above value and the following two thresholds t1 and t2. Step 1. If (1) And (2) is true, D is set to 0. Step 2. If Then proceed to step 3; otherwise proceed to step 4. Step 3. If Then set D to 2; otherwise set D to 1. Step 4. If Then set D to 4; otherwise set D to 3.

[0091] In other words, if Fig. 8E As shown, the directionality parameter D is represented by several discrete levels and is determined based on the gradient value distribution of the luminance block between the horizontal and vertical directions and between the two diagonal directions.

[0092] The activity value A can be calculated as: Therefore, the activity value A represents a comprehensive measure of the horizontal and vertical 1-dimensional Laplace. The activity value A for the luminance block can be further quantized to a range of, for example, 0 to 4, inclusive, and the quantized value is represented as

[0093] For the luma component, the classification index C calculated as above may then be used to select one of multiple classes (e.g., 25 classes) of a diamond-shaped ALF filter. In some implementations, for the chroma components in a picture, block classification may not be applied, and thus a single set of ALF coefficients may be applied for each chroma component. In such implementations, although there may be multiple sets of ALF coefficients available for chroma components, the determination of the ALF coefficients may not depend on any classification of the chroma blocks.

[0094] A geometric transformation may be applied to the filter coefficients and the corresponding filter clipping values ​​(also referred to as clipping values). Before filtering a block (e.g., a 4×4 luma block), for example, depending on the gradient values ​​(e.g., g v , g h , g d1 and / or d2), a geometric transformation such as rotation or diagonal and vertical flipping may be applied to the filter coefficients f(k,l) and the corresponding filter clipping values ​​c(k,l). Applying a geometric transformation to the filter coefficients f(k,l) and the corresponding filter clipping values ​​c(k,l) may be equivalent to applying a geometric transformation to the samples in the region supported by the filter. A geometric transformation may make different blocks to which the ALF is applied more similar by aligning their respective directionality.

[0095] Three geometric transformation options including diagonal flipping, vertical flipping and rotation can be performed as described in equations (9) to (11) respectively. f D (k,l)=f(l,k),c D (k, l) = c(l, k), formula (9) f v (k, l) = f(k, Kl-1), c V (k, l) = c(k, Kl-1) Formula (10) f R (k, l) = f(Kl-1, k), c R (k,l)=c(Kl-1,k) Formula (11) Where K represents the size of the ALF or filter, and 0≤k, 1≤K-1 are the coordinates of the coefficients. For example, position (0, 0) is at the upper left corner of the filter f or the clipping value matrix (or clipping matrix) c, and position (K-1, K-1) is at the lower right corner. Depending on the gradient values ​​calculated for the block, a transform may be applied to the filter coefficients f(k, l) and the clipping values ​​c(k, l). An example of the relationship between the transform and the four gradients is summarized in Table 1. Table 1: Mapping of gradients computed for a block to transformations Gradient Value Transform <![CDATA[g d2 <g d1 And g h <g v ]]> No change <![CDATA[g d2 <g d1 And g v <g h ]]> Diagonal Flip <![CDATA[g d1 <g d2 And g h <g v ]]> Flip vertically <![CDATA[g d1 <g d2 And g v <g h ]]> Rotation

[0096] In some embodiments, the ALF filter parameters derived by the encoder may be signaled in an Adaptation Parameter Set (APS) for a picture. In the APS, one or more groups (e.g., up to 25 groups) of luminance filter coefficients and clipping value indices may be signaled. They may be indexed in the APS. In an example, one of the one or more groups may include luminance filter coefficients and one or more clipping value indices. One or more groups (e.g., up to 8 groups) of chrominance filter coefficients and clipping value indices may be derived and signaled by the encoder. In order to reduce signaling overhead, filter coefficients of different classifications of luminance components (e.g., with different classification indices) may be merged. In a slice header, the index of the APS for the current slice may be signaled. In another example, the signaling of the ALF may be based on the CTU.

[0097] In an embodiment, a clipping value index (also referred to as a clipping index) may be decoded according to the APS. The clipping value index may be used to determine the corresponding clipping value, for example, based on the relationship between the clipping value index and the corresponding clipping value. The relationship may be predefined and stored in a decoder. In an example, the relationship is described by one or more tables such as a table of clipping value indices and corresponding clipping values ​​for a luminance component (e.g., for a luminance CB) and a table of clipping value indices and corresponding clipping values ​​for a chrominance component (e.g., for a chrominance CB). The clipping value may depend on the bit depth B. The bit depth B may refer to an internal bit depth, a bit depth of a reconstructed sample in a CB to be filtered, etc. In some examples, equation (12) may be used to obtain a table of clipping values ​​(e.g., for luminance and / or for chrominance). AlfClip = {round(2 B-α*2 )For n∈[0..N-1]}, Formula (12) Wherein, AlfClip is the clipping value, B is the bit depth (e.g., bitDepth), N (e.g., N=4) is the number of allowed clipping values, and α is a predefined constant value. In the example, α is equal to 2.35. n is the clipping value index (also called clipping index or clipIdx). Table 2 shows an example of a table obtained using equation (12), where N=4. The clipping index n can be 0, 1, 2, and 3 (up to N-1) in Table 2. Table 2 can be used for luminance blocks or chrominance blocks. Table 2 - AlfClip can depend on bit depth B and clipIdx

[0098] In the slice header of the current slice, one or more APS indexes (e.g., up to 7 APS indexes) may be signaled to specify a luma filter set that may be used for the current slice. The filtering process may be controlled at one or more suitable levels such as picture level, slice level, CTB level, etc. In an example embodiment, the filtering process may be further controlled at the CTB level. A flag indicating whether the ALF is applied to the luma CTB may be signaled. The luma CTB may select a filter set from a plurality of fixed filter banks (e.g., 16 fixed filter banks) and (one or more) filter banks signaled in the APS (e.g., up to 25 filters, which are derived by the encoder as described above and are also referred to as (one or more) signaled filter banks). A filter bank index may be signaled for the luma CTB to indicate the filter bank to be applied (e.g., a filter bank in a plurality of fixed filter banks and (one or more) signaled filter banks). A plurality of fixed filter banks may be predefined and hard-coded in the encoder and decoder and may be referred to as predefined filter banks. Therefore, there is no need to signal predefined filter coefficients.

[0099] For chroma components, the APS index may be signaled in the slice header to indicate the chroma filter set to be used for the current slice. At the CTB level, if there is more than one chroma filter set in the APS, the filter set index may be signaled for each chroma CTB.

[0100] The filter coefficients may be quantized with a norm equal to 128. To reduce multiplication complexity, bitstream consistency may be applied so that the coefficient values ​​of non-center positions may be in the range of -27 to 27-1 inclusive. In an example, the center position coefficient is not signaled in the bitstream and may be considered equal to 128.

[0101] In some embodiments, the syntax and semantics of the clip index and clip value are defined as follows: alf_luma_clip_idx[sfIdx][j] can be used to specify the clip index of the clip value to be used before multiplying the jth coefficient of the signaled luminance filter indicated by sfIdx. The requirement for bitstream conformance may include that the value of alf_luma_clip_idx[sfIdx][j] where sfIdx=0 to alf_luma_num_filters_signalled_minus1 and j=0 to 11 should be in the range of, for example, 0 to 3, inclusive.

[0102] The luma filter clipping value AlfClipL[adaptation_parameter_set_id] with elements AlfClipL[adaptation_parameter_set_id][filtIdx][j] may be derived as specified in Table 2 based on bitDepth being set equal to BitDepthY and clipIdx being set equal to alf_luma_clip_idx[alf_luma_coeff_delta_idx[filtIdx]][j], where filtIdx=0 to NumAlfFilters-1 and j=0 to 11.

[0103] Alf_chroma_clip_idx[altIdx][j] may be used to specify the clip index of the clip value to be used before multiplying the j-th coefficient of the alternative chroma filter with index altIdx. Bitstream conformance requirements may include that the value of alf_chroma_clip_idx[altIdx][j] shall be in the range of 0 to 3, inclusive, where altIdx = 0 to alf_chroma_num_alt_filters_minus1, j = 0 to 5.

[0104] The chroma filter clipping value AlfClipC[adaptation_parameter_set_id][altIdx] with elements AlfClipC[adaptation_parameter_set_id][altIdx][j] may be derived as specified in Table 2, based on bitDepth being set equal to BitDepthC and clipIdx being set equal to alf_chroma_clip_idx[altIdx][j], where altIdx = 0 to alf_chroma_num_alt_filters_minus1, j = 0 to 5.

[0105] In an embodiment, the filtering process can be described as follows. At the decoder side, when ALF is enabled for CTB, the sample R(i, j) within the CU (or CB) of the CTB can be filtered, thereby using equation (13) to obtain the filtered sample value R'(i, j) as shown below. In the example, each sample in the CU is filtered. R′(i,j)=R(i,j)+((Σ k≠0 Σ l≠0 f(k,l)×K(R(i+k,j+l)-R(i,j),c(k,l))+64)>>7) Formula (13) Where f(k, l) represents the decoded filter coefficients, K(x, y) is the clipping function, and c(k, l) represents the decoded clipping parameter (or clipping value). The variables k and l can vary between -L / 2 and L / 2, where L represents the filter length (e.g., for Figure 7 , L=5 and 7). The clipping function K(x,y)=min(y,max(-y,x)) corresponds to the clipping function Clip3(-y,y,x). By including the clipping function K(x,y), the loop filtering method (e.g., ALF) becomes a nonlinear process and can be called a nonlinear ALF.

[0106] The selected clipping value may be encoded in the "alf_data" syntax element as follows: A suitable coding scheme, such as a Golomb coding scheme, may be used to encode the clipping index corresponding to the selected clipping value such as shown in Table 2. The coding scheme may be the same as the coding scheme used to encode the filter bank index.

[0107] In an embodiment, a virtual boundary filtering process may be used to reduce the line buffer requirements of the ALF. Thus, a modified block classification and filtering may be employed for samples near a CTU boundary (e.g., a horizontal CTU boundary). The virtual boundary (930) may be defined as a horizontal CTU boundary (920) shifted by “N”. 样本 "The lines generated by the samples are as follows Fig.9A As shown, where N 样本 Can be a positive integer. In the example, for the brightness component, N 样本 is equal to 4, while for the chrominance component, N 样本 Equals 2.

[0108] Reference Fig.9A , a modified block classification may be applied to the luma component. In the example, for the 1-dimensional Laplacian gradient calculation of the 4×4 block (910) above the virtual boundary (930), only samples above the virtual boundary (930) are used. Similarly, referring to Fig. 9B , for the 1-D Laplacian gradient calculation of the 4×4 block (911) below the virtual boundary (931) shifted from the CTU boundary (921), only the samples below the virtual boundary (932) are used. The quantization of the activity value A can be scaled accordingly by considering the reduced number of samples used in the 1-D Laplacian gradient calculation.

[0109] For the filtering process, symmetric padding operations at virtual boundaries may be used for both luma and chroma components. FIG. 10A to FIG. 10FAn example of such modified ALF filtering for the luma component at a virtual boundary is shown. When a filtered sample is below a virtual boundary, neighboring samples above the virtual boundary may be filled. When a filtered sample is above a virtual boundary, neighboring samples below the virtual boundary may be filled. Fig. 10A , the neighboring sample C0 can be filled with the sample C2 located below the virtual boundary (1010). Fig. 10B , the neighboring sample C0 can be filled with the sample C2 located above the virtual boundary (1020). Fig. 10C , neighboring samples C1 to C3 can be filled with samples C5 to C7 located below the virtual boundary (1030), respectively. Sample C0 can be filled with sample C6. Fig. 10D , neighboring samples C1 to C3 can be filled with samples C5 to C7 located above the virtual boundary (1040), respectively. Sample C0 can be filled with sample C6. Fig.10E , neighboring samples C4 to C8 can be filled with samples C10, C11, C12, C11, and C10 located below the virtual boundary (1050), respectively. Samples C1 to C3 can be filled with samples C11, C12, and C11. Sample C0 can be filled with sample C12. Fig.12 F, neighboring samples C4 to C8 can be filled with samples C10, C11, C12, C11, and C10 located above the virtual boundary (1060), respectively. Samples C1 to C3 can be filled with samples C11, C12, and C11. Sample C0 can be filled with sample C12.

[0110] In some examples, when the sample(s) and the neighboring sample(s) are located to the left (or right) and to the right (or left) of the virtual boundary, the above description may be appropriately adjusted.

[0111] The picture quadtree partitioning aligned with the largest coding unit (LCU) can be used. In order to improve the coding efficiency, an adaptive loop filter based on the coding unit synchronized picture quadtree can be used in video coding. In the example, the luma picture can be partitioned into multiple multi-level quadtree partitions, and each partition boundary is aligned with the boundary of the largest coding unit (LCU). Each partition can have a filtering process, and therefore can be called a filter unit (Filter Unit or Filtering Unit, FU).

[0112] The example 2-pass encoding process is described as follows. At the first pass, the quadtree segmentation pattern and the best filter (or optimal filter) for each FU can be determined. During the decision process, the filtering distortion can be estimated by fast filtering distortion estimation (FFDE). According to the determined quadtree segmentation pattern and the selected filter of the FU (e.g., all FUs), the reconstructed picture can be filtered. At the second pass, CU synchronous ALF on / off control can be performed. According to the ALF on / off result, the first filtered picture is partially restored by the reconstructed picture.

[0113] A top-down segmentation strategy can be adopted to divide the picture into multiple levels of quadtree partitions by using a rate-distortion criterion. Each partition can be called a FU. The segmentation process can align the quadtree partitions with the LCU boundaries, such as Fig.11 shown. Fig.11 An example of LCU-aligned picture quadtree partitioning according to an embodiment of the present disclosure is shown. In the example, the encoding order of FUs follows the z-scan order. For example, referring to Fig.11 , the picture is partitioned into ten FUs (e.g., FU0-FU9, the partition depth is 2, where FU0, FU1 and FU9 are first-level FUs, FU2, FU7 and FU8 are second-depth level FUs, and FU3 to FU6 are third-depth level FUs), and the encoding order is from FU0 to FU9, for example, FU0, FU1, FU2, FU3, FU4, FU5, FU6, FU7, FU8 and FU9.

[0114] To indicate the picture quadtree partitioning mode, a partitioning flag ("1" indicates quadtree partitioning, and "0" indicates no quadtree partitioning) may be encoded and transmitted in z-scan order. Fig.12 The embodiment according to the present disclosure corresponds to Fig.11 The quadtree partitioning mode. Fig.12 As shown in the example in , the quadtree split flag is encoded in z-scan order.

[0115] The filter of each FU can be selected from two filter groups based on the rate-distortion criterion. The first group can have 1 / 2 symmetrical square shape and rhombus shape filters newly derived for the current FU. The second group can come from the delay filter buffer. The delay filter buffer can store filters previously derived for FUs in previous pictures. The filter with the minimum rate-distortion cost in the two filter groups can be selected for the current FU. Similarly, if the current FU is not the smallest FU and can be further divided into four sub-FUs, the rate-distortion costs of the four sub-FUs can be calculated. By recursively comparing the rate-distortion costs of the segmented and unsegmented cases, the picture quadtree segmentation pattern (in other words, whether the quadtree segmentation of the current FU should be stopped) can be determined.

[0116] In some examples, the maximum quadtree segmentation level or depth can be limited to a predefined number. For example, the maximum quadtree segmentation level or depth can be 2, and therefore the maximum number of FUs can be 16 (or the 4th power of the maximum depth number). During the quadtree segmentation decision, the correlation values ​​for deriving the Wiener coefficients of the 16 FUs (minimum FUs) at the bottom quadtree level can be reused. The remaining FUs can derive the Wiener filters of the remaining FUs based on the correlation of the 16 FUs at the bottom quadtree level. Therefore, in the example, there is only one frame buffer access for deriving the filter coefficients of all FUs.

[0117] After the quadtree partitioning mode is determined, in order to further reduce the filtering distortion, CU synchronous ALF on / off control can be performed. By comparing the filtering distortion and non-filtering distortion, the leaf CU can explicitly switch the ALF on / off in the corresponding local area. The coding efficiency can be further improved by redesigning the filter coefficients according to the ALF on / off results. In the example, the redesign process requires additional frame buffer access. Therefore, in some examples, such as in the Coding Unit Synchronous PictureQuadtree-Based Adaptive Loop Filter (CS-PQALF) encoder design, no redesign process is required after the CU synchronous ALF on / off decision to minimize the number of frame buffer accesses.

[0118] The cross-component filtering process may apply a cross-component filter, such as a cross-component adaptive loop filter (CC-ALF). The cross-component filter may refine a chroma component (e.g., a chroma CB corresponding to the luma CB) using a luma sample value of a luma component (e.g., a luma CB). In an example, the luma CB and the chroma CB are included in a CU.

[0119] Fig.13A cross component filter (e.g., CC-ALF) for generating chrominance components according to an example implementation of the present disclosure is shown. Fig.13 Filtering processes for a first chroma component (e.g., a first chroma CB), a second chroma component (e.g., a second chroma CB), and a luma component (e.g., a luma CB) are shown. The luma component may be filtered by a sample adaptive offset (SAO) filter (1310) to generate an SAO-filtered luma component (1341). The SAO-filtered luma component (1341) may be further filtered by an ALF luma filter (1316) to become a filtered luma CB (1361) (e.g., "Y").

[0120] The first chroma component may be filtered by the SAO filter (1312) and the ALF chroma filter (1318) to generate a first intermediate component (1352). In addition, the SAO filtered luma component (1341) may be filtered by a cross component filter (e.g., CC-ALF) (1321) for the first chroma component to generate a second intermediate component (1342). Subsequently, a filtered first chroma component (1362) (e.g., 'Cb') may be generated based on at least one of the second intermediate component (1342) and the first intermediate component (1352). In an example, the filtered first chroma component (1362) (e.g., 'Cb') may be generated by combining the second intermediate component (1342) and the first intermediate component (1352) using an adder (1322). An example cross component adaptive loop filtering process for the first chroma component may therefore include steps performed by the CC-ALF (1321) and steps performed by, for example, the adder (1322).

[0121] The above description may be applicable to the second chroma component. The second chroma component may be filtered by the SAO filter (1314) and the ALF chroma filter (1318) to generate a third intermediate component (1353). In addition, the SAO filtered luminance component (1341) may be filtered by a cross component filter (e.g., CC-ALF) (1331) for the second chroma component to generate a fourth intermediate component (1343). Subsequently, a filtered second chroma component (1363) (e.g., 'Cr') may be generated based on at least one of the fourth intermediate component (1343) and the third intermediate component (1353). In an example, a filtered second chroma component (1363) (e.g., 'Cr') may be generated by combining the fourth intermediate component (1343) and the third intermediate component (1353) using an adder (1332). In an example, the cross component adaptive loop filtering process for the second chroma component may therefore include filtering by the CC-ALF filter. (1331) and steps performed by, for example, an adder (1332).

[0122] The cross component filter (e.g., CC-ALF (1321), CC-ALF (1331)) can operate by applying a linear filter having any suitable filter shape to the luma component (or luma channel) to refine each chroma component (e.g., first chroma component, second chroma component). The CC-ALF exploits the correlation between color components to reduce coding distortion in another color component based on samples from one color component.

[0123] Fig.14 An example of a CC-ALF filter (1400) according to an embodiment of the present disclosure is shown. The filter (1400) may include non-zero filter coefficients and zero filter coefficients. The filter (1400) has a diamond shape (1420) formed by filter coefficients (1410) (indicated by circles with black fill). In the example, the non-zero filter coefficients in the filter (1400) are included in the filter coefficients (1410), and the filter coefficients not included in the filter coefficients (1410) are zero. Therefore, the non-zero filter coefficients in the filter (1400) are included in the diamond shape (1420), and the filter coefficients not included in the diamond shape (1420) are zero. In the example, the number of filter coefficients of the filter (1400) is equal to the number of filter coefficients (1410), which is Fig.14 In the example shown, this is 18.

[0124] CC-ALF may include any suitable filter coefficients (also referred to as CC-ALF filter coefficients). Return to reference Fig.13 , CC-ALF (1321) and CC-ALF (1331) may have the same filter shape, e.g. Fig.14 The diamond shape (1420) shown in FIG. 14 and the same number of filter coefficients. In the example, the values ​​of the filter coefficients in CC-ALF (1321) are different from the values ​​of the filter coefficients in CC-ALF (1331).

[0125] Typically, the filter coefficients in CC-ALF (eg, non-zero filter coefficients as derived by the encoder) may be transmitted, for example, in the APS. In an example, the filter coefficients may be multiplied by a factor (eg, 2 10) and may be rounded for fixed point representation. The application of CC-ALF may be controlled for variable block sizes and signaled by a context coded flag (e.g., a CC-ALF enable flag) received for each sample block. Context coded flags such as the CC-ALF enable flag may be signaled at any suitable level, such as the block level. For each chroma component, the block size and the CC-ALF enable flag may be received at the slice level. In some examples, block sizes (in chroma samples) of 16×16, 32×32, and 64×64 may be supported.

[0126] In an example, the syntax changes of CC-ALF are described in Table 3 as follows.

[0127] The semantics of the CC-ALF related syntax in the above example can be described as follows:

[0128] alf_ctb_cross_component_cb_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] equal to 0 may indicate that the cross-component Cb filter is not applied to the Cb color component sample block at luma position (xCtb, yCtb).

[0129] alf_ctb_cross_component_cb_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] not equal to 0 may indicate that the alf_ctb_cross_component_cb_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY]th cross-component Cb filter is applied to the Cb color component sample block at luma position (xCtb, yCtb).

[0130] alf_ctb_cross_component_cr_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] equal to 0 may indicate that the cross-component Cr filter is not applied to the block of Cr color component samples at luma position (xCtb, yCtb).

[0131] alf_ctb_cross_component_cr_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] not equal to 0 may indicate that the alf_ctb_cross_component_cr_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY]th cross-component Cr filter is applied to the Cr color component sample block at luma position (xCtb, yCtb). Table 3: Syntax changes of CC-ALF

[0132] An example of a chroma sample format is described below. In general, a luma block may correspond to one or more chroma blocks, such as two chroma blocks. The number of samples in each of the chroma block(s) may be less than the number of samples in the luma block. The chroma subsampling format (also referred to as the chroma subsampling format, such as specified by chroma_format_idc) may indicate a chroma horizontal subsampling factor (e.g., SubWidthC) and a chroma vertical subsampling factor (e.g., SubHeightC) between each of the chroma block(s) and the corresponding luma block. For a nominal 4 For a (horizontal) by 4 (vertical) blocks, the chroma subsampling scheme may be specified in a 4:x:y format, where x is the horizontal chroma subsampling factor (the number of chroma samples retained in the first row of the block), and y is the number of chroma samples retained in the second row of the block. In an example, the chroma subsampling format may be 4:2:0, indicating that both the chroma horizontal subsampling factor (e.g., SubWidthC) and the chroma vertical subsampling factor (e.g., SubHeightC) are 2, as shown in FIG. Fig.15 A to Fig.15 B. In another example, the chroma subsampling format may be 4:2:2, indicating that the chroma horizontal subsampling factor (e.g., SubWidthC) is 2, and the chroma vertical subsampling factor (e.g., SubHeightC) is 1. In yet another example, the chroma subsampling format may be 4:4:4, indicating that the chroma horizontal subsampling factor (e.g., SubWidthC) and the chroma vertical subsampling factor (e.g., SubHeightC) are 1. Thus, the chroma sample format or type (also referred to as chroma sample position) may indicate the relative position of the chroma samples in the chroma block relative to at least one corresponding luma sample in the luma block.

[0133] Fig.15 A to Fig.15 B shows an exemplary position of chroma samples relative to luma samples according to an embodiment of the present disclosure. Fig.15A, Luma samples (1501) are located in rows (1511) to (1518). Fig.15 The luma sample (1501) shown in A may represent a portion of a picture. In the example, a luma block (e.g., luma CB) includes a luma sample (1501). The luma block may correspond to two chroma blocks having a 4:2:0 chroma subsampling format. In the example, each chroma block includes a chroma sample (1503). Each chroma sample (e.g., chroma sample (1503(1)) corresponds to four luma samples (e.g., luma samples (1501(1) to (1501(4)). In the example, the four luma samples are an upper left sample (1501(1)), an upper right sample (1501(2)), a lower left sample (1501(3)), and a lower right sample (1501(4)). The chroma samples (e.g., (1503(1)) correspond to four luma samples (e.g., luma samples (1501(1) to (1501(4)). In the example, the four luma samples are an upper left sample (1501(1)), an upper right sample (1501(2)), a lower left sample (1501(3)), and a lower right sample (1501(4)). (1))) may be located at a left center position between the upper left sample (1501(1)) and the lower left sample (1501(3)), and the chroma sample type of the chroma block having the chroma sample (1503) may be referred to as chroma sample type 0. Chroma sample type 0 indicates a relative position 0 corresponding to a left center position between the upper left sample (1501(1)) and the lower left sample (1501(3)). Four luma samples (e.g., (1501(1)) to (1501(4))) may be referred to as chroma samples (1503) (1) of the neighboring brightness samples.

[0134] In an example, each chroma block may include a chroma sample (1504). The above description with reference to the chroma sample (1503) may be applicable to the chroma sample (1504), and thus a detailed description may be omitted for the sake of brevity. Each chroma sample in the chroma samples (1504) may be located at a center position of four corresponding luma samples, and a chroma sample type of a chroma block having the chroma sample (150) may be referred to as chroma sample type 1. The chroma sample type 1 indicates a relative position 1 corresponding to a center position of four luma samples (e.g., (1501(1)) to (1501(4))). For example, one of the chroma samples (1504) may be located at a center position of the luma samples (1501(1)) to (1501(4)). (4)) at the center part.

[0135] In an example, each chroma block includes a chroma sample (1505). Each chroma sample in the chroma samples (1505) may be located at an upper left position, co-located with an upper left sample in four corresponding luma samples (1501), and a chroma sample type of a chroma block having the chroma sample (1505) may be referred to as a chroma sample type 2. Therefore, each chroma sample in the chroma samples (1505) is co-located with an upper left sample in four luma samples (1501) corresponding to the corresponding chroma sample. The chroma sample type 2 indicates a relative position 2 corresponding to an upper left position of the four luma samples (1501). For example, one of the chroma samples (1505) may be located at an upper left position of luma samples (1501(1)) to (1501(4)).

[0136] In an example, each chroma block includes a chroma sample (1506). Each chroma sample in the chroma samples (1506) may be located at a top center position between a corresponding top left sample and a corresponding top right sample, and a chroma sample type of a chroma block having the chroma sample (1506) may be referred to as a chroma sample type 3. The chroma sample type 3 indicates a relative position 3 corresponding to a top center position between the top left sample and the top right sample. For example, one of the chroma samples (1506) may be located at a top center position of luma samples (1501(1)) to (1501(4)).

[0137] In an example, each chroma block includes a chroma sample (1507). Each of the chroma samples (1507) may be located at a lower left position, co-located with a lower left sample of four corresponding luma samples (1501), and a chroma sample type of a chroma block having the chroma samples (1507) may be referred to as a chroma sample type 4. Thus, each of the chroma samples (1507) is co-located with a lower left sample of four luma samples (1501) corresponding to the corresponding chroma sample. The chroma sample type 4 indicates a relative position 4 corresponding to a lower left position of the four luma samples (1501). For example, one of the chroma samples (1507) may be located between luma samples (1501(1)) to (1501) (4)) at the lower left position.

[0138] In the example, each chroma block includes a chroma sample (1508). Each chroma sample in the chroma samples (1508) is located at a bottom center position between a bottom left sample and a bottom right sample, and the chroma sample type of the chroma block having the chroma sample (150) may be referred to as a chroma sample type 5. The chroma sample type 5 indicates a relative position 5 corresponding to a bottom center position between a bottom left sample and a bottom right sample in four luma samples (1501). For example, one of the chroma samples (1508) may be located between a bottom left sample and a bottom right sample of luma samples (1501(1)) to (1501)(4)).

[0139] In general, any suitable chroma sample type may be used for a chroma subsampling format. Chroma sample types 0 through 5 provide exemplary chroma sample types described using the chroma subsampling format 4:2:0. Additional chroma sample types may be used for the chroma subsampling format 4:2:0. Furthermore, other chroma sample types and / or variations of chroma sample types 0 through 5 may be used for other chroma subsampling formats, such as 4:2:2, 4:4:4, etc. In an example, a chroma sample type combining chroma samples (1505) and (1507) may be used for the chroma subsampling format 4:2:2.

[0140] In another example, a luma block is considered to have alternating rows, such as rows (1511) to (1512), which respectively include the top two samples (e.g., (1501(1)) to (1501)(2))) and the four luma samples (e.g., (1501(1)) to (1501)(4)) of four luma samples (e.g., (1501(1)) to (1501)(2))). The bottom two samples (e.g., (1501(3)) to (1501(4))) of (1501(1)) to (1501(4))) are located at the current row (e.g., (1511)) and the next row (e.g., (1512)). The upper relative chroma positions 2 to 3 are located in the current row, the upper relative chroma positions 0 to 1 are located between each current row and the corresponding next row, and the upper relative chroma positions 4 to 5 are located in the next row.

[0141] Chroma samples (1503), (1504), (1505), (1506), (1507), or (1508) are located in rows (1551) to (1554) of each chroma block. The specific positions of rows (1551) to (1554) may depend on the chroma sample type of the chroma samples. For example, for chroma samples (1503) to (1504) having corresponding chroma sample types 0 to 1, row (1551) is located between rows (1511) and (1512). For chroma samples (1505) to (1506) having corresponding chroma sample types 2 to 3, row (1551) is co-located with the current row (1511). For chroma samples (1507) to (1508) having corresponding chroma sample types 4 to 5, row (1551) is co-located with the next row (1512). The above description can be appropriately applied to rows (1552) to (1554), and the detailed description is omitted for the sake of brevity.

[0142] Any suitable scanning method may be used to display, store and / or transmit the above Fig.15 A The luminance blocks and (one or more) corresponding chrominance blocks described in . In some example implementations, a line-by-line scan may be used.

[0143] Alternatively, interlaced scanning may be used, such as Fig.15 B. As described above, the chroma subsampling format may be 4:2:0 (e.g., chroma_format_idc is equal to 1). In an example, a variable chroma location type (e.g., ChromaLocType) may indicate a current row (e.g., ChromaLocType is chroma_sample_loc_type_top_field) or a next row (e.g., ChromaLocType is chroma_sample_loc_type_bottom_field). The current row (1511), (1513), (1515), and (1517) and the next row (1512), (1514), (1515), and (1516) may be scanned separately. (1516) and (1518). For example, the current row (1511), (1513), (1515) and (1517) may be scanned first, and then the next row (1512), (1514), (1516) and (1518) may be scanned. The current row may include luma sample (1501) and the next row may include luma sample (1502).

[0144] Similarly, the corresponding chroma blocks can be scanned in an interlaced manner. Rows (1551) and (1553) including chroma samples (1503), (1504), (1505), (1506), (1507), or (1508) without filling can be referred to as the current row (or current chroma row), and rows (1552) and (1554) including chroma samples (1503), (1504), (1505), (1506), (1507), or (1508) with gray filling can be referred to as the next row (or next chroma row). In the example, during interlaced scanning, rows (1551) and (1553) can be scanned first, and then rows (1552) and (1554) can be scanned.

[0145] In addition to the above-mentioned ALF, a constrained directional enhancement filter (Constrained Directional Enhancement Filter, CDEF) can also be used for loop filtering in video coding. The in-loop CDEF can be used to filter out coding artifacts such as quantization ringing artifacts while retaining the details of the image. In some coding techniques, a sample adaptive offset (Sample Adaptive Offset, SAO) algorithm can be used to achieve similar goals by limiting signal offsets for different categories of pixels. Unlike SAO, CDEF is a nonlinear spatial filter. In some examples, the design of the CDEF filter is restricted to be easy to vectorize (ie, it can be implemented by single instruction multiple data (Single Instruction Multiple Data, SIMD) operations), which is not the case for other nonlinear filters such as median filters and bilateral filters.

[0146] The CDEF design is derived from the following observations. In some cases, the amount of ringing artifacts in the encoded image may be approximately proportional to the quantization step size. The minimum detail retained in the quantized image is also proportional to the quantization step size. Therefore, retaining image details will require a smaller quantization step size, which will produce higher undesirable quantization ringing artifacts. Fortunately, for a given quantization step size, the amplitude of the ringing artifact can be smaller than the amplitude of the detail, providing an opportunity to design CDEF to strike a balance between retaining enough detail and filtering out the ringing artifacts.

[0147] CDEF can first identify the direction of each block. CDEF can then adaptively filter along the identified direction and filter to a lesser extent along a direction rotated 45° relative to the identified direction. The filter strength can be explicitly signaled to achieve a high degree of control over blurred details. An effective encoder search can be designed for filter strength. CDEF can be based on two in-loop filters, and the combined filter can be used for video encoding. In some example implementations, (one or more) CDEF filters can be followed by (one or more) deblocking filters for in-loop filtering.

[0148] like Fig.16 As shown, the directional search can operate on reconstructed pixels (or samples), for example, after a deblocking filter. Since the reconstructed pixels are available to the decoder, the direction may not need to be signaled. The directional search can operate on blocks of suitable size (e.g., 8×8 blocks), which are small enough to adequately handle non-straight edges (so that the edges appear straight enough in the filtered block) and large enough to reliably estimate the direction when applied to the quantized image. Having a constant direction over an 8×8 area can make vectorization of the filter easier. For each block, the direction that best matches the pattern in the block can be determined by minimizing a difference metric between the quantized block and each of the fully oriented blocks, such as the Sum of Squared Differences (SSD), Root Mean Square (RMS) error, etc. In the example, the fully oriented blocks (e.g., Fig.16 One of (1620) refers to a block in which all pixels along a line in one direction have the same value. Fig.16 An example of a directional search for an 8×8 block (1610) is shown according to an example implementation of the present disclosure. Fig.16 In the example shown, the 45 degree direction (1623) is selected from the set of directions (1620) because the 45 degree direction (1623) can minimize the error (1640). For example, the error for the 45 degree direction is 12, and is the smallest in the range of errors from 12 to 87 indicated by the row (1640).

[0149] An example nonlinear low-pass directional filter is described in more detail below. Identifying a direction can help align the filter taps along the identified direction to reduce ringing artifacts while preserving directional edges or patterns. However, in some examples, directional filtering alone may not sufficiently reduce ringing artifacts. It is desirable to use additional filter taps on pixels that are not along a primary direction (e.g., the identified direction). To reduce the risk of blurring, additional filter taps may be handled more conservatively. Therefore, CDEF may define primary taps and secondary taps. In some example implementations, a complete two-dimensional (2-D) CDEF filter may be represented as:

[0150] In formula (14), D represents the damping parameter, S (p) and S (s) denote the strength of the primary tap and the secondary tap, respectively, and the function round(·) can round the relationship relative to zero, and Denotes the filter weight, and f(d, S, D) denotes a constraint function that operates on the difference d (e.g., d = x(m, n) - x(i, j)) between the filtered pixel (e.g., x(i, j)) and each of the neighboring pixels (e.g., x(m, n)). When the difference is small, f(d, S, D) can be equal to the difference d (e.g., f(d, S, D) = d), and thus the filter can behave as a linear filter. When the difference is large, f(d, S, D) can be equal to 0 (e.g., f(d, S, D) = 0), which effectively ignores the filter taps.

[0151] As another in-loop processing component, a set of in-loop recovery schemes can be used in video encoding post-deblocking to generally perform denoising and improve edge quality in addition to the deblocking operation. The set of in-loop recovery schemes can be switched within a frame (or picture) according to tiles of appropriate size. Some examples of in-loop recovery schemes are described below based on separable symmetric Wiener filters and dual self-guided filters with subspace projection. Since content statistics can vary significantly within a frame, filters can be integrated in a switchable framework, where different filters can be triggered in different areas of the frame.

[0152] An example separable symmetric Wiener filter is described below. The Wiener filter can be used as one of the switchable filters. Each pixel (or sample) in the degraded frame can be reconstructed as a non-causal filtered version of the pixels within a w×w window around the pixel, where w=2r+1 and is odd for integer r. The 2-dimensional filter taps can be composed of a number of pixels with w 2The filter parameters are represented by a column vectorized form of F with 1 × 1 elements, and direct linear minimum mean square error (LMMSE) optimization can result in the filter parameters being F = H - 1 M is given, where H is equal to E[XX T ] and is the autocovariance of x, where x is the value of w in the w×w window around the pixel 2 The column vectorized version of samples, and where M is equal to E[YX T ], representing the cross-correlation of x with the scalar source sample y to be estimated. The encoder may be configured to estimate H and M based on the implementation in the deblocked frame and source, and send the resulting filter F to the decoder. However, in some example implementations, before transmitting w 2 A large bit rate cost may be incurred when there are 10 taps. In addition, non-separable filtering may make decoding overly complicated. Therefore, multiple additional constraints can be imposed on the properties of F. For example, F can be constrained to be separable so that the filtering can be implemented as separable horizontal w-tap convolution and vertical w-tap convolution. In the example, each of the horizontal filter and the vertical filter is constrained to be symmetrical. In addition, in some example implementations, it can be assumed that the sum of the horizontal filter coefficients and the vertical filter coefficients is 1.

[0153] Double self-guided filtering with subspace projection can also be used as one of the switchable filters for in-loop recovery and is described below. In some example implementations, guided filtering can be used for image filtering, where a local linear model is used to calculate the filtered output y from the unfiltered samples x. The local linear model can be written as y=Fx+G Formula (15) Wherein, F and G can be determined based on the statistical results of the degraded image and the guidance image (also referred to as the guide image) near the filtered pixel. If the guidance image is the same as the degraded image, the resulting self-guided filtering can have an edge-preserving smoothing effect. According to some aspects of the present disclosure, the specific form of the self-guided filtering can depend on two parameters: the radius r and the noise parameter e, and are listed as follows:

[0154] 1. Get the mean μ and variance σ of the pixels in the (2r+1)×(2r+1) window around each pixel 2 For example, to obtain the mean μ and variance σ of the pixels 2 This can be effectively achieved by using box filtering based on panoramic imaging.

[0155] 2. Calculate the parameters f and g for each pixel based on equation (16) f=σ 2 / (σ 2 +e); g = (1-f)μ Formula (16)

[0156] 3. Calculate F and G for each pixel as the average of the values ​​of parameters f and g in a 3×3 window around the pixel for use.

[0157] The dual self-guided filtering can be controlled by the radius r and the noise parameter e, where a larger radius r may imply a higher spatial variance, while a higher noise parameter e may imply a higher range variance.

[0158] Fig.17 An example of subspace projection according to an example implementation of the present disclosure is shown. Fig.17 In the example shown, subspace projection can use cheap restorations X1 and X2 to produce a final restoration X that is closer to the source Y. f Even if the cheap restorations X1 and X2 are not close to the source Y, appropriate multipliers {α, β} can make the cheap restorations X1 and X2 closer to the source Y if the cheap restorations X1 and X2 move in the right direction. For example, the final restoration X f It can be obtained based on the following formula (17). X f =X+α(X1-X)+β(X2-X) Formula ((17)

[0159] In addition to the above-mentioned deblocking filter, ALF, CDEF and loop recovery, a loop filtering method called a cross-component sample offset (CCSO) filter or CCSO can be implemented in the loop filtering process to reduce the distortion of the reconstructed samples (also called reconstruction samples). The CCSO filter can be placed at any position in the loop filtering stage. Fig.18 An example of a CCSO filter associated with a deblocking filter, a CDEF filter, and an LR filter is shown in FIG. In the CCSO filtering process, a nonlinear mapping may be used to determine an output offset based on the processed input reconstructed samples of the first color component. In the CCSO filtering process, the output offset may be added to the reconstructed samples of the second color component.

[0160] The input reconstruction samples can be from the first color component located in the filter support region, such as Fig.19 shown. Specifically, Fig.19 An example of a filter support region in a CCSO filter according to an embodiment of the present disclosure is shown. The filter support region may include four reconstruction samples: p0 and p1. Fig.19 In the example of , the two input reconstruction samples are located on both sides of the center sample in the vertical direction. In the example, the center sample (denoted by rl) in the first color component (e.g., the luminance component) and the sample to be filtered (denoted by rc) in the second color component (e.g., the chrominance component) are co-located. When processing the input reconstruction samples, the following steps may be applied:

[0161] Step 1: Calculate the delta values ​​(eg, differences) between the four reconstructed samples p0 and p1 and the center sample rl, and denote them as m0 and m1, respectively. For example, the delta value between p0 and rl is m0.

[0162] Step 2: The incremental values ​​m0 to m1 may be further quantized into multiple (e.g., 2) discrete values. For example, for m0 and m1, the quantized values ​​may be represented as d0 and d1, respectively. In the example, the quantized value of each of d0 and d1 may be -1, 0, or 1 based on the following quantization process: di = -1, if mi < -N; Formula (18) di = 0, if -N <= mi <= N; Formula (19) di = 1, if mi > N. Formula (20) Wherein, N is the quantization step size, example values ​​of N are 4, 8, 12, 16, etc., di and mi refer to the quantization value and increment value respectively, where i is 0, 1, 2 or 3.

[0163] The quantized values ​​d0 to d3 can be used to identify combinations of nonlinear mappings. Fig.19 In the example shown, the CCSO filter has two filter inputs d0 to d1, and each filter input can have one of three quantized values ​​(e.g., -1, 0, and 1), and thus the total number of combinations is 9 (e.g., 3 2 , the number of quantized values ​​raised to the power of the number of difference values). An example of 9 combinations with offset values ​​is shown as a lookup table (LUT) in Table 4 below. Table 4 Example LUT used in CCSO

[0164] The last column may represent the output offset value for each combination that may be looked up according to the increment. The output offset value may be an integer, such as 0, 1, -1, 3, -3, 5, -5, -7, etc. The first column represents the index assigned to these combinations of quantization values ​​d0 and d1. The middle column represents all possible combinations of quantization values ​​(with three possible quantization levels) d0 and d1. The offset column may include the actual offset value. Alternatively, there may be a finite number of allowed offset values, and the offset column in Table 4 may include an index to the allowed offset values. Therefore, the terms offset value and index may be used interchangeably.

[0165] The final filtering process of the CCSO filter can be applied as follows: f′=clip(f+s), formula (21) where f is the reconstructed sample to be filtered and s is the sample from Fig.21 A to Fig.21 The output offset value retrieved by C. In the example shown in equation (21), the filtered sample value f' of the reconstructed sample f to be filtered may be further clipped within a range associated with the bit depth.

[0166] like Fig.19 As shown, the example CCSO filtering of the reconstructed sample rc in the second color to be filtered corresponding to the sample c of the first color using p0 and p1 of the first color can be called a 3-tap CCSO filter design. Alternatively, other CCSO designs with different numbers of filter taps can be used. For example, two more taps can be added in the horizontal direction to make it a 5-tap filter (4 differential inputs), and with the same quantization level as above, the number of incremental combinations can be 81 (3 4 ).

[0167] Fig. 20 1 shows example implementations of various 3-tap CCSO filter shapes according to embodiments of the present disclosure. The term CCSO filter shape is used to refer to the number of taps and the location of the taps used for CCSO filtering. Multiple filter shapes for a particular number of taps may be predefined as CCSO filter options. For example, in Fig. 20 In , for 3-tap CCSO filtering, any one of 6 different example filter shapes can be defined. Each of the filter shapes can define the positions of three reconstruction samples (also referred to as three taps) in the first component (also referred to as the first color component). The three reconstruction samples may include a center sample (denoted as c) and two symmetrically positioned samples, such as Fig. 20 In the example, the reconstructed sample to be filtered in the second color component is co-located with the center sample c. For the purpose of clarity, Fig. 20The reconstructed samples to be filtered in the second color component are not shown.

[0168] For CCSO filtering, as described above, each filter essentially corresponds to a LUT. The number of entries (rows) in the LUT is determined by the number of increments combined with quantization levels, as shown in Table 4 above. A CCSO filter may be associated with one or more CCSO filters or filter parameters, including but not limited to filter shape, quantization step size (and quantization level), and number of bands. Alternatively, CCSO filters or filter parameters may be referred to as CCSO parameters.

[0169] As mentioned above, the filter shape parameters represent the number of taps and tap positions for the CCSO filter. The quantization step size represents how the increments described above are discretized. The quantization step size determines the number of quantization levels (giving a range of pixel values). The quantization level determines the number of discrete values ​​allowed for the increments.

[0170] In other words, the quantization step size and the number of quantization levels may be related or interrelated. For example, the range of incremental values ​​may be discretized according to the quantization step size. The quantization levels may be spaced at least equally between the levels. The quantized end segments in the incremental value range may match the quantization step size, or may not match the quantization step size. For example, the quantization step size may be 8, 16, 32, or 64. A set of predefined available quantization step sizes may be predefined. These available quantization step sizes may be identified by the encoder and decoder by their indexes.

[0171] The number of band parameters specifies the number of intensity (or pixel amplitude) bands into which the pixel value range is divided. Specifically, CCSO filtering can be further improved by including band features for deriving CCSO offsets (or lookup tables). Band features and edge features can be used jointly to derive offsets, where, as described above, edge features are derived using, for example, delta values ​​between p0 to p1 and rl. A fixed number of bands, such as 1, 2, 4, or 8, can be used, and the number of bands can be signaled in the picture header. For example, for an 8-bit image, the value range of each pixel is 0 to 255, which can be divided into multiple bands. For example, the range can be divided into two bands, the first band ranges from 0 to 127, and the second band ranges from 128 to 255. Therefore, the entries in the CCSO filter or lookup table can depend on the band. In other words, each row for a combination of quantized deltas in the above example Table 4 can be set with different offset values ​​for different bands. In some example implementations, the CCSO lookup table includes CCSO offset values ​​for each band. In some other example implementations, such a CCSO lookup table may be divided into M sub-lookup tables (where M is the number of band parameters), each sub-lookup table including offset values ​​for a quantized delta combination for one band. During CCSO filtering processing, a sub-lookup table may be selected for a particular sample based on pixel magnitude (and therefore its band).

[0172] In some example implementations, signaling CCSO may be performed at both the frame level and the block level. At the frame level, for example, the following example syntax elements may be signaled in the bitstream (which will be generated by the encoder and parsed by the decoder for the bitstream): 1-bit flag indicating whether CCSO is applied in this frame; · 3-bit syntax ext_filter_support indicating that the CCSO filter shape is selected, for example from a predefined set of filter shapes or a signaled set of filter shapes. · 2-bit index indicating the selection of quantization step size • Offset values ​​in the LUT using a predetermined indexing scheme. Each offset value may have a predetermined bit depth or a signaled bit depth. As mentioned above, the number of offset values ​​of the LUT depends on the number of taps and the quantization level.

[0173] For example, at the 128x128 chroma block level, a flag may be signaled to indicate whether the CCSO filter is enabled in a particular block.

[0174] In some example implementations, CCSO filter parameters (including quantization step size, number of bands, filter shape) may be constrained to be the same (not changing) across the entire frame, so that the same CCSO filter is used across the frame. However, in some cases, there may be strong local variations within a frame, and using the same CCSO filter or using CCSO filters with the same filter parameters may not provide optimal encoding.

[0175] In additional example implementations described in further detail below, CCSO filter parameters may be allowed to vary between different filter units. A filter unit may be a filter block in a reconstructed frame or a filter region comprising a plurality of spatially adjacent filter blocks. Corresponding allowed options for CCSO filter parameter combinations may be specified. Each filter unit may select from the allowed combination options. The selected combination option may be signaled in the bitstream or derived in some other manner. Allowing such CCSO filter parameter variations within a frame may provide a filter quality gain that exceeds the overhead cost in signaling.

[0176] The following example methods may be used individually or in combination in any order. In addition, each of the method (or implementation), encoder, and decoder may be implemented by a processing circuit system (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transitory computer-readable medium.

[0177] In addition, in the example implementations below, the term "block" or "filter block" may be interpreted as a prediction block, a coding block, or a coding unit, i.e., a CU. The term "block" herein may also be used to refer to a transform block. The term "block" herein may also be used to refer to a CTU block or a CCSO SB. The term "filter block" or "filter unit" may also be used to refer to a CTU block or CCSO SB associated with a flag indicating whether CCSO filtering is applied to the block (or CCSO SB). The filtered block or filter block may be any block in the above block. In some implementations, the reconstructed filter block may be divided only partially in relation to or independently of the above block. The example implementations below may be applied to cross-component sample offset filtering across any two color components.

[0178] In some example implementations, for each filter block (or filter unit) in a frame, one option may be selected from a plurality (more than one) of available options for CCSO parameter combinations for performing CCSO filtering. Each option may correspond to a CCSO filter or multiple CCSO filters from which further selection may be made. An example value for the number of parameter combination options may be an integer greater than or equal to 2.

[0179] In some example implementations, the selection from the above options may be explicitly signaled in the bitstream. Such signaling may be at the slice level, super-block level, block level, or any other level within a frame.

[0180] CCSO filtering parameters or CCSO parameters may include but are not limited to quantization step size (and / or the number of quantization levels), the number of bands and filter shape parameters. Correspondingly, multiple combinations of the above CCSO parameters may refer to a combination of quantization step size, the number of bands, and filter shape.

[0181] In some example implementations, four CCSO parameter combination options may be provided or predetermined for selection. Specifically, the four combinations may include: Option 1: filter shape 0, quantization step index 1, 2 bands; Option 2: filter shape 1, quantization step index 2, 1 band; Option 3: filter shape 3, quantization step index 1, 4 bands; Option 4: filter shape 5, quantization step index 0, 8 bands.

[0182] The above example assumes that the number of taps in the CCSO filter is 3 (however, the principle applies to other numbers of CCSO filter taps). The filter shape (indicating the position of the taps) is represented by a predefined filter shape index. For example, there may be 6 different 3-tap shapes available indexed from 0 to 5. Similarly, there may be a set of predefined quantization step sizes available, for example, 3 different step sizes indexed from 0 to 2. Regarding the number of bands, the above example directly specifies the number of bands in each option.

[0183] In an example, the selection of which parameter combination option among these available options may be made from filtering block / region to filtering block / region for performing CCSO. A block / region may select one of four options to further determine the CCSO LUT for performing CCSO filtering.

[0184] In some example implementations, the multiple combinations of CCSO parameters available for selection by the filter block may be a subset of the combinations of available quantization step sizes, number of bands, filter shape parameters. A subset of CCSO parameter combinations may be used to refer to a set of combinations of CCSO parameters. For example, in such a subset of CCSO parameter combinations, one or more of these parameters are fixed, and combinations of the remaining parameters are provided for selection by the filter block. The multiple combinations of CCSO parameters may alternatively be referred to as a set of candidate CCSO filter parameter combinations.

[0185] For example, there may also be four options for the combination of quantization step size and number of bands. Another CCSO filter parameter, the filter shape parameter, may be fixed and may not vary in the options. For example, such four CCSO filter parameter combination options may include: Option 1: filter shape 1, quantization step index 1, 2 bands; Option 2: filter shape 1, quantization step index 2, 1 band; Option 3: filter shape 1, quantization step index 1, 4 bands; Option 4: filter shape 1, quantization step index 0, 8 bands, Here, the filter shape is always the same (eg, index 1). The filtering block / region may select one of these four options and determine the CCSO LUT under the selected option to perform CCSO filtering.

[0186] Likewise, in another example, there may be four options for the number of bands combined with the filter shape parameter. The quantization step size parameter may not vary between these combination options. For example, these four options for CCSO parameter combinations may include: Option 1: filter shape 1, quantization step index 1, 2 bands Option 2: filter shape 2, quantization step index 1, 1 band Option 3: filter shape 0, quantization step index 1, 4 bands Option 4: filter shape 3, quantization step index 1, 8 bands wherein the CCSO parameter, ie, the quantization step, does not change (eg, by index 1), and 4 different combinations of the other two CCSO parameters are available for the block to select for further determining the CCSO LUT to perform CCSO filtering.

[0187] In the example implementation above, one CCSO offset LUT may be associated with each combination of CCSO parameters, such that the selection / determination of a combination for a filtering block will determine the CCSO LUT used by that block to perform CCSO filtering. In some other implementations, more than one CCSO offset LUT may be associated with each combination of CCSO parameters. In this case, further selection may be made among the more than one CCSO offset LUTs for performing CCSO filtering, and information about such additional selections may be signaled in the bitstream.

[0188] In some example implementations, the same set of options for CCSO parameter combinations may be provided to different filter blocks in a frame. For example, any one of the four combined sets of options in the above example may be provided to each filter block in a frame. Each block may select one of the four options for performing CCSO filtering.

[0189] Alternatively, for each filter block in a frame, a subset of the available options of the CCSO parameter combination may be designated as selectable by the block, and different filter blocks may be associated with different subsets of options for the CCSO parameter combination. The selectable subsets of the CCSO parameter combination may or may not share the same one or more parameter values ​​(in other words, the selectable subsets for different filter blocks may or may not have overlapping combinations). For example, different filter blocks may share some of the options of the CCSO parameter combination.

[0190] In some example implementations, adjacent filter blocks may share the same options of CCSO parameter combinations or the same subset of selectable options. In other words, different regions of a picture including M spatially adjacent filter blocks (the value of M includes but is not limited to 1, 2, 3, ...) may apply different CCSO parameter combinations or select from different subsets of combination options. The subset of available options for selection may vary according to the region, but may be the same within a region.

[0191] Therefore, in the example implementation above, the available CCSO parameter combinations or options include different option subsets. A filter block or a filter region including multiple filter blocks can be associated with an option subset. The option subset can be indexed. Therefore, the selection of a subset by a filter block or a filter region can be identified by its index. The CCSO parameter combination within the combination subset selected by the filter block can also be indexed within the combination subset. Such a two-level indexing scheme can help reduce signaling overhead in some cases.

[0192] In some example implementations, the value of each CCSO parameter for each option of the CCSO parameter combination may be signaled in HLS (higher layer syntax). HLS includes but is not limited to sequence parameter sets, frame level parameter sets, slice headers, tile headers, etc.

[0193] In some example implementations, for each combination of CCSO parameters, in addition to signaling the value of each CCSO parameter of the combination, an offset lookup table associated with the combination may also be signaled in the bitstream.

[0194] Fig.21A flow chart 2100 is shown. The logic flow 2100 starts at S2101. At S2110, a frame is reconstructed from a video bitstream to generate reconstructed samples of at least a first color component and a second color component. In S2120, for a filter unit at a level lower than the level of the reconstructed frame, a combination of cross-component sample offset (CCSO) filter parameters is selected from multiple candidate combinations of CCSO filter parameters for the filter unit in the reconstructed frame. At S2130, a CCSO filter is selected based on the selected combination of CCSO filter parameters. In S2140, the CCSO filter is applied to the reconstructed samples of the first color component in the filter unit to generate an offset to be applied to the reconstructed samples of the second color component in the filter unit. The logic flow 2100 ends at S2199.

[0195] The above operations can be combined or arranged in any number or order as needed. Two or more steps and / or operations in the steps and / or operations can be performed in parallel. The embodiments and implementations in the present disclosure can be used alone or in combination in any order. In addition, each of the method (or embodiment), encoder and decoder can be implemented by a processing circuit system (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transient computer-readable medium. The embodiments in the present disclosure can be applied to luminance blocks or chrominance blocks. The term "block" can be interpreted as a prediction block, a coding block or a coding unit, i.e., a CU. The term "block" here can also be used to refer to a transform block. In the following, when it comes to block size, it can refer to block width or block height, or the maximum value of width and height, or the minimum value of width and height, or area size (width*height), or the aspect ratio of the block (width:height, or height:width).

[0196] The above techniques may be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. Fig. 22 A computer system (2200) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0197] Computer software may be encoded using any suitable machine code or computer language, which may be subjected to mechanisms such as assembly, compilation, and linking to create code comprising instructions that may be executed directly by one or more computer central processing units (CPU), graphics processing units (GPU), etc., or through interpretation, microcode execution, etc.

[0198] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, etc.

[0199] Fig. 22 The components for the computer system (2200) shown in the example are exemplary in nature and are not intended to suggest any limitation on the scope of use or functionality of computer software implementing embodiments of the present disclosure. Nor should the configuration of components be interpreted as having any dependency or requirement related to any one or combination of components shown in the exemplary embodiment of the computer system (2200).

[0200] The computer system (2200) may include certain human-machine interface input devices. The input human-machine interface devices may include one or more of the following (only one of each is depicted): keyboard (2201), mouse (2202), touch pad (2203), touch screen (2210), data gloves (not shown), joystick (2205), microphone (2206), scanner (2207), camera (2208).

[0201] The computer system (2200) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate one or more senses of a human user through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include: tactile output devices (e.g., tactile feedback through a touch screen (2210), a data glove (not shown), or a joystick (2205), but there may also be tactile feedback devices that are not used as input devices); audio output devices (e.g., speakers (2209), headphones (not depicted)); visual output devices (e.g., screens (2210), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch screen input capabilities, each with or without tactile feedback capabilities - some of which may be capable of outputting two-dimensional visual outputs or more than three-dimensional outputs through means such as stereoscopic image output; virtual reality glasses (not depicted); holographic displays and cigarette cans (not depicted)); and printers (not depicted).

[0202] The computer system (2200) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (2220) with CD / DVD etc. media (2221), thumb drives (2222), removable hard drives or solid-state drives (2223), traditional magnetic media such as tapes and floppy disks (not depicted), dedicated ROM / ASIC / PLD based devices such as security dongles (not depicted), etc.

[0203] Those skilled in the art should also understand that the term "computer-readable media" used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transitory signals.

[0204] The computer system (2200) may also include an interface (2254) to one or more communication networks (2255). The network may be, for example, wireless, wired, optical. The network may also be a local area, wide area, metropolitan area network, vehicle-mounted and industrial network, real-time, delay-tolerant, etc. Examples of networks include: local area networks such as Ethernet; wireless LAN; cellular networks including GSM, 3G, 4G, 5G, LTE, etc.; TV wired or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV; vehicle-mounted networks and industrial networks including CAN bus, etc.

[0205] The above-mentioned human-machine interface device, human-accessible storage device, and network interface may be attached to the core ( 2240 ) of the computer system ( 2200 ).

[0206] The core (2240) may include one or more central processing units (CPUs) (2241), graphics processing units (GPUs) (2242), dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) (2243), hardware accelerators for certain tasks (2244), graphics adapters (2250), etc. These devices, along with read-only memory (ROM) (2245), random access memory (2246), internal mass storage devices such as internal non-user accessible hard drives, SSDs, etc. (2247) may be connected via a system bus (2248). In some computer systems, the system bus (2248) may be accessed in the form of one or more physical plugs to enable expansion by additional CPUs, GPUs, etc. Peripheral devices may be attached to the core's system bus (2248) directly or via a peripheral bus (2249). In an example, a screen (2210) may be connected to a graphics adapter (2250). The architecture of the peripheral bus includes PCI, USB, etc.

[0207] The computer readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or the medium and computer code may be those of a type well known and available to those skilled in the art of computer software.

[0208] Although the present disclosure has described several exemplary embodiments, there are changes, permutations, and various alternative equivalents that fall within the scope of the present disclosure. Therefore, it should be understood that those skilled in the art will be able to conceive of many systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within its spirit and scope.

Claims

1. A method for in-loop filtering a video bitstream, comprising: reconstructing a frame from the video bitstream to generate reconstructed samples of at least a first color component and a second color component; For a filtering unit at a level lower than the level of the reconstructed frame, selecting a combination of cross-component sample offset (CCSO) filtering parameters from a plurality of candidate combinations of CCSO filtering parameters for use with the filtering unit in the reconstructed frame; determining a CCSO filter according to a combination of selected CCSO filter parameters; as well as The CCSO filter is applied to reconstructed samples of the first color component in the filtering unit to generate an offset to be applied to reconstructed samples of the second color component in the filtering unit.

2. The method according to claim 1, wherein: Each candidate combination of the multiple candidate combinations of CCSO filtering parameters includes: filter shape parameters, which indicate the number and tap locations of the CCSO filter taps; a quantization step size used to discretize the cross-component taps relative to sample increments; and The number of bands indicates the number of sample amplitude bands within the allowed sample value range.

3. The method according to claim 2, wherein: The multiple candidate combinations of CCSO filter parameters include four candidate combinations.

4. The method according to claim 3, wherein: The four candidate combinations differ in both the filter shape parameters and the number of bands.

5. The method according to claim 3, wherein: The four candidate combinations share the same filter shape parameters.

6. The method according to claim 3, wherein: The four candidate combinations share the same quantization step size.

7. The method according to any one of claims 1 to 6, wherein: The combination of CCSO filter parameters is selected among a plurality of candidate combinations of CCSO filter parameters according to syntax elements explicitly signaled in the video bitstream.

8. The method according to any one of claims 1 to 6, wherein: The CCSO filter includes a CCSO lookup table for determining a cross-component sample offset for the filtering unit.

9. The method according to claim 8, wherein: Each candidate combination of the plurality of candidate combinations of CCSO filter parameters is associated with at least one CCSO lookup table.

10. The method according to claim 9, wherein: A CCSO lookup table for each candidate combination of a plurality of candidate combinations of CCSO filter parameters is signaled in the video bitstream.

11. The method according to any one of claims 1 to 6, wherein: The plurality of candidate combinations of CCSO filter parameters are the same in all filter units in the reconstructed frame.

12. The method according to any one of claims 1 to 6, wherein: The plurality of candidate combinations of CCSO filter parameters are different between at least two different filtering units in the reconstructed frame.

13. The method according to claim 12, wherein: The plurality of candidate combinations of CCSO filter parameters are associated with a combination set identifier.

14. The method according to claim 13, wherein: A combination set identifier for the filtering unit to select the combination of CCSO filter parameters is signaled in the video bitstream.

15. The method according to claim 13, wherein: The filtering unit includes a filtering block or a filtering area, and the filtering area includes a plurality of spatially adjacent filtering blocks.

16. The method according to any one of claims 1 to 6, wherein: The values ​​of the CCSO filter parameters for the multiple candidate combinations of the CCSO filter parameters are signaled in a sequence, frame, slice or tile header in a high-level syntax.

17. An electronic device comprising a memory and a processor, wherein the memory is used to store instructions, and the processor is used to execute the instructions to perform the following operations: reconstructing a frame from a video bitstream to generate reconstructed samples of at least a first color component and a second color component; For a filtering unit at a level lower than the level of the reconstructed frame, selecting a combination of cross-component sample offset (CCSO) filtering parameters from a plurality of candidate combinations of CCSO filtering parameters for use with the filtering unit in the reconstructed frame; Determining a CCSO filter according to the selected combination of CCSO filter parameters; and The CCSO filter is applied to the reconstructed samples of the first color component in the filtering unit to generate an offset to be applied to the reconstructed samples of the second color component in the filtering unit.

18. The electronic device according to claim 17, wherein: Each candidate combination of the multiple candidate combinations of CCSO filtering parameters includes: filter shape parameters, which indicate the number and tap locations of the CCSO filter taps; a quantization step size used to discretize the cross-component taps relative to sample increments; and The number of bands indicates the number of sampling amplitude bands within the allowed sample value range.

19. The electronic device according to claim 17, wherein: The combination of CCSO filter parameters is selected among a plurality of candidate combinations of CCSO filter parameters according to syntax elements explicitly signaled in the video bitstream.

20. The electronic device according to claim 17, wherein: The CCSO filter includes a CCSO lookup table for determining a cross-component sample offset for the filtering unit.

Images