METHOD AND APPARATUS FOR VIDEO CODING - Patent application

Abstract

To provide a method and an apparatus for video decoding.SOLUTION: A processing circuit of an apparatus decodes coded information on a reconstructed sample of a current component in a current picture from a coded video bitstream. The coded information indicates a sample offset filter to be applied to the reconstructed sample. The filter shape of the sample offset filter is one of a plurality of filter shapes. Each of the plurality of filter shapes includes a first reconstructed sample of a first component in the current picture. A filtered sample value of the reconstructed sample is determined on the basis of the first reconstructed sample in the filter shape. The sample offset filter is an in-loop filter whose output value is applied to the reconstructed sample as an offset to remove coding artifacts while preserving details of the current component in the current picture.SELECTED DRAWING: Figure 1A

Description

[Technical Field]

[0001] Incorporation by Reference This application claims the benefit of priority to U.S. Provisional Application No. 63 / 162,956, entitled "LOW-COMPLEXITY FILTER DESIGN FOR SAMPLE OFFSET," filed on March 18, 2021, which claims the benefit of priority to U.S. Provisional Application No. 17 / 495,318, entitled "Method and apparatus for video coding," filed on October 6, 2021. The entire disclosures of the prior applications are incorporated herein by reference in their entireties.

[0002] This disclosure generally describes embodiments related to video coding. [Background technology]

[0003] The discussion of the background art provided herein is intended to generally present the context for the present disclosure. The inventors' work is not admitted expressly or implicitly as prior art to the present disclosure, to the extent that that work is described in this background art section, along with aspects of the description that may not otherwise be admitted as prior art at the time of filing.

[0004] Video coding and decoding can be performed using inter-picture prediction with motion compensation. Uncompressed digital video may include a series of pictures, each with spatial dimensions of, for example, 1920 x 1080 luminance samples and associated chrominance samples. The series of pictures may have a fixed or variable picture rate (also informally known as a frame rate), for example, 60 pictures per second or 60 Hz. Uncompressed video has specific bitrate requirements. For example, 1080p60 4:2:0 video (1920 x 1080 luminance sample resolution at a 60 Hz frame rate) with 8 bits per sample requires a bandwidth approaching 1.5 Gbit / s. One hour of such video requires more than 600 GBytes of storage space.

[0005] One goal of video coding and decoding is to reduce redundancy in an input video signal through compression. Compression can help reduce the aforementioned bandwidth and / or storage space requirements, sometimes by more than two orders of magnitude. Both lossless and lossy compression, as well as combinations thereof, can be used. Lossless compression refers to techniques that allow an exact copy of the original signal to be reconstructed from a compressed version of the original signal. When lossy compression is used, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signal is small enough to make the reconstructed signal useful for its intended use. For video, lossy compression is widely adopted. The amount of acceptable distortion depends on the application; for example, users of certain consumer streaming applications may tolerate higher distortion than users of television distribution applications. The achievable compression ratio can reflect the fact that the higher the tolerable / acceptable distortion, the higher the compression ratio that can be obtained.

[0006] Video encoders and decoders can utilize techniques from several broad categories, including, for example, motion compensation, transform, quantization, and entropy coding.

[0007] Video codec technology can include a technique known as intra-coding. In intra-coding, sample values ​​are represented without reference to samples or other data from previously reconstructed reference pictures. In some video codecs, pictures are spatially subdivided into blocks of samples. If all blocks of samples are coded in intra mode, the picture may be an intra-picture. Intra-pictures and their derivatives, such as independent decoder refresh pictures, can be used to reset the decoder state and therefore can be used as the first picture in a coded video bitstream and video session or as a still image. Samples of intra-blocks can be transformed, and the transform coefficients can be quantized before entropy coding. Intra-prediction can be a technique that minimizes sample values ​​in the pre-transform domain. In some cases, the smaller the DC value and AC coefficients after the transform, the fewer bits required for a given quantization step size to represent the block after entropy coding.

[0008] For example, conventional intra-coding, such as that known from MPEG-2 generation coding techniques, does not use intra-prediction. However, some newer video compression techniques include techniques that attempt to predict from surrounding sample data and / or metadata obtained during the encoding and / or decoding of spatially adjacent and preceding data blocks in decoding order. Such techniques are hereinafter referred to as "intra-prediction" techniques. Note that in at least some cases, intra-prediction uses reference data only from the current picture being reconstructed, and not from reference pictures.

[0009] Intra-prediction can take many different forms. If two or more of such techniques can be used in a given video coding technique, the technique used can be coded as an intra-prediction mode. In certain cases, a mode can have sub-modes and / or parameters, which can be coded separately or included in the mode's codeword. The codeword used for a given mode, sub-mode, and / or parameter combination can affect coding efficiency via intra-prediction, and therefore can also affect the entropy coding technique used to convert the codeword into a bitstream.

[0010] Certain modes of intra prediction were introduced in H.264, improved in H.265, and further refined in newer coding techniques such as Joint Search Model (JEM), Versatile Video Coding (VVC), and Benchmark Set (BMS). Predictor blocks can be formed using neighboring sample values ​​belonging to already available samples. The sample values ​​of the neighboring samples are copied into the predictor block according to their direction. The reference to the direction used can be coded in the bitstream or can itself be predicted.

[0011] Referring to FIG. 1A, shown at the bottom right is a subset of nine known predictor directions from the 33 possible predictor directions in H.265 (corresponding to 33 of the 35 intra modes). The point where the arrows converge (101) represents the sample being predicted. The arrows represent the direction in which the sample is predicted. For example, arrow (102) indicates that sample (101) is predicted from one or more samples to the upper right, at an angle of 45 degrees from horizontal. Similarly, arrow (103) indicates that sample (101) is predicted from one or more samples to the lower left of sample (101), at an angle of 22.5 degrees from horizontal.

[0012] 1A, a square block (104) of 4x4 samples (indicated by a bold dashed line) is shown in the upper left. The square block (104) contains 16 samples, each labeled with "S," its Y-dimensional position (e.g., row number), and its X-dimensional position (e.g., column number). For example, sample S21 is the second sample (from the top) in the Y dimension and the first sample (from the left) in the X dimension. Similarly, sample S44 is the fourth sample in the block (104) in both the Y and X dimensions. Because the block size is 4x4 samples, S44 is located in the lower right. Reference samples, which follow a similar numbering scheme, are also shown. The reference samples are labeled R, their Y-position (e.g., row number) and X-position (column number) relative to the block (104). In both H.264 and H.265, predicted samples are adjacent to the block being reconstructed, and therefore negative values ​​need not be used.

[0013] Intra-picture prediction can work by copying reference sample values ​​from neighboring samples as assigned by the signaled prediction direction. For example, assume that the coded video bitstream includes signaling for this block indicating a prediction direction consistent with the arrow (102), i.e., the sample is predicted from one or more prediction samples to the upper right, at a 45-degree angle from horizontal. In this case, samples S41, S32, S23, and S14 are predicted from the same reference sample R05. Then, sample S44 is predicted from reference sample R08.

[0014] In certain cases, particularly when the orientation is not evenly divisible by 45 degrees, multiple reference sample values ​​may be combined, for example by interpolation, to calculate the reference sample.

[0015] The number of possible directions has increased as video coding technology has evolved. In H.264 (2003), nine different directions could be represented. This increased to 33 in H.265 (2013), and JEM / VVC / BMS, as of the time of disclosure, can support up to 65 directions. Experiments have been conducted to identify the most likely directions, and specific techniques of entropy coding can be used to accept certain penalties for less likely directions, so that these more likely directions can be represented with fewer bits. Furthermore, the direction itself may be predictable from neighboring directions used in neighboring, already decoded blocks.

[0016] FIG. 1B shows a schematic diagram (180) showing 65 intra-prediction directions with JEM to illustrate the increasing number of prediction directions over time.

[0017] The mapping of intra-prediction direction bits in a coded video bitstream representing direction can vary between video coding techniques, ranging from simple direct mapping of prediction direction to intra-prediction modes, for example, to complex adaptation schemes involving codewords, most likely modes, and similar techniques. However, in all cases, there may be certain directions that are statistically less likely to occur in the video content than certain other directions. Because the goal of video compression is to reduce redundancy, in well-performing video coding techniques, these less likely directions are represented with more bits than the more likely directions.

[0018] Motion compensation may be a lossy compression technique and may refer to a technique used to predict a newly reconstructed picture or picture portion after blocks of sample data from a previously reconstructed picture or portion thereof (reference picture) have been spatially shifted in a direction indicated by a motion vector (hereafter MV). In some cases, the reference picture may be the same as the picture currently being reconstructed. The MV may have two dimensions, X and Y, or three dimensions, with the third dimension being an indication of the reference picture used (the third dimension may indirectly be a temporal dimension).

[0019] In some video compression techniques, the MV applicable to a particular area of ​​sample data can be predicted from other MVs, for example, from an MV associated with another area of ​​sample data that is spatially adjacent to the area being reconstructed and precedes that MV in decoding order. Doing so can substantially reduce the amount of data required to code the MV, thereby eliminating redundancy and increasing compression. MV prediction can work effectively because, for example, when coding an input video signal derived from a camera (known as natural video), areas larger than the area to which a single MV is applicable have a statistical likelihood of moving in a similar direction and therefore, in some cases, can be predicted using similar motion vectors derived from MVs in neighboring areas. This ensures that the MV found for a given area is similar or identical to the MV predicted from surrounding MVs, which, after entropy coding, can be represented with fewer bits than would be used to code the MV directly. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., an MV) derived from an original signal (i.e., a sample stream). In some cases, MV prediction itself can be lossy, for example, due to rounding errors when calculating a predictor from several surrounding MVs.

[0020] H.265 / HEVC (ITU-T Rec. H.265, “High Efficiency Video Coding”, December 2016) describes various MV prediction mechanisms. Among the many MV prediction mechanisms provided by H.265, the one described in this specification is a technique hereinafter referred to as “spatial merging.”

[0021] Referring to Figure 2, a current block (201) contains samples that the encoder found during the motion search process to be predictable from a previous block of the same size but spatially shifted. Instead of coding its MV directly, the MV can be derived from metadata associated with one or more reference pictures, e.g., the last reference picture (in decoding order), using the MV associated with any one of five surrounding samples represented by A0, A1, and B0, B1, B2 (202 to 206, respectively). In H.265, MV prediction can use predictors from the same reference picture as neighboring blocks. Summary of the Invention [Means for solving the problem]

[0022] Aspects of the present disclosure provide methods and apparatuses for video encoding and decoding. In some examples, the apparatus for video decoding includes a processing circuit. The processing circuit can decode coded information about a reconstructed sample of a current component in a current picture from a coded video bitstream. The coded information indicates a sample offset filter to be applied to the reconstructed sample. A filter shape of the sample offset filter can be one of a plurality of filter shapes. Each of the plurality of filter shapes can include a first reconstructed sample of a first component in the current picture. The processing circuit can determine an output value of the sample offset filter based on the first reconstructed sample in the filter shape. Further, the processing circuit can determine a filtered sample value of the reconstructed sample of the current component based on the reconstructed sample and the output value of the sample offset filter. The sample offset filter is an in-loop filter whose output value is applied to the reconstructed sample as an offset to remove coding artifacts while preserving details of the current component in the current picture.

[0023] For each of the plurality of filter shapes, the top sample of the first reconstructed samples is located within N rows of a central reconstructed sample in the first component that is co-located with the reconstructed sample of the current component, where N may be 3 or less.

[0024] In one embodiment, the number of first reconstruction samples in each of the plurality of filter shapes is less than five.

[0025] In one embodiment, each of the plurality of filter shapes includes a central reconstructed sample co-located with the reconstructed sample of the current component, and the number of first reconstructed samples in each of the plurality of filter shapes is 3. In one example, the first reconstructed sample in each of the plurality of filter shapes includes a central reconstructed sample and two samples symmetrical about the central reconstructed sample.

[0026] In one example, the sample offset filter comprises one of a local sample offset filter (LSO), a cross-component sample offset filter (CCSO), and an adaptive sample offset filter (ASO).

[0027] In one embodiment, N is 1. In one example, each of the plurality of filter shapes includes a central reconstructed sample co-located with the reconstructed sample of the current component, and the number of first reconstructed samples in each of the plurality of filter shapes is 3.

[0028] In one example, N is 1, and the position of the central reconstructed sample in the first component of the current picture is (i, j), where i and j may be integers. The plurality of filter shapes includes six filter shapes. The locations of the two samples in a first of the six filter shapes are (i-1, j) and (i+1, j), the locations of the two samples in a second of the six filter shapes are (i-1, j-1) and (i+1, j+1), the locations of the two samples in a third of the six filter shapes are (i, j-1) and (i, j+1), the locations of the two samples in a fourth of the six filter shapes are (i+1, j-1) and (i-1, j+1), the locations of the two samples in a fifth of the six filter shapes are (i, j-4) and (i, j+4), and the locations of the two samples in a sixth of the six filter shapes are (i, j-7) and (i, j+7).

[0029] In one example, N is 1, and the position of the central reconstructed sample in the first component of the current picture is (i, j), where i and j can be integers. The plurality of filter shapes includes six different filter shapes. The positions of two samples in the first filter shape are (i-1, j) and (i+1, j), the positions of two samples in the second filter shape are (i-1, j-1) and (i+1, j+1), the positions of two samples in the third filter shape are (i, j-1) and (i, j+1), the positions of two samples in the fourth filter shape are (i+1, j-1) and (i-1, j+1), the positions of two samples in the fifth filter shape are (i, j-3) and (i, j+3), and the positions of two samples in the sixth filter shape are (i, j-5) and (i, j+5).

[0030] In one example, the sample offset filter is CCSO, e.g., the first component is the luma component and the current component is the chroma component.

[0031] In one example, the sample offset filter is an LSO, the first component is the current component, and the reconstructed sample is a central reconstructed sample.

[0032] In some examples, the current component is within a filtering unit within the current picture. The processing circuitry can determine the filter shape of the sample offset filter based on the position of the reconstructed sample within the filtering unit.

[0033] In one example, the sample offset filter is a CCSO filter and the filtering unit is a chroma block having 128x128 chroma samples.

[0034] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by a computer, cause the computer to perform a method for video decoding and / or video encoding.

[0035] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings. [Brief explanation of the drawings]

[0036] [Figure 1A] FIG. 2 is a schematic diagram of an example subset of intra-prediction modes. [Figure 1B] FIG. 1 is a diagram of an exemplary intra-prediction direction. [Figure 2] FIG. 1 is a schematic diagram of a current block and its surrounding spatial merge candidates in one example. [Figure 3] FIG. 3 is a simplified block diagram schematic of a communication system (300) according to one embodiment. [Figure 4] FIG. 4 is a simplified block diagram schematic of a communication system (400) according to one embodiment. [Figure 5] FIG. 2 is a schematic diagram of a simplified block diagram of a decoder according to one embodiment. [Figure 6] FIG. 2 is a schematic diagram of a simplified block diagram of an encoder according to one embodiment. [Figure 7] FIG. 2 is a block diagram illustrating an encoder according to another embodiment. [Figure 8] FIG. 4 is a block diagram illustrating a decoder according to another embodiment. [Figure 9] FIG. 2 illustrates an exemplary adaptive loop filter according to an embodiment of the present disclosure. [Figure 10A] FIG. 10 illustrates an example of sub-sampled positions used to calculate vertical gradients, according to an embodiment of the present disclosure. [Figure 10B] FIG. 10 illustrates an example of sub-sampled positions used to calculate horizontal gradients, according to an embodiment of the present disclosure. [Figure 10C] FIG. 10 illustrates an example of subsampled positions used to calculate diagonal gradients, according to an embodiment of the present disclosure. [Figure 10D]FIG. 10 illustrates an example of subsampled positions used to calculate diagonal gradients, according to an embodiment of the present disclosure. [Figure 11A] FIG. 10 illustrates modified block classification at a virtual boundary according to an embodiment of the present disclosure. [Figure 11B] FIG. 10 illustrates modified block classification at a virtual boundary according to an embodiment of the present disclosure. [Figure 12A] FIG. 1 illustrates an example adaptive loop filter with padding operations at virtual boundaries according to an embodiment of the present disclosure. [Figure 12B] FIG. 1 illustrates an example adaptive loop filter with padding operations at virtual boundaries according to an embodiment of the present disclosure. [Figure 12C] FIG. 1 illustrates an example adaptive loop filter with padding operations at virtual boundaries according to an embodiment of the present disclosure. [Figure 12D] FIG. 1 illustrates an example adaptive loop filter with padding operations at virtual boundaries according to an embodiment of the present disclosure. [Figure 12E] FIG. 1 illustrates an example adaptive loop filter with padding operations at virtual boundaries according to an embodiment of the present disclosure. [Figure 12F] FIG. 1 illustrates an example adaptive loop filter with padding operations at virtual boundaries according to an embodiment of the present disclosure. [Figure 13] FIG. 10 illustrates an example of maximum coding unit-aligned picture quadtree partitioning, according to one embodiment of this disclosure. [Figure 14] FIG. 14 illustrates a quadtree division pattern corresponding to FIG. 13 according to one embodiment of the present disclosure. [Figure 15] FIG. 2 illustrates a cross-component filter used to generate chroma components according to one embodiment of the present disclosure. [Figure 16] FIG. 1 illustrates an example of a filter according to an embodiment of the present disclosure. [Figure 17A] FIG. 2 illustrates an exemplary location of chroma samples relative to luma samples according to an embodiment of the present disclosure. [Figure 17B]FIG. 2 illustrates an exemplary location of chroma samples relative to luma samples according to an embodiment of the present disclosure. [Figure 18] FIG. 10 is a diagram illustrating an example of a direction search for a block according to an embodiment of the present disclosure. [Figure 19] FIG. 10 illustrates an example of subspace projection according to an embodiment of the present disclosure. [Figure 20] FIG. 1 illustrates an example of a filter support area for a cross-component sample offset (CCSO) filter according to one embodiment of the present disclosure. [Figure 21A] FIG. 10 illustrates an exemplary mapping used in a CCSO filter according to an embodiment of the present disclosure. [Figure 21B] FIG. 10 illustrates an exemplary mapping used in a CCSO filter according to an embodiment of the present disclosure. [Figure 21C] FIG. 10 illustrates an exemplary mapping used in a CCSO filter according to an embodiment of the present disclosure. [Figure 22] FIG. 1 illustrates an exemplary implementation of a CCSO filter according to an embodiment of the present disclosure. [Figure 23] FIG. 10 illustrates four patterns for pixel classification in edge offset according to one embodiment of the present disclosure. [Figure 24] FIG. 10 illustrates an exemplary filter shape used in a sample offset filter according to an embodiment of the present disclosure. [Figure 25] FIG. 10 illustrates an exemplary filter shape used in a sample offset filter according to an embodiment of the present disclosure. [Figure 26] FIG. 10 illustrates an exemplary filter shape used in a sample offset filter according to an embodiment of the present disclosure. [Figure 27] FIG. 10 illustrates an exemplary filter shape used in a sample offset filter according to an embodiment of the present disclosure. [Figure 28] FIG. 10 illustrates an exemplary filter shape used in a sample offset filter according to an embodiment of the present disclosure. [Figure 29]FIG. 10 illustrates an exemplary filter shape used in a sample offset filter according to an embodiment of the present disclosure. [Figure 30] FIG. 10 illustrates an example in which one or more filter taps are moved, according to one embodiment of the present disclosure. [Figure 31] 3 is a flowchart outlining a process (3100) according to one embodiment of the present disclosure. [Figure 32] 32 is a flowchart outlining a process (3200) according to one embodiment of the present disclosure. [Figure 33] FIG. 1 is a schematic diagram of a computer system according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0037] Figure 3 shows a simplified block diagram of a communication system (300) according to one embodiment of the present disclosure. The communication system (300) includes multiple terminal devices that can communicate with each other, for example, via a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) interconnected via the network (350). In the example of Figure 3, the first pair of terminal devices (310) and (320) perform unidirectional transmission of data. For example, the terminal device (310) may code video data (e.g., a stream of video pictures captured by the terminal device (310)) for transmission to the other terminal device (320) via the network (350). The encoded video data may be transmitted in the form of one or more coded video bitstreams. The terminal device (320) may receive the coded video data from the network (350), decode the coded video data to reconstruct the video pictures, and display the video pictures according to the reconstructed video data. One-way data transmission may be common, such as in media serving applications.

[0038] In another example, the communication system (300) includes a second pair of terminal devices (330) and (340) that perform bidirectional transmission of coded video data, such as may occur during a video conference. For the bidirectional transmission of data, in one example, each of the terminal devices (330) and (340) may code video data (e.g., a stream of video pictures captured by that terminal device) for transmission to the other of the terminal devices (330) and (340) over the network (350). Each of the terminal devices (330) and (340) may also receive coded video data transmitted by the other of the terminal devices (330) and (340), decode the coded video data to recover the video pictures, and display the video pictures on an accessible display device according to the recovered video data.

[0039] In the example of FIG. 3 , the terminal devices 310, 320, 330, and 340 may be depicted as a server, a personal computer, and a smartphone, but the principles of the present disclosure are not so limited. Embodiments of the present disclosure apply to laptop computers, tablet computers, media players, and / or dedicated videoconferencing equipment. The network 350 represents any number of networks that convey coded video data between the terminal devices 310, 320, 330, and 340, including, for example, wired (wired) and / or wireless communication networks. The communication network 350 may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this discussion, the architecture and topology of the network 350 may not be important to the operation of the present disclosure, unless otherwise described herein below.

[0040] 4 illustrates the arrangement of a video encoder and a video decoder in a streaming environment as an example of an application of the disclosed subject matter. The disclosed subject matter is equally applicable to other video-enabled applications including, for example, video conferencing, digital television, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0041] The streaming system may include a video source (401) and a capture subsystem (413), which may include, for example, a digital camera, that creates a stream of uncompressed video pictures (402). In one example, the stream of video pictures (402) includes samples captured by the digital camera. The stream of video pictures (402) is shown with a bold line to emphasize its high data volume compared to the encoded video data (404) (or coded video bitstream) and can be processed by an electronic device (420) that includes a video encoder (403) coupled to the video source (401). The video encoder (403) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video data (404) (or coded video bitstream (404)), which is shown with a thin line to emphasize its low data volume compared to the stream of video pictures (402), can be stored on a streaming server (405) for future use. One or more streaming client subsystems, such as the client subsystems (406) and (408) of Figure 4, can access the streaming server (405) to obtain copies (407) and (409) of the encoded video data (404). The client subsystem (406) can include a video decoder (410), for example, within an electronic device (430). The video decoder (410) decodes the input copy (407) of the encoded video data and creates an output stream (411) of video pictures that can be rendered on a display (412) (e.g., a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data (404), (407), and (409) (e.g., a video bitstream) can be encoded according to a particular video coding / compression standard. An example of such a standard is ITU-T Recommendation H.265.In one example, a developing video coding standard is informally known as Versatile Video Coding (VVC), and the disclosed subject matter may be used in the context of VVC.

[0042] It should be noted that the electronic devices (420) and (430) may include other components (not shown). For example, the electronic device (420) may include a video decoder (not shown), and the electronic device (430) may also include a video encoder (not shown).

[0043] 5 shows a block diagram of a video decoder (510) according to one embodiment of the present disclosure. The video decoder (510) can be included in an electronic device (530). The electronic device (530) can include a receiver (531) (e.g., receiving circuitry). The video decoder (510) can be used in place of the video decoder (410) in the example of FIG. 4.

[0044] The receiver (531) can receive one or more coded video sequences to be decoded by the video decoder (510), and in the same or another embodiment, one coded video sequence is decoded at a time, with the decoding of each coded video sequence being independent of the other coded video sequences. The coded video sequences may be received from a channel (501), which may be a hardware / software link to a storage device that stores the coded video data. The receiver (531) may receive the coded video data along with other data, such as coded audio data and / or auxiliary data streams, that may be forwarded to a respective using entity (not shown). The receiver (531) may separate the coded video sequences from other data. To combat network jitter, a buffer memory (515) may be coupled between the receiver (531) and the entropy decoder / parser (520) (hereinafter "parser (520)"). In certain applications, the buffer memory (515) is part of the video decoder (510). In other cases, it may be external to the video decoder (510) (not shown). In still other cases, there may be a buffer memory (not shown) external to the video decoder (510), for example, to combat network jitter, and another buffer memory (515) internal to the video decoder (510), for example, to handle playback timing. When the receiver (531) is receiving data from a storage / forwarding device with sufficient bandwidth and controllability, or from an isosynchronous network, the buffer memory (515) may be unnecessary or may be small. For use with best-effort packet networks such as the Internet, the buffer memory (515) may be required and may be relatively large, advantageously adaptively sized, and may be implemented at least in part in an operating system or similar element (not shown) external to the video decoder (510).

[0045] The video decoder (510) may include a parser (520) for reconstructing symbols (521) from the coded video sequence. These symbol categories include information used to manage the operation of the video decoder (510) and, potentially, information for controlling a rendering device, such as a rendering device (512) (e.g., a display screen) that is not an integral part of the electronic device (530) but may be coupled to the electronic device (530), as shown in FIG. 5. The control information for the rendering device(s) may be in the form of a supplemental enhancement information (SEI) message or a video usability information (VUI) parameter set fragment (not shown). The parser (520) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may be in accordance with a video coding technique or standard and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (520) may extract from the coded video sequence a set of subgroup parameters for at least one of a subgroup of pixels in the video decoder based on at least one parameter corresponding to the group. The subgroups may include groups of pictures (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The parser (520) may also extract information from the coded video sequence, such as transform coefficients, quantization parameter values, motion vectors, etc.

[0046] The parser (520) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (515) to create symbols (521).

[0047] The reconstruction of the symbols (521) can include several different units, depending on the type of coded video picture or portion thereof (e.g., inter-picture and intra-picture, inter-block and intra-block, etc.), as well as other factors. Which units are included and how can be controlled by subgroup control information parsed from the coded video sequence by the parser (520). The flow of such subgroup control information between the parser (520) and the following units is not shown for clarity.

[0048] Beyond the functional blocks already described, the video decoder (510) can be conceptually subdivided into several functional units, as described below. In actual implementations operating under commercial constraints, many of these units will interact closely with each other and may be, at least partially, integrated with each other. However, for purposes of describing the disclosed subject matter, the following conceptual subdivision into functional units is appropriate.

[0049] The first unit is a scalar / inverse transform unit (551), which receives quantized transform coefficients as well as control information from the parser (520) including which transform to use, block size, quantization coefficients, quantization scaling matrix, etc. as symbol(s) (521). The scalar / inverse transform unit (551) can output blocks comprising sample values ​​that can be input to an aggregator (555).

[0050] In some cases, the output samples of the scaler / inverse transform (551) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by an intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) generates blocks of the same size and shape as the block being reconstructed using surrounding, already reconstructed information fetched from a current picture buffer (558). The current picture buffer (558), for example, buffers partially reconstructed and / or fully reconstructed current pictures. The aggregator (555) may add, on a sample-by-sample basis, the prediction information generated by the intra-prediction unit (552) to the output sample information provided by the scaler / inverse transform unit (551).

[0051] In other cases, the output samples of the scalar / inverse transform unit (551) may relate to an inter-coded, potentially motion-compensated, block. In such cases, the motion-compensated prediction unit (553) may access a reference picture memory (557) to fetch samples used for prediction. After motion-compensating the fetched samples according to the symbols (521) associated with the block, these samples may be added by an aggregator (555) to the output of the scalar / inverse transform unit (551) to generate output sample information (in this case, referred to as residual samples or residual signals). The addresses in the reference picture memory (557) from which the motion-compensated prediction unit (553) fetches prediction samples may be controlled by motion vectors available to the motion-compensated prediction unit (553), for example, in the form of symbols (521) that may have X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​fetched from the reference picture memory (557) when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, and the like.

[0052] The output samples of the aggregator (555) can be subjected to various loop filtering techniques in the loop filter unit (556). The video compression techniques are controlled by parameters contained in the coded video sequence (also referred to as the coded video bitstream) and can include in-loop filter techniques available to the loop filter unit (556) as symbols (521) from the parser (520), but can also be responsive to meta-information obtained during decoding of a previous portion (in decoding order) of the coded picture or coded video sequence, or to previously reconstructed, loop-filtered sample values.

[0053] The output of the loop filter unit (556) may be a sample stream that can be output to a rendering device (512) as well as stored in a reference picture memory (557) for use in future inter-picture prediction.

[0054] Once a particular coded picture is fully reconstructed, it can be used as a reference picture for future prediction. For example, once the coded picture corresponding to the current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (520)), the current picture buffer (558) can become part of the reference picture memory (557), and a new current picture buffer can be reallocated before beginning reconstruction of the next coded picture.

[0055] The video decoder (510) may perform decoding operations according to a predetermined video compression technique of a standard, such as ITU-T Rec. H.265. A coded video sequence may conform to the syntax specified by the video compression technique or standard being used, in the sense that the coded video sequence adheres to both the syntax of the video compression technique or standard and the profile documented in the video compression technique or standard. Specifically, a profile may select certain tools from among all tools available in the video compression technique or standard as tools reserved for use only under that profile. Compliance may also require that the complexity of the coded video sequence be within a range defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level may, in some cases, be further constrained by a hypothetical reference decoder (HRD) specification and metadata for HRD buffer management signaled in the coded video sequence.

[0056] In one embodiment, the receiver (531) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the coded video sequence(s). The additional data may be used by the video decoder (510) to properly decode that data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, error correction codes, etc.

[0057] 6 shows a block diagram of a video encoder (603) according to one embodiment of the present disclosure. The video encoder (603) is included in an electronic device (620). The electronic device (620) includes a transmitter (640) (e.g., a transmitting circuit). The video encoder (603) can be used in place of the video encoder (403) in the example of FIG. 4.

[0058] The video encoder (603) may receive video samples from a video source (601) (not part of the electronic device (620) in the example of FIG. 6) that may capture video image(s) to be coded by the video encoder (603). In another example, the video source (601) is part of the electronic device (620).

[0059] The video source (601) may provide a source video sequence to be coded by the video encoder (603) in the form of a digital video sample stream that can be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source (601) may be a storage device that stores previously prepared video. In a video conferencing system, the video source (601) may be a camera that captures local image information as a video sequence. The video data may be provided as multiple individual pictures that impart motion when viewed in sequence. The pictures themselves may be organized as a spatial array of pixels, each of which may contain one or more samples, depending on the sampling structure, color space, etc. used. Those skilled in the art can readily understand the relationship between pixels and samples. The following description focuses on samples.

[0060] According to one embodiment, the video encoder (603) may code and compress pictures of a source video sequence into a coded video sequence (643) in real time or under any other time constraints required by the application. Enforcing an appropriate coding rate is one function of the controller (650). In some embodiments, the controller (650) controls and is operatively coupled to other functional units as described below. For clarity, coupling is not shown. Parameters set by the controller (650) may include rate control-related parameters (e.g., picture skip, quantizer, lambda value for rate-distortion optimization techniques), picture size, group-of-picture (GOP) layout, maximum motion vector search range, etc. The controller (650) may be configured with other appropriate functions related to the video encoder (603) optimized for a particular system design.

[0061] In some embodiments, the video encoder (603) is configured to operate in a coding loop. As an overly simplified explanation, in one example, the coding loop can include a source coder (630) (e.g., responsible for creating symbols, such as a symbol stream, based on an input picture to be coded and one or more reference pictures) and a (local) decoder (633) embedded in the video encoder (603). The decoder (633) reconstructs the symbols to create sample data in a manner similar to that which a (remote) decoder would also create (since in the video compression techniques contemplated by the disclosed subject matter, any compression between the symbols and the coded video bitstream is lossless). The reconstructed sample stream (sample data) is input to a reference picture memory (634). Because decoding of the symbol stream produces bit-exact results regardless of the location (local or remote) of the decoder, the contents in the reference picture memory (634) are also bit-exact between the local and remote encoders. In other words, the prediction part of the encoder "sees" as reference picture samples exactly the same sample values ​​that the decoder will "see" when using the prediction during decoding. This basic principle of reference picture synchrony (and the resulting drift if synchrony cannot be maintained, e.g., due to channel error) is also used in several related technologies.

[0062] The operation of the "local" decoder (633) may be the same as the operation of a "remote" decoder, such as the video decoder (510), described in detail above in connection with Figure 5. Referring also briefly to Figure 5, however, because symbols are available and the encoding / decoding of symbols into a coded video sequence by the entropy coder (645) and parser (520) may be lossless, the entropy decoding portion of the video decoder (510), including the buffer memory (515) and parser (520), may not be fully implemented in the local decoder (633).

[0063] At this point, it can be said that any decoder technology, other than parsing / entropy decoding, present in the decoder must necessarily also exist in the corresponding encoder in substantially the same functional form. For this reason, the subject matter of the disclosure focuses on decoder operation. A description of the encoder technology can be omitted, as it is the reverse of the decoder technology, which has been comprehensively described. Only in certain areas is a more detailed description necessary, as will be presented below.

[0064] In operation, in some examples, the source coder (630) may perform motion-compensated predictive coding, which predictively codes an input picture with reference to one or more previously coded pictures from a video sequence designated as “reference pictures.” In this manner, the coding engine (632) codes differences between pixel blocks of the input picture and pixel blocks of the reference picture(s) that may be selected as predictive reference(s) for the input picture.

[0065] The local video decoder (633) may decode coded video data of pictures that may be designated as reference pictures based on symbols created by the source coder (630). The operation of the coding engine (632) may advantageously be a lossy process. If the coded video data can be decoded by a video decoder (not shown in FIG. 6), the reconstructed video sequence may typically be a copy of the source video sequence, with some error. The local video decoder (633) may replicate the decoding process that may be performed by the video decoder on the reference pictures and store the reconstructed reference pictures in a reference picture cache (634). In this way, the video encoder (603) may locally store copies of reconstructed reference pictures that have common content with reconstructed reference pictures obtained by the far-end video decoder (without transmission error).

[0066] The predictor (635) may perform predictive searches for the coding engine (632). That is, for a new picture to be coded, the predictor (635) may search the reference picture memory (634) for sample data (as candidate reference pixel blocks) or specific metadata, such as reference picture motion vectors, block shapes, etc., that can serve as suitable predictive references for the new picture. The predictor (635) may operate sample block by pixel block to find suitable predictive references. In some cases, as determined by search results obtained by the predictor (635), the input picture may have predictive references drawn from multiple reference pictures stored in the reference picture memory (634).

[0067] The controller (650) may manage the coding operations of the source coder (630), including, for example, setting parameters and subgroup parameters used to encode the video data.

[0068] The output of all of the aforementioned functional units may be entropy coded by an entropy coder (645), which converts the symbols produced by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0069] The transmitter (640) may buffer the coded video sequence(s) created by the entropy coder (645) in preparation for transmission over a communication channel (660), which may be a hardware / software link to a storage device that will store the encoded video data. The transmitter (640) may merge the coded video data from the video coder (603) with other data to be transmitted, such as coded audio data and / or an auxiliary data stream (source not shown).

[0070] The controller (650) may manage the operation of the video encoder (603). During coding, the controller (650) may assign a particular coded picture type to each coded picture, which may affect the coding technique that may be applied to the respective picture. For example, pictures may often be assigned as one of the following picture types:

[0071] An intra-picture (I-picture) may be a picture that can be coded and decoded without using any other picture in a sequence as a prediction source. Some video codecs allow different types of intra-pictures, including, for example, independent decoder refresh ("IDR") pictures. Those skilled in the art are aware of these variations of I-pictures and their respective uses and characteristics.

[0072] A predicted picture (P picture) may be a picture that can be coded and decoded using intra- or inter-prediction, which uses at most one motion vector and reference index to predict the sample values ​​of each block.

[0073] A bidirectionally predicted picture (B picture) may be a picture that can be coded and decoded using intra- or inter-prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, a multi-predicted picture may use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0074] A source picture is generally spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and may be coded block by block. Blocks may be predictively coded with reference to other (already coded) blocks as determined by the coding assignment applied to the block's respective picture. For example, blocks of an I-picture may be non-predictively coded or predictively coded with reference to already coded blocks of the same picture (spatial prediction or intra-prediction). Pixel blocks of a P-picture may be predictively coded by spatial prediction with reference to one previously coded reference picture or by temporal prediction. Blocks of a B-picture may be predictively coded by spatial prediction with reference to one or two previously coded reference pictures or via temporal prediction.

[0075] The video encoder (603) may perform coding operations in accordance with a predetermined video coding technique or standard, such as ITU-T Rec. H.265. In doing so, the video encoder (603) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the coded video data may conform to a syntax specified by the video coding technique or standard being used.

[0076] In one embodiment, the transmitter (640) may transmit additional data along with the encoded video. The source coder (630) may include such data as part of the coded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures or slices, SEI messages, VUI parameter set fragments, etc.

[0077] Video may be captured in time sequence as multiple source pictures (video pictures). Intra-picture prediction (often abbreviated as intra-prediction) exploits spatial correlation within a given picture, while inter-picture prediction exploits correlation (temporal or otherwise) between pictures. In one example, a particular picture being encoded / decoded, called the current picture, is divided into blocks. If a block in the current picture is similar to a reference block in a previously coded, still-buffered reference picture in the video, the block in the current picture may be coded by a vector called a motion vector. The motion vector points to a reference block within the reference picture and may have a third dimension that identifies the reference picture if multiple reference pictures are used.

[0078] In some embodiments, bi-prediction techniques can be used in inter-picture prediction. According to bi-prediction techniques, two reference pictures, such as a first reference picture and a second reference picture, are used, both of which are before the current picture in the video in decoding order (but may be past and future, respectively, in display order). A block in the current picture can be coded by a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. A block can be predicted by a combination of the first reference block and the second reference block.

[0079] Furthermore, in inter-picture prediction, merge mode techniques can be used to improve coding efficiency.

[0080] According to some embodiments of the present disclosure, predictions such as inter-picture prediction and intra-picture prediction are performed on a block-by-block basis. For example, according to the HEVC standard, pictures in a sequence of video pictures are divided into coding tree units (CTUs) for compression, and the CTUs within a picture have the same size, such as 64x64 pixels, 32x32 pixels, or 16x16 pixels. Generally, a CTU includes three coding tree blocks (CTBs), one luma CTB and two chroma CTBs. Each CTU can be recursively quadtree-decomposed into one or more coding units (CUs). For example, a 64x64 pixel CTU can be divided into one CU of 64x64 pixels, four CUs of 32x32 pixels, or 16 CUs of 16x16 pixels. In one example, each CU is analyzed to determine the CU's prediction type, such as an inter-prediction type or an intra-prediction type. A CU is divided into one or more prediction units (PUs) according to temporal predictability and / or spatial predictability. Generally, each PU includes one luma prediction block (PB) and two chroma PBs. In one embodiment, prediction operations in coding (encoding / decoding) are performed on a prediction block basis. Taking a luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values ​​(e.g., luma values), such as 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 pixels, etc.

[0081] 7 shows a diagram of a video encoder (703) according to another embodiment of this disclosure. The video encoder (703) is configured to receive a processed block (e.g., a predictive block) of sample values ​​in a current video picture in a sequence of video pictures and encode the processed block into a coded picture that is part of a coded video sequence. In one example, the video encoder (703) is used in place of the video encoder (403) of the example of FIG. 4.

[0082] In an HEVC example, the video encoder (703) receives a matrix of sample values ​​for a processing block, such as a predictive block of 8x8 samples. The video encoder (703) determines, using, for example, rate-distortion optimization, whether the processing block is best coded using intra-mode, inter-mode, or bi-predictive mode. If the processing block is coded in intra-mode, the video encoder (703) may encode the processing block into a coded picture using intra-prediction techniques; if the processing block is coded in inter-mode or bi-predictive mode, the video encoder (703) may encode the processing block into a coded picture using inter-prediction techniques or bi-prediction techniques, respectively. In certain video coding techniques, the merge mode may be an inter-picture prediction sub-mode in which a motion vector is derived from one or more motion vector predictors without the benefit of coded motion vector components outside the predictors. In certain other video coding techniques, there may be motion vector components applicable to the current block. In one example, the video encoder (703) includes other components, such as a mode decision module (not shown) for determining the mode of the processing block.

[0083] In the example of Figure 7, the video encoder (703) includes an inter-encoder (730), an intra-encoder (722), a residual calculator (723), a switch (726), a residual encoder (724), a general controller (721), and an entropy encoder (725), which are coupled together as shown in Figure 7.

[0084] The inter-encoder (730) is configured to receive samples of a current block (e.g., a processing block), compare the block with one or more reference blocks in a reference picture (e.g., blocks in a previous picture and a subsequent picture), generate inter-prediction information (e.g., a description of redundancy information, motion vectors, merge mode information according to the inter-coding technique), and calculate an inter-prediction result (e.g., a predicted block) based on the inter-prediction information using any suitable technique. In some examples, the reference picture is a decoded reference picture that is decoded based on the coded video information.

[0085] The intra encoder (722) is configured to receive samples of a current block (e.g., a processing block), possibly compare the block with previously coded blocks in the same picture, generate transformed quantized coefficients, and possibly also generate intra prediction information (e.g., intra prediction direction information according to one or more intra encoding techniques). In one example, the intra encoder (722) also calculates intra prediction results (e.g., predicted blocks) based on the intra prediction information and reference blocks in the same picture.

[0086] The general-purpose controller (721) is configured to determine general-purpose control data and control other components of the video encoder (703) based on the general-purpose control data. In one example, the general-purpose controller (721) determines the mode of the block and provides a control signal to the switch (726) based on the mode. For example, if the mode is intra-mode, the general-purpose controller (721) controls the switch (726) to select intra-mode results for use by the residual calculator (723) and controls the entropy encoder (725) to select intra-prediction information and include the intra-prediction information in the bitstream. If the mode is inter-mode, the general-purpose controller (721) controls the switch (726) to select inter-prediction results for use by the residual calculator (723) and controls the entropy encoder (725) to select inter-prediction information and include the inter-prediction information in the bitstream.

[0087] The residual calculator (723) is configured to calculate the difference (residual data) between the received block and a prediction result selected from the intra-encoder (722) or inter-encoder (730). The residual encoder (724) is configured to operate on the residual data and encode the residual data to generate transform coefficients. In one example, the residual encoder (724) is configured to transform the residual data from the spatial domain to the frequency domain to generate transform coefficients. The transform coefficients then undergo a quantization process to obtain quantized transform coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data can be used by the intra-encoder (722) and inter-encoder (730) as appropriate. For example, the inter-encoder (730) can generate decoded blocks based on the decoded residual data and inter-prediction information, and the intra-encoder (722) can generate decoded blocks based on the decoded residual data and intra-prediction information. The decoded blocks are processed appropriately to generate decoded pictures, which, in some examples, can be buffered in a memory circuit (not shown) and used as reference pictures.

[0088] The entropy encoder (725) is configured to format a bitstream to include the encoded blocks. The entropy encoder (725) is configured to include various information in the bitstream according to an appropriate standard, such as the HEVC standard. In one example, the entropy encoder (725) is configured to include general control data, selected prediction information (e.g., intra-prediction information or inter-prediction information), residual information, and other appropriate information in the bitstream. It should be noted that, according to the disclosed subject matter, when coding a block in a merged sub-mode of either an inter mode or a bi-prediction mode, residual information is not present.

[0089] 8 shows a diagram of a video decoder (810) according to another embodiment of the present disclosure. The video decoder (810) is configured to receive coded pictures that are part of a coded video sequence and decode the coded pictures to generate reconstructed pictures. In one example, the video decoder (810) is used in place of the video decoder (410) of the example of FIG. 4.

[0090] In the example of Figure 8, the video decoder (810) includes an entropy decoder (871), an inter-decoder (880), a residual decoder (873), a reconstruction module (874), and an intra-decoder (872), which are coupled together as shown in Figure 8.

[0091] The entropy decoder (871) can be configured to reconstruct, from a coded picture, certain symbols that represent the syntax elements that make up the coded picture. Such symbols can include, for example, prediction information (e.g., intra-prediction information or inter-prediction information) that can identify the mode in which the block is coded (e.g., intra-mode, inter-mode, bi-prediction mode, the latter two being merged or another submode), certain samples or metadata used for prediction by the intra decoder (872) or inter decoder (880), respectively, residual information in the form of quantized transform coefficients, etc. In one example, if the prediction mode is an inter-mode or bi-prediction mode, the inter-prediction information is provided to the inter decoder (880), and if the prediction type is an intra-prediction type, the intra-prediction information is provided to the intra decoder (872). The residual information can undergo inverse quantization and be provided to the residual decoder (873).

[0092] The inter decoder (880) is configured to receive the inter prediction information and generate an inter prediction result based on the inter prediction information.

[0093] The intra decoder (872) is configured to receive the intra prediction information and to generate a prediction result based on the intra prediction information.

[0094] The residual decoder (873) is configured to perform inverse quantization to extract inverse quantized transform coefficients and process the inverse quantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (873) may also require certain control information (to include quantization parameters (QPs)), which may be provided by the entropy decoder (871) (datapath not shown as this may only be a small amount of control information).

[0095] The reconstruction module (874) is configured to combine, in the spatial domain, the residual as output by the residual decoder (873) and the prediction result (possibly as output by the inter-prediction module or the intra-prediction module) to form a reconstructed block that may be part of a reconstructed picture, and the reconstructed block may be part of a reconstructed video. It should be noted that other appropriate operations, such as deblocking operations, may be performed to improve visual quality.

[0096] It should be noted that the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) may be implemented using any suitable technology. In one embodiment, the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (403), (603), and (603) and the video decoders (410), (510), and (810) may be implemented using one or more processors executing software instructions.

[0097] This disclosure includes a set of video coding techniques, such as low-complexity filters or filter shapes for sample offset. For example, low-complexity filter shapes for sample offset filters are disclosed.

[0098] The video coding format may include any suitable video coding format, for example, an open video coding format designed for video transmission over the Internet, such as the video coding format Alliance for Open Media (AOMedia) Video 1 (AV1) or a next-generation AOMedia Video format beyond AV1. The video coding standard may include the High Efficiency Video Coding (HEVC) standard, next-generation video coding beyond HEVC (e.g., Versatile Video Coding (VVC)), etc.

[0099] To reduce artifacts, the encoder / decoder can apply an adaptive loop filter (ALF) that uses block-based filter adaptation. For the luma component, for example, one of multiple filters (e.g., 25 filters) can be selected for a 4x4 luma block based on local gradient direction and activity.

[0100] The ALF may have any suitable shape and size. Referring to FIG. 9, ALF(910) through ALF(911) have diamond shapes, such as a 5x5 diamond shape for ALF(910) and a 7x7 diamond shape for ALF(911). In ALF(910), elements (920) through (932) may be used in the filtering process to form the diamond shape. Seven values ​​(e.g., C0 through C6) may be used for elements (920) through (932). In ALF(911), elements (940) through (964) may be used in the filtering process to form the diamond shape. Thirteen values ​​(e.g., C0 through C12) may be used for elements (940) through (964).

[0101] Referring to Figure 9, in some examples, two ALFs (910)-(911) with diamond filter shapes are used. A 5x5 diamond-shaped filter (910) can be applied to a chroma component (e.g., a chroma block, chroma CB), and a 7x7 diamond-shaped filter (911) can be applied to a luma component (e.g., a luma block, luma CB). Other suitable shape(s) and size(s) can be used in the ALFs. For example, a 9x9 diamond-shaped filter can be used.

[0102] The filter coefficients at the locations indicated by the values ​​(e.g., C0-C6 in (910) or C0-C12 in (920)) may be non-zero. Furthermore, if the ALF includes a clipping function, the clip values ​​at those locations may be non-zero.

[0103] For block classification of the luma component, a 4x4 block (or luma block, luma CB) can be classified or categorized as one of multiple (e.g., 25) classes. Using equation (1), the quantized values ​​of the directionality parameter D and activity value A are

number

number

number

number

number

number

number

[0104] To reduce the complexity of the block classification described above, a subsampled one-dimensional Laplacian calculation can be applied. Figures 10A-10D show the gradient g in the vertical direction (Figure 10A), horizontal direction (Figure 10B), and two diagonal directions d1 (Figure 10C) and d2 (Figure 10D). v , g h , g d1 , and g d2 10A shows an example of the subsampled positions used to compute the vertical gradient g. The same subsampled positions can be used for gradient computations in different directions. In FIG. 10A, the label "V" indicates the vertical gradient g. vIn Figure 10B, the label "H" indicates the subsampled positions for computing the horizontal gradient g h In FIG. 10C, the label "D1" indicates the subsampled positions for computing the d1 diagonal gradient g d1 In Figure 10D, the label "D2" indicates the subsampled positions for computing the d2 diagonal gradient g d2 indicates the subsampled positions for computing

[0105] horizontal gradient g v and the vertical gradient g h Maximum value of

number

number

number

number

number

number

number

number

number

number

number

[0106] The activity value A can be calculated as follows:

number

number

[0107] For the chroma components in a picture, no block classification is applied, so a single set of ALF coefficients can be applied for each chroma component.

[0108] A geometric transformation can be applied to the filter coefficients and the corresponding filter clip values ​​(also called clip values). Before filtering a block (e.g., a 4x4 luma block), for example, the gradient values ​​(e.g., g v , g h , g d1 and / or g d2Depending on the filter coefficients f(k,l) and the corresponding filter clip values ​​c(k,l), a geometric transformation such as a rotation or a diagonal or vertical flip can be applied to the filter coefficients f(k,l) and the corresponding filter clip values ​​c(k,l). The geometric transformation applied to the filter coefficients f(k,l) and the corresponding filter clip values ​​c(k,l) can be equivalent to applying a geometric transformation to the samples within the region supported by the filter. The geometric transformation can make the different blocks to which the ALF is applied more similar by aligning their respective directionality.

[0109] Three geometric transformations, including diagonal flip, vertical flip, and rotation, can be performed as described in equations (9) to (11), respectively. f D (k,l)=f(l,k), c D (k,l)=c(l,k) Equation (9) f V (k,l)=f(k,Kl-1), c V (k,l)=c(k,Kl-1) Equation (10) f R (k,l)=f(Kl-1,k), c R (k,l)=c(Kl-1,k) Equation (11) where K is the size of the ALF or filter, and 0≦k, l≦K−1 are the coordinates of the coefficients. For example, the filter f or clip value matrix (or clip matrix) c has position (0,0) in the upper left corner and position (K−1,K−1) in the lower right corner. Transforms can be applied to the filter coefficients f(k,l) and clip values ​​c(k,l) depending on the gradient values ​​calculated for the block. An example of the relationship between the transforms and the four gradients is summarized in Table 1.

[0110] [Table 1]

[0111] In some embodiments, such as VVC, ALF filter parameters are signaled in an adaptive parameter set (APS) for a picture. In the APS, one or more sets (e.g., up to 25 sets) of luma filter coefficients and clip value indices may be signaled. In one example, one set of the one or more sets may include luma filter coefficients and one or more clip value indices. One or more sets (e.g., up to 8 sets) of chroma filter coefficients and clip value indices may be signaled. To reduce signaling overhead, filter coefficients of different classifications (e.g., having different classification indices) of the luma component may be merged. In the slice header, the index of the APS used for the current slice may be signaled. In examples such as VVC, the signaling of ALF is CTU-based.

[0112] In one embodiment, a clip value index (also referred to as a clipping index) can be decoded from the APS. The clip value index can be used to determine a corresponding clip value, for example, based on a relationship between the clip value index and the corresponding clip value. This relationship can be predefined and stored in the decoder. In one example, this relationship is described by a table, such as a table of clip value indexes and corresponding clip values ​​for the luma component (e.g., used for the luma CB) and a table of clip value indexes and corresponding clip values ​​for the chroma components (e.g., used for the chroma CB). The clip value may depend on the bit depth B. The bit depth B may refer to an internal bit depth, the bit depth of the reconstructed samples in the CB to be filtered, etc. In some examples, the table of clip values ​​(e.g., for luma and / or chroma) is obtained using Equation (12). AlfClip = {round(2 for n∈[0 .. N-1] B-α*n )} Formula (12) where AlfClip is the clip value, B is the bit depth (e.g., bitDepth), N (e.g., N=4) is the number of allowed clip values, and α is a predefined constant value. In one example, α is equal to 2.35. n is the clip value index (also called the clipping index or clipIdx). Table 2 shows an example of a table obtained using equation (12) with N=4. The clipping index n can be 0, 1, 2, and 3 in Table 2. Table 2 can be used for luma blocks or chroma blocks.

[0113] [Table 2]

[0114] In the slice header of the current slice, one or more APS indices (e.g., up to seven APS indices) may be signaled to specify the luma filter sets available for the current slice. The filtering process may be controlled at one or more appropriate levels, such as the picture level, the slice level, the CTB level, etc. In one embodiment, the filtering process may be further controlled at the CTB level. A flag may be signaled to indicate whether an ALF is applied to the luma CTB. The luma CTB may select a filter set from among multiple fixed filter sets (e.g., 16 fixed filter sets) and filter set(s) (also referred to as signaled filter set(s)) signaled in the APS. A filter set index may be signaled to the luma CTB to indicate the filter set to be applied (e.g., from among the multiple fixed filter sets and signaled filter set(s)). The multiple fixed filter sets may be predefined and hard-coded in the encoder and decoder and may be referred to as predefined filter sets.

[0115] For chroma components, an APS index can be signaled in the slice header to indicate the chroma filter set that should be used for the current slice. At the CTB level, if there is more than one chroma filter set in an APS, a filter set index can be signaled per chroma CTB.

[0116] The filter coefficients can be quantized with a norm equal to 128. To reduce multiplication complexity, bitstream adaptation can be applied to ensure that the coefficient values ​​of non-center positions are within the range of -27 to 27-1. In one example, center position coefficients are not signaled in the bitstream and can be assumed to be equal to 128.

[0117] In some embodiments, the syntax and semantics of the clipping index and clip value are defined as follows: alf_luma_clip_idx[ sfIdx ][ j ] may be used to specify the clipping index of the clip value to be used before multiplication by the j-th coefficient of the signaled luma filter indicated by sfIdx. Bitstream conformance requirements may include that the values ​​of alf_luma_clip_idx[ sfIdx ][ j ], for sfIdx=0 to alf_luma_num_filters_signalled_minus1 and j=0 to 11, be in the range 0 to 3, inclusive.

[0118] The luma filter clip value AlfClipL[adaptation_parameter_set_id], with elements AlfClipL[ adaptation_parameter_set_id] [ filtIdx ] [ j ], where filtIdx = 0 to NumAlfFilters - 1 and j = 0 to 11, may be derived as specified in Table 2 depending on bitDepth set equal to BitDepthY and clipIdx set equal to alf_luma_clip_idx[ alf_luma_coeff_delta_idx[ filtIdx ] [ j ].

[0119] alf_chroma_clip_idx[ altIdx ][ j ] can be used to specify the clipping index of the clip value that should be used before multiplying by the j-th coefficient of the alternative chroma filter with index altIdx. Bitstream conformance requirements include that the value of alf_chroma_clip_idx[ altIdx ][ j ], for altIdx=0 to alf_chroma_num_alt_filters_minus1, j=0 to 5, be in the range 0 to 3, inclusive.

[0120] Chroma filter clip values ​​AlfClipC[ adaptation_parameter_set_id ][ altIdx ][ j ] with elements AlfClipC[ adaptation_parameter_set_id ][ altIdx ][ j ], where altIdx = 0 to alf_chroma_num_alt_filters_minus1, j = 0 to 5, can be derived as specified in Table 2 depending on bitDepth set equal to BitDepthC and clipIdx set equal to alf_chroma_clip_idx[ altIdx ][ j ].

[0121] In one embodiment, the filtering process can be described as follows: On the decoder side, when ALF is enabled for a CTB, the sample R(i,j) in a CU (or CB) can be filtered, and a filtered sample value R'(i,j) is obtained, as shown below using equation (13). In one example, each sample in a CU is filtered.

number

[0122] The selected clip value can be coded in the "alf_data" syntax element as follows: An appropriate coding scheme (e.g., Golomb coding) can be used to code the clipping index corresponding to the selected clip value as shown in Table 2. The coding scheme can be the same coding scheme used to code the filter set index.

[0123] In one embodiment, a virtual boundary filtering process can be used to reduce the line buffer requirements of ALF. Therefore, modified block classification and filtering can be employed for samples near CTU boundaries (e.g., horizontal CTU boundaries). The virtual boundary (1130) is defined as the horizontal CTU boundary (1120) as "N" as shown in FIG. 11A. samples " can be defined as a line by shifting the sample by N samplescan be a positive integer. In one example, N samples is equal to 4 for the luma component and N samples is equal to 2 for the chroma component.

[0124] Referring to Figure 11A, modified block classification can be applied to the luma component. In one example, a one-dimensional Laplacian gradient calculation for a 4x4 block (1110) above a virtual boundary (1130) uses only samples above the virtual boundary (1130). Similarly, referring to Figure 11B, a one-dimensional Laplacian gradient calculation for a 4x4 block (1111) below a virtual boundary (1131) shifted from the CTU boundary (1121) uses only samples below the virtual boundary (1131). Thus, the quantization of the activity value A can be scaled by taking into account the reduction in the number of samples used in the one-dimensional Laplacian gradient calculation.

[0125] For the filtering process, a symmetric padding operation at the virtual boundary can be used for both the luma and chroma components. Figures 12A-12F show examples of such modified ALF filtering for the luma component at the virtual boundary. If the sample being filtered is located below the virtual boundary, adjacent samples located above the virtual boundary can be padded. If the sample being filtered is located above the virtual boundary, adjacent samples located below the virtual boundary can be padded. Referring to Figure 12A, adjacent sample C0 can be padded with sample C2 located below the virtual boundary (1210). Referring to Figure 12B, adjacent sample C0 can be padded with sample C2 located above the virtual boundary (1220). Referring to Figure 12C, adjacent samples C1-C3 can be padded with samples C5-C7 located below the virtual boundary (1230), respectively. Referring to Figure 12D, adjacent samples C1-C3 can be padded with samples C5-C7 located above the virtual boundary (1240), respectively. Referring to Figure 12E, adjacent samples C4-C8 can be padded with samples C10, C11, C12, C11, and C10, respectively, located below the virtual boundary (1250). Referring to Figure 12F, adjacent samples C4-C8 can be padded with samples C10, C11, C12, C11, and C10, respectively, located above the virtual boundary (1260).

[0126] In some instances, the above description can be appropriately adapted when a sample and an adjacent sample are located to the left (or right) and right (or left) of a virtual boundary.

[0127] A maximum coding unit (LCU)-aligned picture quadtree partitioning can be used. To improve coding efficiency, a coding unit-synchronized picture quadtree-based adaptive loop filter can be used in video coding. In one example, a luma picture is divided into multiple multi-level quadtree partitions, and each partition boundary is aligned with the maximum coding unit (LCU) boundary. Each partition can have a filtering process and can therefore be called a filter unit or filtering unit (FU).

[0128] The two-pass encoding flow is described as follows: In the first pass, the quadtree division pattern and the best filter (or optimal filter) of each FU can be determined. During the decision process, the filtering distortion can be estimated by fast filtering distortion estimation (FFDE). According to the determined quadtree division pattern of the FU (e.g., all FUs) and the selected filter, the reconstructed picture can be filtered. In the second pass, the CU-synchronized ALF on / off control can be performed. According to the ALF on / off result, the initially filtered picture is partially restored by the reconstructed picture.

[0129] A top-down partitioning strategy can be adopted to divide a picture into multi-level quadtree partitions by using a rate-distortion criterion. Each partition can be referred to as an FU. The partitioning process can align the quadtree partitions with LCU boundaries, as shown in FIG. 13. FIG. 13 illustrates LCU-aligned picture quadtree partitioning according to one embodiment of the present disclosure. In one example, the coding order of the FUs follows the z-scan order. For example, referring to FIG. 13, a picture is divided into 10 FUs (e.g., FU0-FU9), and the coding order is from FU0 to FU9, e.g., FU0, FU1, FU2, FU3, FU4, FU5, FU6, FU7, FU8, and FU9.

[0130] To indicate the picture quadtree division pattern, the division flags can be coded and sent in z-scan order. Figure 14 shows the quadtree division pattern corresponding to Figure 13 according to one embodiment of the present disclosure. As shown in the example of Figure 14, the quadtree division flags are coded in z-scan order.

[0131] A filter for each FU can be selected from two filter sets based on a rate-distortion criterion. The first set can have newly derived ½-symmetric square and diamond-shaped filters for the current FU. The second set can be from a time-delay filter buffer. The time-delay filter buffer can store filters previously derived for FUs in previous pictures. The filter with the smallest rate-distortion cost of the two filter sets can be selected for the current FU. Similarly, if the current FU is not the smallest FU and can be further divided into four child FUs, the rate-distortion costs of those four child FUs can be calculated. By recursively comparing the rate-distortion costs of the division and no division, a picture quadtree division pattern can be determined.

[0132] In some examples, the maximum quadtree division level is 2, and therefore the maximum number of FUs is 16. When determining the quadtree division, correlation values ​​for deriving Wiener coefficients for the 16 FUs at the lowest quadtree level (smallest FUs) can be reused. For the remaining FUs, Wiener filters for the remaining FUs can be derived from the correlations of the 16 FUs at the lowest quadtree level. Therefore, in one example, only one frame buffer access is required to derive filter coefficients for all FUs.

[0133] After the quadtree division pattern is determined, CU-synchronized ALF on / off control can be performed to further reduce filtering distortion. By comparing the filtering distortion with the non-filtering distortion, the leaf CU can explicitly switch ALF on / off in the corresponding local region. Coding efficiency can be further improved by redesigning the filter coefficients according to the ALF on / off result. In one example, the redesign process requires additional frame buffer access. Therefore, in some examples, such as a coding unit-synchronized picture quadtree-based adaptive loop filter (CS-PQALF) encoder design, redesign is not required after the CU-synchronized ALF on / off decision to minimize the number of frame buffer accesses.

[0134] The cross-component filtering process may apply a cross-component filter, such as a cross-component adaptive loop filter (CC-ALF). The cross-component filter may use luma sample values ​​of a luma component (e.g., luma CB) to refine a chroma component (e.g., chroma CB corresponding to the luma CB). In one example, the luma CB and the chroma CB are included in a CU.

[0135] Figure 15 illustrates a cross-component filter (e.g., CC-ALF) used to generate chroma components according to one embodiment of the present disclosure. In some examples, Figure 15 illustrates a filtering process for a first chroma component (e.g., first chroma CB), a second chroma component (e.g., second chroma CB), and a luma component (e.g., luma CB). The luma component may be filtered by a sample adaptive offset (SAO) filter (1510) to generate an SAO-filtered luma component (1541). The SAO-filtered luma component (1541) may be further filtered by an ALF luma filter (1516) to become a filtered luma CB (1561) (e.g., "Y").

[0136] The first chroma component may be filtered by an SAO filter (1512) and an ALF chroma filter (1518) to generate a first intermediate component (1552). Further, the SAO-filtered luma component (1541) may be filtered by a cross-component filter (e.g., CC-ALF) for the first chroma component (1521) to generate a second intermediate component (1542). Subsequently, a filtered first chroma component (1562) (e.g., “Cb”) may be generated based on at least one of the second intermediate component (1542) and the first intermediate component (1552). In one example, the filtered first chroma component (1562) (e.g., “Cb”) may be generated by combining the second intermediate component (1542) and the first intermediate component (1552) using an adder (1522). The cross-component adaptive loop filtering process for the first chroma component may include steps performed by the CC-ALF (1521) and steps performed by, for example, an adder (1522).

[0137] The above description can be adapted to the second chroma component. The second chroma component can be filtered by the SAO filter (1514) and the ALF chroma filter (1518) to generate a third intermediate component (1553). Furthermore, the SAO-filtered luma component (1541) can be filtered by a cross-component filter (e.g., CC-ALF) for the second chroma component (1531) to generate a fourth intermediate component (1543). Subsequently, a filtered second chroma component (1563) (e.g., “Cr”) can be generated based on at least one of the fourth intermediate component (1543) and the third intermediate component (1553). In one example, the filtered second chroma component (1563) (e.g., “Cr”) can be generated by combining the fourth intermediate component (1543) and the third intermediate component (1553) using an adder (1532). In one example, the cross-component adaptive loop filtering process for the second chroma component may include steps performed by a CC-ALF (1531) and steps performed, for example, by an adder (1532).

[0138] The cross-component filters (e.g., CC-ALF(1521), CC-ALF(1531)) can operate by applying a linear filter having any suitable filter shape to the luma component (or luma channel) to refine each chroma component (e.g., first chroma component, second chroma component).

[0139] FIG. 16 illustrates an example of a filter (1600) according to one embodiment of the present disclosure. The filter (1600) may include non-zero filter coefficients and zero filter coefficients. The filter (1600) has a diamond shape (1620) formed by the filter coefficients (1610) (shown as solid circles). In one example, the non-zero filter coefficients in the filter (1600) are included in the filter coefficients (1610), and the filter coefficients not included in the filter coefficients (1610) are zero. Thus, the non-zero filter coefficients in the filter (1600) are included in the diamond shape (1620), and the filter coefficients not included in the diamond shape (1620) are zero. In one example, the number of filter coefficients in the filter (1600) is equal to the number of filter coefficients (1610), which is 18 in the example shown in FIG. 14.

[0140] The CC-ALF may include any suitable filter coefficients (also referred to as CC-ALF filter coefficients). Returning to Figure 15, the CC-ALF (1521) and the CC-ALF (1531) may have the same filter shape, such as the diamond shape (1620) shown in Figure 14, and the same number of filter coefficients. In one example, the values ​​of the filter coefficients in the CC-ALF (1521) are different from the values ​​of the filter coefficients in the CC-ALF (1531).

[0141] In general, filter coefficients (e.g., non-zero filter coefficients) in CC-ALF can be sent, for example, in APS. In one example, the filter coefficients can be multiplied by a factor (e.g., 2 10) and may be rounded for fixed-point representation. The application of CC-ALF may be controlled by variable block sizes and signaled by a context-coded flag (e.g., a CC-ALF enable flag) received for each block of samples. Context-coded flags, such as the CC-ALF enable flag, may be signaled at any appropriate level, such as the block level. Block sizes, along with CC-ALF enable flags, may be received at the slice level for each chroma component. In some examples, block sizes (in chroma samples) of 16x16, 32x32, and 64x64 may be supported.

[0142] In one example, the syntax changes of CC-ALF are listed in Table 3 below.

[0143] [Table 3]

[0144] The meaning of CC-ALF related syntax can be explained as follows:

[0145] alf_ctb_cross_component_cb_idc[xCtb >> CtbLog2SizeY][yCtb >> CtbLog2SizeY] equal to 0 may indicate that the cross-component Cb filter is not applied to the block of Cb color component samples at luma position (xCtb, yCtb).

[0146] alf_cross_component_cb_idc[ xCtb >> CtbLog2SizeY ][ yCtb >> CtbLog2SizeY ] not equal to 0 may indicate that the alf_cross_component_cb_idc[ xCtb >> CtbLog2SizeY ][ yCtb >> CtbLog2SizeY ]th cross-component Cb filter is applied to the block of Cb color component samples at luma position (xCtb, yCtb).

[0147] alf_ctb_cross_component_cr_idc[ xCtb >> CtbLog2SizeY ][ yCtb >> CtbLog2SizeY ] equal to 0 may indicate that the cross-component Cr filter is not applied to the block of Cr color component samples at luma position (xCtb, yCtb).

[0148] alf_cross_component_cr_idc[ xCtb >> CtbLog2SizeY ][ yCtb >> CtbLog2SizeY ] not equal to 0 may indicate that the alf_cross_component_cr_idc[ xCtb >> CtbLog2SizeY ][ yCtb >> CtbLog2SizeY ]th cross-component Cr filter is applied to the block of Cr color component samples at luma position (xCtb, yCtb).

[0149] Examples of chroma sampling formats are described below. Generally, a luma block may correspond to one or more chroma blocks, such as two chroma blocks. The number of samples in each of the one or more chroma blocks may be less than the number of samples in the luma block. The chroma subsampling format (also referred to as chroma subsampling format and specified, for example, by chroma_format_idc) may indicate a chroma horizontal subsampling factor (e.g., SubWidthC) and a chroma vertical subsampling factor (e.g., SubHeightC) between each of the one or more chroma blocks and the corresponding luma block. In one example, the chroma subsampling format is 4:2:0, and thus the chroma horizontal subsampling factor (e.g., SubWidthC) and the chroma vertical subsampling factor (e.g., SubHeightC) are 2, as shown in Figures 15A-15B. In one example, the chroma subsampling format is 4:2:2, so the chroma horizontal subsampling factor (e.g., SubWidthC) is 2 and the chroma vertical subsampling factor (e.g., SubHeightC) is 1. In one example, the chroma subsampling format is 4:4:4, so the chroma horizontal subsampling factor (e.g., SubWidthC) and the chroma vertical subsampling factor (e.g., SubHeightC) are 1. The chroma sample type (also referred to as the chroma sample position) may indicate the relative position of a chroma sample within a chroma block with respect to at least one corresponding luma sample within the luma block.

[0150] 17A-17B illustrate exemplary locations of chroma samples relative to luma samples according to an embodiment of the present disclosure. Referring to FIG. 17A, luma samples (1701) are arranged in rows (1711)-(1718). The luma samples (1701) shown in FIG. 17A may represent a portion of a picture. In one example, a luma block (e.g., luma CB) includes luma samples (1701). The luma block may correspond to two chroma blocks having a 4:2:0 chroma subsampling format. In one example, each chroma block includes chroma samples (1703). Each chroma sample (e.g., chroma sample (1703(1))) corresponds to four luma samples (e.g., luma samples (1701(1)) to (1701(4))). In one example, the four luma samples are the top-left sample (1701(1)), the top-right sample (1701(2)), the bottom-left sample (1701(3)), and the bottom-right sample (1701(4)). A chroma sample (e.g., (1703(1))) corresponds to the top-left sample (1701(1)) and the bottom-left sample (1701(4)). The chroma sample type of the chroma block having chroma sample (1703) located at the left center position between the top left sample (1701(1)) and the bottom left sample (1701(3)) can be referred to as chroma sample type 0. Chroma sample type 0 indicates relative position 0, which corresponds to the left center position halfway between the top left sample (1701(1)) and the bottom left sample (1701(3)). The four luma samples (e.g., (1701(1)) to (1701(4))) can be referred to as neighboring luma samples of chroma sample (1703)(1).

[0151] In one example, each chroma block includes a chroma sample (1704). The above description of the chroma sample (1703) may be adapted to the chroma sample (1704), and therefore, detailed description may be omitted for brevity. Each of the chroma samples (1704) may be located at the center of four corresponding luma samples, and the chroma sample type of a chroma block having the chroma sample (1704) may be referred to as chroma sample type 1. Chroma sample type 1 indicates relative position 1, which corresponds to the center position of the four luma samples (e.g., (1701(1)) to (1701(4))). For example, one of the chroma samples (1704) may be located at the center of the luma samples (1701(1)) to (1701(4)).

[0152] In one example, each chroma block includes a chroma sample (1705). Each chroma sample (1705) can be located at the top left position, which is the same as the top left sample of the four corresponding luma samples (1701). The chroma sample type of a chroma block having a chroma sample (1705) can be referred to as chroma sample type 2. Therefore, each chroma sample (1705) is located at the same location as the top left sample of the four luma samples (1701) corresponding to the respective chroma sample. Chroma sample type 2 indicates relative position 2, which corresponds to the top left position of the four luma samples (1701). For example, one of the chroma samples (1705) can be located at the top left position of luma samples (1701(1)) to (1701(4)).

[0153] In one example, each chroma block includes chroma samples (1706). Each of the chroma samples (1706) can be located at the top center between the corresponding top-left sample and the corresponding top-right sample, and the chroma sample type of a chroma block having the chroma sample (1706) can be referred to as chroma sample type 3. Chroma sample type 3 indicates relative position 3, which corresponds to the top center between the top-left sample and the top-right sample. For example, one of the chroma samples (1706) can be located at the top center of the luma samples (1701(1)) to (1701(4)).

[0154] In one example, each chroma block includes a chroma sample (1707). Each chroma sample (1707) can be located at the bottom left position, which is the same as the bottom left sample of the four corresponding luma samples (1701). The chroma sample type of a chroma block having a chroma sample (1707) can be referred to as chroma sample type 4. Therefore, each chroma sample (1707) is located at the same location as the bottom left sample of the four luma samples (1701) corresponding to the respective chroma sample. Chroma sample type 4 indicates relative position 4, which corresponds to the bottom left position of the four luma samples (1701). For example, one of the chroma samples (1707) can be located at the bottom left position of luma samples (1701(1)) to (1701(4)).

[0155] In one example, each chroma block includes chroma samples (1708). Each of the chroma samples (1708) is located at the bottom center between the bottom left and bottom right samples, and the chroma sample type of a chroma block having the chroma samples (1708) can be referred to as chroma sample type 5. Chroma sample type 5 indicates relative position 5, which corresponds to the bottom center between the bottom left and bottom right samples of the four luma samples (1701). For example, one of the chroma samples (1708) can be located between the bottom left and bottom right samples of luma samples (1701(1)) to (1701(4)).

[0156] In general, any suitable chroma sample type can be used for the chroma subsampling format. Chroma sample types 0-5 are exemplary chroma sample types described for the chroma subsampling format 4:2:0. Additional chroma sample types can be used for the chroma subsampling format 4:2:0. Additionally, other chroma sample types and / or variations of chroma sample types 0-5 can be used for other chroma subsampling formats, such as 4:2:2, 4:4:4, etc. In one example, a chroma sample type combining chroma sample (1705) and chroma sample (1707) is used for the chroma subsampling format 4:2:2.

[0157] In one example, a luma block is considered to have alternating rows such as rows 1711-1712, each containing the top two samples (e.g., 1701(1)-1701(2)) of the four luma samples (e.g., 1701(1)-1701(4))) and the bottom two samples (e.g., 1701(3)-1701(4)) of the four luma samples (e.g., 1701(1)-1701(4)). Thus, rows 1711, 1713, 1715, and 1717 may be defined as: The current row (also referred to as the upper field) and rows 1712, 1714, 1716, and 1718 can be referred to as the next rows (also referred to as the bottom field). Four luma samples (e.g., 1701(1)) through 1701(4)) are located in the current row (e.g., 1711) and the next row (e.g., 1712). Relative positions 2 and 3 are located in the current row, relative positions 0 and 1 are located between each current row and the respective next row, and relative positions 4 and 5 are located in the next row.

[0158] Chroma samples 1703, 1704, 1705, 1706, 1707, or 1708 are located in rows 1751-1754 within each chroma block. The specific locations of rows 1751-1754 may depend on the chroma sample's chroma sample type. For example, for chroma samples 1703-1704 with chroma sample types 0-1, respectively, row 1751 is located between rows 1711 and 1712. For chroma samples 1705-1706 with chroma sample types 2-3, respectively, row 1751 is located in the same location as the current row 1711. For chroma samples 1707-1708 having respective chroma sample types 4-5, row 1751 is co-located with the next row 1712. The above description can be adapted appropriately for rows 1752-1754, and a detailed description will be omitted for the sake of brevity.

[0159] Any suitable scanning method may be used to display, store, and / or transmit the luma block and corresponding chroma block(s) described above in Figure 17A. In one example, progressive scanning is used.

[0160] Interlaced scanning can be used, as shown in Figure 17B. As mentioned above, the chroma subsampling format is 4:2:0 (e.g., chroma_format_idc equals 1). In one example, the variable chroma location type (e.g., ChromaLocType) indicates the current row (e.g., ChromaLocType is chroma_sample_loc_type_top_field) or the next row (e.g., ChromaLocType is chroma_sample_loc_type_bottom_field). The current rows (1711), (1713), (1715), and (1717) and the next rows (1712), (1714), (1716), and (1718) can be scanned separately, for example, the current rows (1711), (1713), (1715), and (1717) can be scanned first, followed by the next rows (1712), (1714), (1716), and (1718). The current row can include luma sample (1701), and the next row can include luma sample (1702).

[0161] Similarly, corresponding chroma blocks can be interlaced. Rows (1751) and (1753) containing chroma samples (1703), (1704), (1705), (1706), (1707), or (1708) with no fill can be referred to as the current row (or current chroma row), and rows (1752) and (1754) containing chroma samples (1703), (1704), (1705), (1706), (1707), or (1708) with gray fill can be referred to as the next row (or next chroma row). In one example, during interlacing, rows (1751) and (1753) are scanned first, followed by rows (1752) and (1754).

[0162] Constrained directional enhancement filters (CDEFs) can be used in video coding. In-loop CDEFs can be used to remove coding artifacts while preserving image details. In some examples, such as HEVC, the sample adaptive offset (SAO) algorithm achieves a similar objective by defining signal offsets for different classes of pixels. Unlike SAO, CDEFs are nonlinear spatial filters. In some examples, the filter design is constrained to be easily vectorizable (e.g., implementable with single instruction multiple data (SIMD) operations), which is not true for other nonlinear filters such as median filters and bilateral filters.

[0163] In one example, the design of the CDEF is inspired by the following observations: In some examples, the amount of ringing artifacts in a coded image may be approximately proportional to the quantization step size. The amount of detail is a characteristic of the input image. However, the minimum detail retained in the quantized image may be proportional to the quantization step size. For a given quantization step size, the magnitude of the ringing artifacts may be smaller than the magnitude of the detail.

[0164] The CDEF can identify the orientation of each block. The CDEF can then adaptively filter along the identified orientation to smaller angles along a direction rotated 45° from the identified orientation. The filter strength can be explicitly signaled, allowing for a high degree of control over blurring. An efficient encoder search can be designed for the filter strength. The CDEF can be based on two in-loop filters, and the combined filter can be used for video coding, such as in the AV1 codec.

[0165] The direction search can operate on reconstructed pixels (or samples), for example, after a deblocking filter. Because the reconstructed pixels are available to the decoder, the direction may not require signaling. The search can operate on blocks (e.g., 8x8 blocks) that, when applied to a quantized image, are small enough to properly handle non-linear edges and large enough to reliably estimate direction. Having a consistent direction across the 8x8 region facilitates vectorization of the filter. For each block, the direction that best matches the pattern within the block can be determined by minimizing a difference measure, such as the sum of squared differences (SSD) or RMS error, for each of the quantized block and the fully directional block (1830). In one example, a fully directional block (e.g., one of (1830)) is a block in which all pixels along a line in a certain direction have the same value. Figure 18 shows an example of a direction search for an 8x8 block (1810) according to one embodiment of the present disclosure. In the example shown in Figure 18, the 45 degree direction (1823) is selected from the set of directions (1820) because it minimizes the error. For example, the error in the 45 degree direction is 12, which is the smallest among the errors in the range of 12 to 87 shown in row (1840).

[0166] A nonlinear low-pass directional filter is described below. Identifying the direction can help align filter taps along the identified direction to reduce ringing artifacts while preserving directional edges or patterns. However, in some examples, directional filtering alone cannot sufficiently reduce ringing artifacts. It is desirable to use additional filter taps for pixels that are not along the main direction (e.g., the identified direction). These additional filter taps can be treated more conservatively to reduce the risk of blurring. Therefore, the CDEF can define primary and secondary taps. In one example, a complete two-dimensional (2D) CDEF filter is expressed as follows:

number

[0167] In equation (14), D is the damping parameter and S (p) and S (s) are the intensities of the primary and secondary taps, respectively, and the function round(·) allows rounding values ​​away from zero.

number

number

[0168] In post-deblocking video coding, such as AV1, a set of in-loop restoration schemes can be used to generally denoise and improve edge quality beyond the deblocking operation. The set of in-loop restoration schemes can be switchable within a frame (or picture) for tiles of appropriate size. Some examples of in-loop restoration schemes are described based on a separable symmetric Wiener filter and a dual self-guided filter with subspace projection. Because content statistics can vary significantly within a frame, these tools can be integrated into a switchable framework that can trigger different tools in different regions of the frame.

[0169] A separable symmetric Wiener filter can be described below. One restoration tool can be a Wiener filter. Every pixel (or sample) in a corrupted frame can be reconstructed as a non-causal filtered version of the pixels in a w × w window around it, where w = 2r + 1, odd for integer r. The two-dimensional filter taps are denoted by w 2 can be represented as a vector F in column vectorized form with ×1 elements, and a simple linear minimum mean square error (LMMSE) optimization gives F=H -1 We can obtain the filter parameters given by M, where H is the sum of E[XX T ], the autocovariance of x, w in a w × w window around the pixel 2 is a column vectorized version of the samples, E[YX T ] is the cross-correlation between x and the scalar source sample y to be estimated. The encoder can estimate H and M from their realizations in the deblocked frame and the source, and send the resulting filter F to the decoder. However, in some instances, w 2 There can be a significant bitrate cost in transmitting the taps. Furthermore, non-separable filtering can significantly complicate decoding. Therefore, several additional constraints can be placed on the properties of F. For example, F is constrained to be separable so that the filtering can be implemented as separate horizontal and vertical w-tap convolutions. In one example, each of the horizontal and vertical filters is constrained to be symmetric. Furthermore, in one example, the horizontal and vertical filter coefficients can be assumed to sum to one.

[0170] Dual self-guided filtering with subspace projection can be described below. In some examples, guided filtering can be used in image filtering, where a local linear model is used to calculate the filtered output from the unfiltered samples x. The local linear model can be written as: y=Fx+G Equation (15) where F and G can be determined based on the degraded image and a guide image (also called a guide image) in the neighborhood of the filtered pixel. If the guide image is identical to the degraded image, the resulting self-guided filtering can have the effect of edge-preserving smoothing. According to some aspects of the present disclosure, a particular form of self-guided filtering can depend on two parameters, namely, a radius r and a noise parameter e, which can be enumerated as follows:

[0171] 1. The mean μ and variance σ of the pixels in a (2r+1) × (2r+1) window around each pixel 2 The mean value μ and variance σ of the pixels are obtained. 2 The acquisition of can be efficiently performed by box filtering based on integral imaging.

[0172] 2. Calculate the parameters f and g for each pixel based on equation (16). f=σ 2 / (σ 2 +e);g=(1-f)μ Equation (16)

[0173] 3. Calculate F and G for each pixel as the average of the values ​​of parameters f and g in a 3x3 window around the pixel being used.

[0174] The filtering can be controlled by the radius r and the noise parameter e, where a larger radius r can mean higher spatial variance and a higher noise parameter e can mean higher range variance.

[0175] 19 shows an example of subspace projection according to one embodiment of the present disclosure. In the example shown in FIG. 19, the subspace projection uses readily available reconstruction values ​​X1 and X2 to obtain a final reconstruction value X that is closer to the source Y. fEven if the readily available reconstruction values ​​X1 and X2 are not close to the source Y, a suitable multiplier {α, β} can bring the readily available reconstruction values ​​X1 and X2 very close to the source Y if the readily available reconstruction values ​​X1 and X2 move in the right direction. In one example, the final reconstruction value X f is obtained based on the following equation (17). X f =X+α(X1-X)+β(X2-X) Equation (17)

[0176] A loop filtering method called a cross-component sample offset (CCSO) filter, or CCSO, can be used to reduce distortion in the reconstructed samples. In the CCSO filtering process, a nonlinear mapping can be used to determine an output offset based on the processed input reconstructed samples of a first color component. The output offset can be added to the reconstructed samples of a second color component in the CCSO filtering process.

[0177] The input reconstructed samples may be from a first color component located in a filter support area, as shown in FIG. 20. FIG. 20 illustrates an example of a filter support area in a CCSO filter according to one embodiment of the present disclosure. The filter support area may include four reconstructed samples, namely, p0, p1, p2, and p3. The four input reconstructed samples may trace a vertical and horizontal cross shape. In one example, a central sample (denoted as c) in the first color component and a sample to be filtered (denoted as f) in the second color component are co-located. When processing the input reconstructed samples, the following steps may be applied:

[0178] Step 1: The delta values ​​(e.g., differences) between the four reconstruction samples p0, p1, p2, and p3 and the center sample c are calculated and denoted as m0, m1, m2, and m3, respectively. For example, the delta value between p0 and c is m0.

[0179] Step 2: The delta values ​​m0-m3 can be further quantized. The quantized values ​​can be represented as d0, d1, d2, and d3. In one example, the quantized values ​​are −1, 0, or 1 based on the following quantization process: If mi<-N, di=-1 Equation (18) - When N≦mi≦N, di=0 Equation (19) If mi>N, di=1 Equation (20) where N is the quantization step size, example values ​​of N are 4, 8, 12, 16, etc., di and mi refer to the respective quantization and delta values, and i is 0, 1, 2, or 3.

[0180] The quantized values ​​d0-d3 can be used to identify nonlinear mapping combinations. In the example shown in Figure 20, the CCSO filter has four filter inputs d0-d3, and each filter input can have one of three quantized values ​​(e.g., -1, 0, and 1), so the total number of combinations is 81 (e.g., 3 4 21A-21C show an example of 81 combinations according to one embodiment of the present disclosure. The last column may represent the output offset value for each combination. The output offset value may be an integer such as 0, 1, -1, 3, -3, 5, -5, -7, etc.

[0181] The final filtering process of the CCSO filter can be applied as follows: f'=clip(f+s) Equation (21) where f is the reconstructed sample to be filtered and s is the output offset value retrieved, for example, from the tables of Figures 21A-21C. In the example shown in equation (21), the filtered sample value f' of the reconstructed sample to be filtered f can be further clipped to a range associated with the bit depth.

[0182] A local sample offset (LSO) method or LSO filtering process can be used in video coding. In LSO, a filtering technique similar to that used in CCSO can be applied. However, the output offset value can be applied to a color component that is the same color component as the input reconstructed sample used in the filtering process. Thus, in LSO, the input reconstructed samples used in the filtering process (e.g., p0-p3 and c) and the reconstructed sample to be filtered (e.g., f) are in the same component, such as the luma component, the chroma component, or any suitable component. In LSO, the filter shape can be similar or identical to that of CCSO (e.g., as shown in FIG. 20).

[0183] A less complex 3-tap CCSO design can be used for video coding. FIG. 22 shows an example implementation of CCSO according to one embodiment of the present disclosure. Eight different filter shapes can be defined. Each filter shape can define the positions of three reconstructed samples (also called three taps) in a first component (also called a first color component). The three reconstructed samples can include a center sample (denoted as c). In one example, the reconstructed sample in the second color component to be filtered is co-located with the center sample c. For clarity, the reconstructed sample in the second color component to be filtered is not shown in FIG. 22.

[0184] SAO filters can be used in video coding. In examples such as HEVC, an SAO filter or SAO filtering process can be applied to the reconstructed signal after the deblocking filter, for example, by using an offset value in the slice header. For luma samples, the encoder can determine whether an SAO filter is applied to the current slice. If an SAO filter is enabled, the current picture can be recursively divided into four subregions, as shown in Table 4, and one of six SAO types (e.g., SAO types 1-6) can be selected for each subregion. The SAO filter can reduce distortion by classifying reconstructed pixels into multiple categories and adding an offset to pixels of each category within the current subregion. Edge characteristics can be used for pixel classification in SAO types 1-4, and pixel intensity can be used for pixel classification in SAO types 5-6.

[0185] [Table 4]

[0186] Using band offsets (BO), the pixels of a subregion (e.g., all pixels) can be classified into multiple bands, where each band can contain pixels of the same intensity interval. The intensity range can be evenly divided into multiple intervals (e.g., 32 intervals) from the minimum intensity value (e.g., 0) to the maximum intensity value (e.g., 255 for 8-bit pixels), and each interval can have an offset. Subsequently, the multiple intervals or bands (e.g., 32 bands) can be divided into two groups. One group can contain the 16 central bands, and the other group can contain the 16 remaining bands. In one example, only the offset of one group is sent. For pixel classification operations in BO, the most significant 5 bits of each pixel can be directly used as a band index.

[0187] Edge offset (EO) can use four one-dimensional three-pixel patterns for pixel classification that considers edge direction information, as shown in Figure 23. Figure 23 shows examples of four one-dimensional three-pixel patterns for pixel classification in EO. From left to right, the four one-dimensional three-pixel patterns are a one-dimensional 0-degree pattern (2310), a one-dimensional 90-degree pattern (2320), a one-dimensional 135-degree pattern (2330), and a one-dimensional 45-degree pattern (2340). For each subregion of a picture (e.g., the current picture), one of the four patterns can be selected to classify pixels into multiple categories by comparing each pixel with its two neighboring pixels. The selection can be sent in the bitstream as side information. Table 5 shows the pixel classification rules for EO.

[0188] [Table 5]

[0189] In one example, it is desirable for the decoder-side SAO to operate independently of the LCU so as to conserve line buffers. To operate the SAO independently of the LCU, in one example, when the 90-degree, 135-degree, and 45-degree classification patterns are selected, the pixels in the top row and bottom row in each LCU are not subjected to SAO processing, and when the 0-degree, 135-degree, and 45-degree patterns are selected, the pixels in the leftmost column and the rightmost column in each LCU are not subjected to SAO processing.

[0190] Table 6 below describes the syntax that may be signaled to a CTU when parameters are not merged from neighboring CTUs.

[0191] [Table 6]

[0192] Certain filters or filter shapes, such as those used in the CCSO filtering process, require accessing one or more neighboring samples that may be relatively far away from a central sample co-located with the reconstructed sample to be filtered (e.g., filter shapes 5, 7, and 8 shown in FIG. 22 ). In some examples, accessing one or more neighboring samples that are relatively far away from the central sample may be too costly to implement in hardware for the CCSO filter. Therefore, a sample offset filter (e.g., CCSO or LSO) that uses samples relatively close to the central sample may be used, as shown in FIGS. 24-28. In some examples, accessing rows below the central sample and relatively far away from it does not increase implementation costs as significantly as accessing rows above the central sample and relatively far away from it. Therefore, in some examples, a constraint is applied to access a limited number of rows above the central sample, and a sample offset filter such as that shown in FIG. 29 may be used.

[0193] According to aspects of the present disclosure, a sample offset filter (or sample offset filtering process) may refer to a filter or filtering process in which reconstructed samples of a first component (also referred to as a first color component) are used as input to the sample offset filter (or sample offset filtering process), and the output of the sample offset filter (or sample offset filtering process) may be applied to reconstructed samples (to be filtered) of a second component (or second color component). As shown in Equation (21), the reconstructed samples of the second component may be filtered based on the output of the sample offset filter. The sample offset filter may be an in-loop filter whose output (also referred to as an output value) is applied to the reconstructed samples as an offset to remove coding artifacts while preserving details of the second component. The sample offset filter may include one of an LSO, a CCSO, and an adaptive sample offset filter (ASO).

[0194] The first component, second component, etc. components may be any suitable components in a color space used in video coding (e.g., YCrCB, RGB, etc.), such as a luma component, a chroma component, a Y component, a Cb component, a Cr component, an R component, a G component, a B component, etc. The second component may also be referred to as a current component, because the reconstructed sample to be filtered will be processed by an encoder and / or decoder.

[0195] In one embodiment, the first component (or first color component) is different from the second component (or second color component), and the sample offset filter is referred to as a CCSO filter. More specifically, CCSO may refer to a filter or filtering process in which reconstructed samples of the first component are used as input to a CCSO filter (or CCSO filtering process), and the output of the CCSO filter (or CCSO filtering process) may be applied to the reconstructed samples (to be filtered) of the second component (or second color component). As shown in Equation (21), the reconstructed samples of the second component may be filtered based on the output of the CCSO filter. In one example, in the case of CCSO, the first component is a luma component, and the second component is a chroma component. In one example, in the case of CCSO, the first component and the second component are two different chroma components.

[0196] In one embodiment, the first component of the sample offset filter is the second component of the sample offset filter, and the sample offset filter may be referred to as an LSO filter. The LSO filter may be a filter or filtering process in which reconstructed samples of a first component (or first color component) (e.g., Y, Cb, or Cr) are used as input, and the output may be applied to the same first component, for example, to filter the reconstructed samples of the first component.

[0197] When the sample offset filter is applied to a reconstructed sample (to be filtered) of a current component (or current color component) in a current picture, the filter shape of the sample offset filter can be one of a plurality of filter shapes (or a plurality of filters). Each of the plurality of filter shapes can include a first reconstructed sample of a first component (or first color component) in the current picture. According to aspects of the present disclosure, for each of the plurality of filter shapes, the top sample of the first reconstructed sample is located within N rows (or lines) from a central sample (also referred to as a central reconstructed sample) of the first component that is co-located with the reconstructed sample (to be filtered) of the current component. The reconstructed sample to be filtered is in the current component.

[0198] In some embodiments, the center sample of the first component is co-located with the reconstructed sample (to be filtered) of the current component. In one example, the first component is a luma component and the current component is a chroma component. Referring to Figure 17A, the center sample is the luma sample (1701(1)), the reconstructed sample to be filtered is the chroma sample 1705, and the center sample (e.g., (1701(1))) and the reconstructed sample to be filtered (e.g., (1705)) are co-located.

[0199] In some embodiments, the central sample of the first component is at a slightly different position from the position of the reconstructed sample (to be filtered) of the current component. Referring to Figure 17A, the central sample is a luma sample (1701(1)), the reconstructed sample to be filtered is a chroma sample (1703(1)), and the central sample (e.g., (1701(1))) and the reconstructed sample to be filtered (e.g., (1703(1))) are located at slightly different positions (e.g., the difference along the vertical direction is less than one row). For example, referring to Figure 17A, the offset along the vertical direction in the current picture is 0.5 rows. The central sample (e.g., (1701(1))) and the reconstructed sample to be filtered (e.g., (1703(1))) are located at the same location.

[0200] In some embodiments, for each of a plurality of filter shapes, the top sample of the first reconstructed sample is located within N rows (or lines) of the reconstructed sample to be filtered. N can be any suitable positive integer, such as in the range of 1 to 1024. For example, to reduce the hardware implementation cost of a sample offset filter (e.g., CCSO, LSO), N can be constrained to a relatively small integer. In some examples, when N is constrained to a relatively small integer, only neighboring samples within the constrained number of rows or lines from the center sample are used. In one example, N is 3 or less.

[0201] In one example, the number of first reconstruction samples in each of the plurality of filter shapes is less than five.

[0202] In one example, each of the plurality of filter shapes has the same number M of first reconstructed samples of the first components, where M is a positive integer. A filter or filter shape having the number M of first reconstructed samples can be referred to as an M-tap filter or a filter with M taps.

[0203] In one example, N is 1. In one example, when N is 1, the number of first reconstruction samples in each of the plurality of filter shapes is 3.

[0204] In one embodiment, each of the plurality of filter shapes includes a center sample co-located with the reconstructed sample (to be filtered) of the current component. The first reconstructed sample includes the center sample. The top sample of the first reconstructed sample is located within N rows (or lines) from the center sample. As mentioned above, N can be 3 or less. In one example, N is 1.

[0205] Subsequently, an output value of the sample-offset filter can be determined based on the first reconstructed sample in the filter shape. Furthermore, a filtered sample value of the reconstructed sample of the current component can be determined based on the reconstructed sample and the output value of the sample-offset filter, as shown in equation (21).

[0206] In one embodiment, each of the plurality of filter shapes includes a central sample co-located with the reconstructed sample of the current component, and the number of first reconstructed samples in each of the plurality of filter shapes is three. The remaining first reconstructed samples (different from the central sample) in each of the plurality of filter shapes can be arranged symmetrically or asymmetrically with respect to the central sample. Furthermore, in one example, the first reconstructed sample in each of the plurality of filter shapes includes a central sample and two samples symmetrical with respect to the central sample. In one example, N is 1.

[0207] In one example, the plurality of filter shapes (or filters) used in a sample offset filter (e.g., CCSO or LSO) includes one or more of the following seven different 3-tap filter shapes, as shown in FIG. 24. FIG. 24 illustrates an example of the plurality of filter shapes (or filters) used in a sample offset filter (e.g., CCSO or LSO) according to one embodiment of the present disclosure. The plurality of filter shapes include filter shapes (or filters) 1 through 7. Each of the plurality of filter shapes can include three first reconstructed samples, including a central sample (denoted as c) and two samples symmetrical about the central sample. N is 1. For example, for filter shapes 1, 2, 4, 6, and 7, the top sample of the first reconstructed samples (e.g., each of samples (2401) through (2405)) is located within one row (e.g., one row above) of the reconstructed sample to be filtered (co-located with central sample c). For example, for filter shapes 3 and 5, the top sample of the first reconstructed sample is located within one row (eg, on the same row) of the reconstructed sample to be filtered (co-located with center sample c).

[0208] More specifically, referring to FIG. 24 , the position of a central sample c in a first component of a current picture is (i, j), where i and j are integers. The plurality of filter shapes includes seven filter shapes 1 to 7. The positions of two samples in a first filter shape among the seven filter shapes are (i-1, j) and (i+1, j). The positions of two samples in a second filter shape among the seven filter shapes are (i-1, j-1) and (i+1, j+1). The positions of two samples in a third filter shape among the seven filter shapes are (i, j-1) and (i, j+1). The positions of two samples in a fourth filter shape among the seven filter shapes are (i+1, j-1) and (i-1, j+1). The positions of two samples in a fifth filter shape among the seven filter shapes are (i, j-4) and (i, j+4). The two sample positions in a sixth of the seven filter shapes are (i-1, j-7) and (i+1, j+7). The two sample positions in a seventh of the seven filter shapes are (i+1, j-7) and (i-1, j+7). A subset of the filter shapes of Figure 24 can be used in other embodiments. In one example, the filter shapes (or filters) used in a sample offset filter (e.g., CCSO or LSO) include one or more of filter shapes 1 through 7 of Figure 24.

[0209] In one example, the plurality of filter shapes (or filters) used in a sample offset filter (e.g., CCSO or LSO) includes one or more of the following six different 3-tap filter shapes, as shown in FIG. 25. FIG. 25 illustrates an example of the plurality of filter shapes (or filters) used in a sample offset filter (e.g., CCSO or LSO) according to one embodiment of the present disclosure. The plurality of filter shapes includes filter shapes (or filters) 1 through 6. Each of the plurality of filter shapes can include three first reconstructed samples, including a central sample (denoted as c) and two samples symmetrical about the central sample. N is 1. For example, for filter shapes 1, 2, and 4, the top sample of the first reconstructed samples (e.g., each of samples (2501) through (2503)) is located within one row (e.g., one row above) of the reconstructed sample to be filtered (co-located with central sample c). For example, for filter shapes 3, 5, and 6, the top sample of the first reconstructed sample is located within one row (eg, on the same row) of the reconstructed sample to be filtered (co-located with center sample c).

[0210] More specifically, referring to FIG. 25, the position of a central sample c in a first component of a current picture is (i, j), where i and j are integers. The plurality of filter shapes includes seven filter shapes 1 to 6. The positions of two samples in a first filter shape among the six filter shapes are (i-1, j) and (i+1, j). The positions of two samples in a second filter shape among the six filter shapes are (i-1, j-1) and (i+1, j+1). The positions of two samples in a third filter shape among the six filter shapes are (i, j-1) and (i, j+1). The positions of two samples in a fourth filter shape among the six filter shapes are (i+1, j-1) and (i-1, j+1). The positions of two samples in a fifth filter shape among the six filter shapes are (i, j-4) and (i, j+4). The two sample positions in the sixth of the six filter shapes are (i, j-7) and (i, j+7). A subset of these filter shapes may be used in other embodiments. In one example, the filter shapes (or filters) used in the sample offset filter (e.g., CCSO or LSO) include one or more of filter shapes 1-6 of FIG. 25.

[0211] In one example, the plurality of filter shapes (or filters) used in a sample offset filter (e.g., CCSO or LSO) includes one or more of the following seven different 3-tap filter shapes, as shown in FIG. 26. FIG. 26 illustrates an example of the plurality of filter shapes (or filters) used in a sample offset filter (e.g., CCSO or LSO) according to one embodiment of the present disclosure. The plurality of filter shapes includes filter shapes (or filters) 1 through 7. Each of the plurality of filter shapes can include three first reconstructed samples, including a central sample (denoted as c) and two samples symmetrical about the central sample. N is 1. For example, for filter shapes 1, 2, and 4 through 7, the top sample of the first reconstructed samples (e.g., each of samples (2601) through (2606)) is located within one row (e.g., one row above) of the reconstructed sample to be filtered (co-located with central sample c). For example, for filter shape 3, the top sample of the first reconstructed sample is located within one row (eg, on the same row) of the reconstructed sample to be filtered (co-located with center sample c).

[0212] More specifically, referring to FIG. 26, the position of a central sample c in a first component of a current picture is (i, j), where i and j are integers. The plurality of filter shapes includes seven filter shapes 1 to 7. The positions of two samples in a first filter shape among the seven filter shapes are (i-1, j) and (i+1, j). The positions of two samples in a second filter shape among the seven filter shapes are (i-1, j-1) and (i+1, j+1). The positions of two samples in a third filter shape among the seven filter shapes are (i, j-1) and (i, j+1). The positions of two samples in a fourth filter shape among the seven filter shapes are (i+1, j-1) and (i-1, j+1). The positions of two samples in a fifth filter shape among the seven filter shapes are (i-1, j-3) and (i+1, j+3). The two sample positions in a sixth of the seven filter shapes are (i+1, j-5) and (i-1, j+5). The two sample positions in a seventh of the seven filter shapes are (i-1, j-7) and (i+1, j+7). A subset of these filter shapes may be used in other embodiments. In one example, the filter shapes (or filters) used in a sample offset filter (e.g., CCSO or LSO) include one or more of filter shapes 1 through 7 of FIG. 26.

[0213] In one example, the plurality of filter shapes (or filters) used in a sample offset filter (e.g., CCSO or LSO) includes one or more of the following seven different 3-tap filter shapes, as shown in FIG. 27. FIG. 27 illustrates an example of the plurality of filter shapes (or filters) used in a sample offset filter (e.g., CCSO or LSO) according to one embodiment of the present disclosure. The plurality of filter shapes includes filter shapes (or filters) 1 through 7. Each of the plurality of filter shapes can include three first reconstructed samples, including a central sample (denoted as c) and two samples symmetrical about the central sample. N is 1. For example, for filter shapes 1, 2, and 4 through 7, the top sample of the first reconstructed samples (e.g., each of samples (2701) through (2706)) is located within one row (e.g., one row above) of the reconstructed sample to be filtered (co-located with central sample c). For example, for filter shape 3, the top sample of the first reconstructed sample is located within one row (eg, on the same row) of the reconstructed sample to be filtered (co-located with center sample c).

[0214] More specifically, referring to FIG. 27, the position of a central sample c in a first component of a current picture is (i, j), where i and j are integers. The plurality of filter shapes includes seven filter shapes 1 to 7. The positions of two samples in a first filter shape among the seven filter shapes are (i-1, j) and (i+1, j). The positions of two samples in a second filter shape among the seven filter shapes are (i-1, j-1) and (i+1, j+1). The positions of two samples in a third filter shape among the seven filter shapes are (i, j-1) and (i, j+1). The positions of two samples in a fourth filter shape among the seven filter shapes are (i+1, j-1) and (i-1, j+1). The positions of two samples in a fifth filter shape among the seven filter shapes are (i+1, j-3) and (i-1, j+3). The two sample locations of the sixth of the seven filter shapes are (i-1, j-5) and (i+1, j+5). The two sample locations of the seven filter shapes are (i+1, j-7) and (i-1, j+7). A subset of these filter shapes may be used in other embodiments. In one example, the filter shapes (or filters) used in a sample offset filter (e.g., CCSO or LSO) include one or more of filter shapes 1 through 7 of FIG. 27.

[0215] In one example, the filter shapes (or filters) used in a sample offset filter (e.g., CCSO or LSO) include one or more of the following seven different 3-tap filter shapes (filter shapes or filters 1-7), as shown in FIG. 28. FIG. 28 illustrates an example of the filter shapes (or filters) used in a sample offset filter (e.g., CCSO or LSO) according to one embodiment of the present disclosure. Each of filter shapes 1-7 can include three first reconstructed samples, including a center sample (denoted as c) and two samples symmetrical about the center sample. N is 1. For example, for filter shapes 1, 2, and 4, the top sample of the first reconstructed samples (e.g., each of samples (2801)-(2803)) is located within one row (e.g., one row above) of the reconstructed sample to be filtered (co-located with center sample c). For example, for filter shapes 3 and 5-7, the top sample of the first reconstructed samples is located within one row (e.g., in the same row) of the reconstructed sample to be filtered (co-located with center sample c).

[0216] More specifically, referring to Figure 28, the position of the center sample c in the first component of the current picture is (i, j), where i and j are integers. The positions of two samples in a first filter shape of the seven filter shapes are (i-1, j) and (i+1, j). The positions of two samples in a second filter shape of the seven filter shapes are (i-1, j-1) and (i+1, j+1). The positions of two samples in a third filter shape of the seven filter shapes are (i, j-1) and (i, j+1). The positions of two samples in a fourth filter shape of the seven filter shapes are (i+1, j-1) and (i-1, j+1). The positions of two samples in a fifth filter shape of the seven filter shapes are (i, j-3) and (i, j+3). The two sample positions in a sixth of the seven filter shapes are (i,j-5) and (i,j+5). The two sample positions in a seventh of the seven filter shapes are (i,j-7) and (i,j+7). In one example, the filter shapes (or filters) used in a sample offset filter (e.g., CCSO or LSO) include one or more of filter shapes 1-7 of FIG. 28. According to one embodiment, the filter shapes include filters 1-6 of FIG. 28.

[0217] The filters or filter shapes used in sample offset filters (e.g., CCSO and LSO) may not be symmetric. In one example, the maximum number of top row(s) (e.g., row(s) above the central sample of the first component co-located with the reconstructed sample to be filtered) used as input to a sample offset filter (e.g., CCSO or LSO) is different from the maximum number of bottom row(s) (e.g., row(s) below the central sample co-located with the reconstructed sample to be filtered). FIG. 29 shows example filter shapes (or filters) 1-4 and 6-8 according to one embodiment of the present disclosure. Each of filter shapes (or filters) 1-4 and 6-8 is a three-tap filter including a central sample c and two other samples of the first component. Filter shapes 1-4 and 6 are symmetric, with the other two samples symmetrically positioned relative to central sample c. Filter shapes 7 and 8 are asymmetric, with the other two samples not symmetrically positioned relative to central sample c. In one example, when an asymmetric filter (e.g., filter 7 or filter 8 in FIG. 29) is used in a sample offset filter (e.g., CCSO or LSO), the maximum number of rows (or lines) of the top sample is limited to N relative to the current sample to be filtered (e.g., the current reconstructed sample in the current component). In one example, the current sample is co-located with the center sample c of the first component. As mentioned above, N can be any suitable positive integer. Example values ​​of N can include, but are not limited to, 1 to 1024. In one example, N is 3 or less. In one example, N is 1.

[0218] According to some aspects of the present disclosure, the filter shape used in a filter (e.g., the position(s) of a filter tap within the filter) or a sample offset filter (e.g., CCSO or LSO) may depend on the relative position of the reconstructed sample (also referred to as the current sample) to be filtered within the FU (e.g., a 128x128 chroma block).

[0219] In one embodiment, coded information about a reconstructed sample of a current component in a filtering unit can be decoded from a coded video bitstream. The coded information can indicate a sample offset filter to be applied to the reconstructed sample to be filtered. The filter shape of the sample offset filter can include a first reconstructed sample of a first component in the current picture. The FU is within the current picture. As described above, the filter shape can be determined based on the position (or relative position) of the reconstructed sample in the FU. The reconstructed sample can be co-located with the center sample of the filter shape. Subsequently, an output value of the sample offset filter can be determined based on the first reconstructed sample in the filter shape. Then, a filtered sample value of the reconstructed sample of the current component can be determined based on the reconstructed sample and the output value of the sample offset filter.

[0220] In one embodiment, the current component is within a filtering unit within the current picture. The filter shape of the sample offset filter can be determined based on the position of the reconstructed sample within the filtering unit.

[0221] In one embodiment, when applying a sample offset filter (e.g., CCSO or LSO), if the sample offset filter has one or more filter taps that reach beyond the top K rows relative to the current sample to be filtered (or the center sample co-located with the current sample), the sample offset filter is not allowed. In one example, the sample offset filter is replaced with another sample offset filter that has filter taps within the top K rows relative to the current sample to be filtered. K can be a positive integer. In one example, K is equal to N.

[0222] In one embodiment, when applying a sample offset filter (e.g., CCSO or LSO), if the sample offset filter has one or more filter taps that reach beyond the top L rows relative to a current FU (or CTU) to be filtered, the one or more filter taps may be moved to the nearest sample in the same column but located within the top L rows relative to the current FU (or CTU) to be filtered or otherwise padded. The top L rows may be relative to samples in the current FU (or CTU). In one example, the top L rows are relative to the top boundary of the current FU (or CTU). For example, if the sample offset filter has one or more filter taps that reach beyond the top L rows relative to the top boundary of the current FU (or CTU) to be filtered or otherwise padded, the one or more filter taps may be moved to the nearest sample in the same column but located within the top L rows relative to the top boundary of the current FU (or CTU). L may be an integer. In one example, L is equal to N.

[0223] FIG. 30 illustrates an example in which one or more filter taps (e.g., filter taps 7 and 8) are moved, according to one embodiment of the present disclosure. FIG. 30 illustrates seven filters (or filter shapes) 1-4 and 6-8. Filter taps 1-4 and 6 are located within the top four rows of the current FU (or CTU), such as at the top boundary of the current FU, and two filter taps 7-8 (denoted by (3001)-(3002)) are located beyond the top four rows from the current FU (or CTU). Filter taps 7-8 are then moved to new positions, denoted by (3003)-(3004). The new positions of filter taps 7-8 can be located within the top four rows of the current FU (or CTU), such as within the top one row of the current FU (or CTU).

[0224] Example filters, such as those shown in Figures 20, 22, and 24-30, include a center sample c that is co-located with the reconstructed sample (or current sample) to be filtered. In general, sample offset filters can either include or exclude the center sample.

[0225] FIG. 31 shows a flowchart outlining a process (3100) according to one embodiment of the present disclosure. The process (3100) may be used to filter sample(s) in a current picture, such as sample(s) in an FU of the current picture. In various embodiments, the process (3100) is performed by a processing circuit, such as processing circuitry in the terminal devices (310), (330), and (340), a processing circuit that performs the functions of a video encoder (403), a processing circuit that performs the functions of a video decoder (410), a processing circuit that performs the functions of a video decoder (510), a processing circuit that performs the functions of a video encoder (603), or the like. In some embodiments, the process (3100) is implemented with software instructions, such that the processing circuit performs the process (3100) when it executes the software instructions. The process starts at (S3101) and proceeds to (S3110).

[0226] At (S3110), coded information about reconstructed samples of a current component in a current picture may be decoded from the coded video bitstream. The coded information may indicate a sample offset filter to be applied to the reconstructed samples. The filter shape of the sample offset filter may be one of a plurality of filter shapes. Each of the plurality of filter shapes may include a first reconstructed sample of a first component in the current picture. In one embodiment, for each of the plurality of filter shapes, a top sample of the first reconstructed sample may be located within N rows (or lines) of a central reconstructed sample (also referred to as a central sample) in the first component that is co-located with the reconstructed sample of the current component. N may be 3 or less.

[0227] In one example, the number of first reconstruction samples in each of the plurality of filter shapes is less than five.

[0228] In one example, each of the plurality of filter shapes includes a central reconstructed sample co-located with the reconstructed sample of the current component, and the number of first reconstructed samples in each of the plurality of filter shapes is 3. The first reconstructed samples in each of the plurality of filter shapes may include a central reconstructed sample and two samples symmetrical about the central reconstructed sample.

[0229] In one example, N is 1.

[0230] In one example, the position of a central reconstructed sample in a first component of a current picture is (i, j), where i and j are integers, and the plurality of filter shapes includes six filter shapes. The positions of two samples in a first filter shape of the six filter shapes are (i-1, j) and (i+1, j). The positions of two samples in a second filter shape of the six filter shapes are (i-1, j-1) and (i+1, j+1). The positions of two samples in a third filter shape of the six filter shapes are (i, j-1) and (i, j+1). The positions of two samples in a fourth filter shape of the six filter shapes are (i+1, j-1) and (i-1, j+1). The positions of two samples in a fifth filter shape of the six filter shapes are (i, j-4) and (i, j+4). The two sample locations in the sixth of the six filter shapes are (i,j-7) and (i,j+7).

[0231] In one example, the position of the central reconstructed sample in the first component of the current picture is (i, j), where i and j are integers, and the plurality of filter shapes includes six different filter shapes. The positions of two samples in the first filter shape are (i-1, j) and (i+1, j). The positions of two samples in the second filter shape are (i-1, j-1) and (i+1, j+1). The positions of two samples in the third filter shape are (i, j-1) and (i, j+1). The positions of two samples in the fourth filter shape are (i+1, j-1) and (i-1, j+1). The positions of two samples in the fifth filter shape are (i, j-3) and (i, j+3). The positions of two samples in the sixth filter shape are (i, j-5) and (i, j+5).

[0232] The sample offset filter can be a CCSO filter or an LSO filter.

[0233] In one example, the first component is the second component. In one example, the first component is different from the second component. The first component can be a luma component and the current component can be a chroma component.

[0234] In one example, the sample offset filter is an LSO filter, the first component is the current component, and the reconstructed sample within the current component is the central reconstructed sample.

[0235] At (S3120), an output value of the sample offset filter may be determined based on the first reconstructed sample in the filter shape.

[0236] At (S3130), filtered sample values ​​of the reconstructed samples of the current component can be determined based on the reconstructed samples and the output values ​​of the sample offset filter. In various examples, the sample offset filter is an in-loop filter whose output values ​​are applied to the reconstructed samples as offsets to remove coding artifacts while preserving details of the current component in the current picture. The sample offset filter can include one of an LSO, a CCSO, and an adaptive sample offset filter (ASO). The process (3100) proceeds to (S3199) and ends.

[0237] The process (3100) may be adapted as appropriate. Step(s) of the process (3100) may be modified and / or omitted. Additional step(s) may be added. Any suitable order of implementation may be used. In one example, the current component is in a filtering unit in the current picture. The filter shape of the sample offset filter may be determined based on the position of the reconstructed sample in the filtering unit. The sample offset filter is a CCSO filter, and the filtering unit is a chroma block having 128x128 chroma samples.

[0238] FIG. 32 shows a flowchart outlining the process (3200) according to one embodiment of the present disclosure. The process (3200) may be used to filter sample(s) in a current picture, such as sample(s) in an FU of the current picture. In various embodiments, the process (3200) is performed by a processing circuit, such as processing circuitry in the terminal devices (310), (330), and (340), a processing circuit that performs the functions of a video encoder (403), a processing circuit that performs the functions of a video decoder (410), a processing circuit that performs the functions of a video decoder (510), a processing circuit that performs the functions of a video encoder (603), etc. In some embodiments, the process (3200) is implemented by software instructions, such that the processing circuit performs the process (3200) when it executes the software instructions. The process starts at (S3201) and proceeds to (S3210).

[0239] At (S3210), coded information about reconstructed samples of a current component in the filtering unit can be decoded from the coded video bitstream. The coded information can indicate a sample offset filter to be applied to the reconstructed samples. A filter shape of the sample offset filter can include a first reconstructed sample of a first component in the current picture, and the filtering unit is in the current picture.

[0240] In one example, the filtering unit is a chroma block having 128x128 chroma samples.

[0241] In one example, the sample offset filter is a cross-component sample offset (CCSO) filter.

[0242] In one example, the sample offset filter is a local sample offset (LSO) filter.

[0243] At (S3220), the filter shape (e.g., the position(s) of each(s) filter tap(s)) can be determined based on the position of the reconstructed sample within the filtering unit, as described with reference to FIG. 30.

[0244] At (S3230), an output value of the sample offset filter can be determined based on the first reconstructed sample in the filter shape.

[0245] At (S3240), a filtered sample value of the reconstructed sample of the current component can be determined based on the reconstructed sample and the output value of the sample offset filter. The process (3200) proceeds to (S3299) and ends.

[0246] The process (3200) may be adapted as appropriate. Step(s) of the process (3200) may be modified and / or omitted. Additional step(s) may be added. Any suitable order of performance may be used.

[0247] The embodiments of the present disclosure may be used separately or combined in any order. Furthermore, each of the methods (or embodiments), encoders, and decoders may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium. The embodiments of the present disclosure may be applied to luma blocks or chroma blocks.

[0248] The techniques described above can be implemented as computer software physically stored on one or more computer-readable media using computer-readable instructions. For example, Figure 33 illustrates a computer system (3300) suitable for implementing certain embodiments of the disclosed subject matter.

[0249] Computer software can be coded using any suitable machine code or computer language that can be subjected to mechanisms such as assembly, compilation, linking, etc. to create code containing instructions that can be executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc. directly, or via interpretation, microcode execution, etc.

[0250] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, and the like.

[0251] The components illustrated in Figure 33 for the computer system (3300) are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure. The arrangement of components should not be interpreted as having a dependency or requirement regarding any one or combination of components illustrated in the exemplary embodiment of the computer system (3300).

[0252] The computer system (3300) may include certain human interface input devices that may respond to input by one or more human users, for example, via tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), and olfactory input (not shown). The human interface devices may also be used to capture certain media not necessarily directly associated with conscious human input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from still image cameras), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0253] The input human interface devices may include one or more (only one of each shown) of a keyboard (3301), a mouse (3302), a trackpad (3303), a touchscreen (3310), a data glove (not shown), a joystick (3305), a microphone (3306), a scanner (3307), and a camera (3308).

[0254] The computer system (3300) may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touchscreen (3310), data gloves (not shown), or joystick (3305), although haptic feedback devices that do not function as input devices may also be possible), audio output devices (e.g., speakers (3309), headphones (not shown)), visual output devices (e.g., screens (3310), including CRT screens, LCD screens, plasma screens, and OLED screens, each with or without touchscreen input capability and each with or without haptic feedback capability, some of which may be capable of two-dimensional visual output or four or more dimensions of output via means such as stereoscopic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0255] The computer system (3300) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (3320) with media such as CD / DVD (3321), thumb drives (3322), removable hard drives or solid state drives (3323), legacy magnetic media such as tape or floppy disks (not shown), dedicated ROM / ASIC / PLD-based devices (not shown) such as security dongles, etc.

[0256] Those skilled in the art will also understand that the term "computer-readable medium" as used in connection with the subject matter of this disclosure does not encompass transmission media, carrier waves, or other transitory signals.

[0257] The computer system (3300) may also include an interface (3354) to one or more communication networks (3355). The network may be, for example, wireless, wired, or optical. The network may further be local, wide-area, metropolitan, vehicular, and industrial, real-time, delay-tolerant, and the like. Examples of networks include local area networks such as Ethernet and wireless LAN; cellular networks including GSM, 3G, 4G, 5G, LTE, and the like; television wired or wireless wide-area digital networks including cable television, satellite television, and terrestrial broadcast television; and vehicular and industrial networks including CANbus. Certain networks generally require an external network interface adapter attached to a particular general-purpose data port or peripheral bus (3349) (e.g., a USB port on the computer system (3300)). Other networks are generally integrated into the core of the computer system (3300) by attachment to a system bus, as described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (3300) can communicate with other entities. Such communications may be one-way receive only (e.g., television broadcast), one-way transmit only (e.g., CANbus to a particular CANbus device), or bidirectional, for example, to other computer systems using local or wide area digital networks. Specific protocols and protocol stacks may be used with each of the networks and network interfaces described above.

[0258] The aforementioned human interface devices, human-accessible storage devices, and network interfaces may be attached to the core (3340) of the computer system (3300).

[0259] The core (3340) may include one or more central processing units (CPUs) (3341), graphics processing units (GPUs) (3342), dedicated programmable processing devices in the form of field programmable gate arrays (FPGAs) (3343), hardware accelerators for specific tasks (3344), graphics adapters (3350), etc. These devices may be connected via a system bus (3348), along with read-only memory (ROM) (3345), random access memory (3346), and internal mass storage (3347) such as a non-user-accessible hard drive or SSD. In some computer systems, the system bus (3348) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be attached directly to the core's system bus (3348) or via a peripheral bus (3349). In one example, a screen (3310) may be connected to the graphics adapter (3350). Peripheral bus architectures include PCI, USB, etc.

[0260] The CPU (3341), GPU (3342), FPGA (3343), and accelerator (3344) can execute specific instructions that, in combination, can constitute the aforementioned computer code. The computer code can be stored in ROM (3345) or RAM (3346). Transient data can also be stored in RAM (3346), while persistent data can be stored, for example, in internal mass storage (3347). Rapid storage and retrieval from any of the memory devices can be enabled through the use of cache memory, which can be closely associated with one or more of the CPU (3341), GPU (3342), mass storage (3347), ROM (3345), RAM (3346), etc.

[0261] The computer-readable medium may bear computer code for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0262] By way of example and not limitation, the computer system (3300) having the architecture, and in particular the core (3340), may provide functionality as a result of processor(s) (including CPUs, GPUs, FPGAs, accelerators, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media may be user-accessible mass storage, as described above, as well as media associated with specific storage of the core (3340) that is non-transitory in nature, such as the core's internal mass storage (3347) or ROM (3345). Software implementing various embodiments of the present disclosure may be stored in such devices and executed by the core (3340). The computer-readable media may include one or more memory devices or chips, depending on particular needs. The software may cause the core (3340), and in particular the processors (including CPUs, GPUs, FPGAs, etc.) therein, to perform particular processes or particular portions of particular processes described herein, including defining data structures stored in RAM (3346) and modifying such data structures according to software-defined processes. Additionally, or alternatively, a computer system may provide functionality as a result of hardwired or otherwise embodied logic in circuitry (e.g., accelerator (3344)) that can operate in place of or in conjunction with software to perform particular processes or portions of particular processes described herein. When referring to software, it may include logic, and vice versa, where appropriate. When referring to computer-readable media, it may include, where appropriate, circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry that embodies logic for execution, or both. The present disclosure encompasses any appropriate combination of hardware and software.

[0263] Appendix A: Acronyms JEM: Joint Exploration Model VVC: Versatile Video Coding BMS: Benchmark Set MV: Motion Vector HEVC: High Efficiency Video Coding SEI: Supplemental Enhancement Information VUI: Video Usability Information GOP: Group of Pictures TU: Conversion unit PU: Prediction Unit CTU: Coding Tree Unit CTB: coding tree block PB: Predicted Block HRD: Hypothetical Reference Decoder SNR: Signal-to-Noise Ratio CPU: Central Processing Unit GPU: Graphics Processing Unit CRT: cathode ray tube LCD: Liquid crystal display OLED: Organic Light Emitting Diode CD: Compact Disc DVD: Digital Video Disc ROM: Read-only memory RAM: Random Access Memory ASIC: Application Specific Integrated Circuit PLD: Programmable Logic Device LAN: Local Area Network GSM: Global System for Mobile Communications LTE: Long Term Evolution CANBus: Controller Area Network Bus USB: Universal Serial Bus PCI: Peripheral Component Interconnect FPGA: Field Programmable Gate Area SSD: Solid State Drive IC: Integrated Circuit HDR: High Dynamic Range SDR: Standard Dynamic Range JVET: Joint Video Exploration Team MPM: Most Probable Mode WAIP: Wide-angle Intra Prediction CU: Coding Unit PU: Prediction Unit TU: Conversion unit CTU: Coding Tree Unit PDPC: Position-dependent prediction combination ISP: Intra-subpartition SPS: Sequence parameter settings PPS: Picture Parameter Set APS: Adaptive Parameter Set VPS: Video Parameter Set DPS: Decoding Parameter Set ALF: Adaptive Loop Filter SAO: Sample Adaptive Offset CC-ALF: Cross-component adaptive loop filter CDEF: Constrained Directional Enhancement Filter CCSO: Cross-component sample offset LSO: Local Sample Offset LR: Loop Recovery Filter AV1:AOMedia Video 1 AV2:AOMedia Video 2

[0264] While this disclosure has described several exemplary embodiments, there are modifications, substitutions, and various substitute equivalents that fall within the scope of this disclosure. Thus, it will be appreciated that those skilled in the art can devise numerous systems and methods that, while not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within the spirit and scope of the present disclosure. [Explanation of symbols]

[0265] 101 Samples 102 Arrow 103 Arrow 104 Square Blocks 180 Schematic 201 Current Block 202 Surrounding Samples 203 Surrounding Samples 204 Surrounding Samples 205 Surrounding Samples 206 Surrounding Samples 300 Communication Systems 310 Terminal Equipment 320 Terminal Equipment 330 Terminal Equipment 340 Terminal Equipment 350 Network 400 Communication Systems 401 Video Source 402 Video Picture Stream 403 Video Encoder 404 encoded video data 405 Streaming Server 406 Client Subsystem 407 Copy of encoded video data 408 Client Subsystem 409 Copy of encoded video data 410 Video Decoder 411 Video Picture Output Stream 412 Display 413 Ingestion Subsystem 420 Electronic equipment 430 Electronic equipment 501 Channel 510 Video Decoder 512 Rendering Device 515 buffer memory 520 Entropy Decoder / Parser 521 Symbol 530 Electronic equipment 531 Receiver 551 Scaler / Descaler Unit 552 Intra-picture prediction unit 553 Motion Compensation Prediction Unit 555 Aggregator 556 Loop Filter Unit 557 Reference Picture Memory 558 Current Picture Buffer 601 Video Sources 603 Video Encoder 620 Electronic equipment 630 Source Coder 632 Coding Engine 633 (local) decoder 634 Reference Picture Memory 635 Predictor 640 Transmitter 643 coded video sequence 645 Entropy Coder 650 Controller 660 Communication Channels 703 Video Encoder 721 General-purpose controller 722 Intra Encoder 723 Residual Calculator 724 Residual Encoder 725 Entropy Encoder 726 Switch 728 Residual Decoder 730 Interencoder 810 Video Decoder 871 Entropy Decoder 872 Intra Decoder 873 Residual Decoder 874 Reconstruction Module 880 Interdecoder 910 ALF 911 ALF 920 elements 921 elements 922 elements 923 elements 924 elements 925 elements 926 elements 927 elements 928 elements 929 elements 930 elements 931 elements 932 elements 941 elements 942 elements 943 elements 944 elements 945 elements 946 elements 947 elements 948 elements 949 elements 950 elements 951 elements 952 elements 953 elements 954 elements 955 elements 956 elements 957 elements 958 elements 959 elements 960 elements 961 elements 962 elements 963 elements 964 elements C0 value C1 value C2 value C3 value C4 value C5 value C6 value C7 value C8 value C9 value C10 value C11 value C12 value H horizontal direction V vertical direction D1 Diagonal direction D2 Diagonal 1110 4x4 Block 1120 Horizontal CTU Boundary 1130 Virtual Boundary 1111 4x4 blocks 1121 CTU boundary 1131 Virtual Boundary 1210 Virtual Boundary 1220 Virtual Boundary 1230 Virtual Boundary 1240 Virtual Boundary 1250 Virtual Boundary 1260 Virtual Boundary 1510 Sample Adaptive Offset (SAO) Filter 1512 SAO filter 1514 SAO filter 1516 ALF Luma Filter 1518 ALF Chroma Filter 1521 Cross-Component Filter 1522 Adder 1531 Cross-Component Filter 1532 Adder 1541 SAO filtered luma component 1542 Second intermediate component 1543 Fourth intermediate component 1552 First intermediate component 1553 Third intermediate component 1561 Filtered Luma CB 1562 filtered first chroma component 1563 filtered second chroma component 1600 filters 1610 filter coefficients 1620 Diamond shape 1701 Luma Sample 1701(1) Upper left luma sample 1701(2) Upper right luma sample 1701(3) Lower left luma sample 1701(4) Lower right luma sample 1702 Luma Sample 1703 Chroma Sample 1703(1) Chroma Sample 1704 Chroma Sample 1705 Chroma Sample 1706 Chroma Sample 1707 Chroma Sample 1708 Chroma Sample Line 1711 Line 1712 Line 1713 Line 1714 Line 1715 1716 lines Line 1717 Line 1718 Line 1751 1752 lines 1753 lines 1754 lines 1810 8x8 blocks 1820 one set of directions 1823 45 degree direction 1830 Fully directional block 1840 lines p0 sample p1 sample p2 sample p3 sample 2310 1D 0 degree pattern 2320 1D 90 degree pattern 2330 1-dimensional 135 degree pattern 2340 1-dimensional 45 degree pattern 2401 Top Sample 2402 Top Sample 2403 Top Sample 2404 Top Sample 2405 Top Sample 2501 Top Sample 2502 Top Sample 2503 Top Sample 2601 Top Sample 2602 Top Sample 2603 Top Sample 2604 Top Sample 2605 Top Sample 2606 Top Sample 2701 Top Sample 2702 Top Sample 2703 Top Sample 2704 Top Sample 2705 ​​Top Sample 2706 Top Sample 2801 Top Sample 2802 Top Sample 2803 Top Sample 3001 Filter tap position 3002 Filter tap position 3003 Filter tap position 3004 Filter tap position 3100 Process 3200 processes 3300 Computer Systems 3301 Keyboard 3302 Mouse 3303 Trackpad 3305 Joystick 3306 Microphone 3307 Scanner 3308 Camera 3309 Speaker 3310 Touchscreen 3320 CD / DVD ROM / RW 3321 CD / DVD and other media 3322 thumb drive 3323 Removable Hard Drive or Solid State Drive 3340 cores 3341 Central Processing Unit (CPU) 3342 Graphics Processing Unit (GPU) 3343 Field Programmable Gate Area (FPGA) 3344 Hardware accelerators for specific tasks 3345 Read-Only Memory (ROM) 3346 Random Access Memory 3347 core internal mass storage 3348 System Bus 3349 General Purpose Data Port or Peripheral Bus 3350 graphics adapter 3354 Interface 3355 Communication Networks

Claims

1. 1. A method for video decoding in a decoder, comprising: decoding coded information about a reconstructed sample of a current component in a current picture from a coded video bitstream, the coded information indicating a sample offset filter to be applied to the reconstructed sample, the filter shape of the sample offset filter being one of a plurality of filter shapes, each of the plurality of filter shapes including a first reconstructed sample of a first component in the current picture; determining an output value of the sample offset filter based on the first reconstructed sample within the filter shape; determining a filtered sample value of the reconstructed sample of the current component based on the reconstructed sample and the output value of the sample offset filter; the sample offset filter is an in-loop filter whose output values ​​are applied to the reconstructed samples as offsets to remove coding artifacts while preserving details of the current component in the current picture; the number of first reconstruction samples in each of the plurality of filter shapes is three; each of the plurality of filter shapes includes a central reconstructed sample co-located with the reconstructed sample of the current component; at least one of the plurality of filter shapes includes a reconstructed sample adjacent to and in the same row as the central reconstructed sample; 10. The method of claim 9, wherein at least one of the plurality of filter shapes includes a reconstructed sample located in the same row as the central reconstructed sample that is spaced apart from the reconstructed sample adjacent to the central reconstructed sample and located in the same row as the central reconstructed sample.

2. 2. The method of claim 1, wherein for each of the plurality of filter shapes, a top sample of the first reconstructed sample is located within N rows of a central reconstructed sample in the first component that is co-located with the reconstructed sample of the current component, where N is less than or equal to 3.

3. The method described in claim 1, wherein if the top sample of the first reconstructed sample is located a predetermined number of rows away from the current filter unit or coding tree unit, the top sample is moved within four rows above the current filter unit or coding tree unit.

4. The method of claim 1 , wherein the first reconstructed sample in each of the plurality of filter shapes includes the central reconstructed sample and two samples symmetrical about the central reconstructed sample.

5. 5. The method of claim 1, wherein the sample offset filter comprises one of a local sample offset filter (LSO), a cross-component sample offset filter (CCSO), and an adaptive sample offset filter (ASO).

6. 3. The method of claim 2, wherein N is 1.

7. The method of claim 6 , wherein each of the plurality of filter shapes includes a central reconstructed sample co-located with the reconstructed sample of the current component.

8. N is 1, the position of the central reconstructed sample within the first component of the current picture is (i, j), where i and j are integers; the plurality of filter shapes includes six filter shapes; the locations of the two samples in a first of the six filter shapes are (i-1, j) and (i+1, j); the locations of the two samples in a second one of the six filter shapes are (i-1, j-1) and (i+1, j+1); the positions of the two samples in a third filter shape of the six filter shapes are (i, j-1) and (i, j+1); the locations of the two samples in a fourth of the six filter shapes are (i+1, j-1) and (i-1, j+1); the locations of the two samples in a fifth filter shape of the six filter shapes are (i, j-4) and (i, j+4); the positions of the two samples in a sixth filter shape of the six filter shapes are (i, j-7) and (i, j+7); The method of claim 4.

9. N is 1, the position of the central reconstructed sample within the first component of the current picture is (i, j), where i and j are integers; the plurality of filter shapes includes six filter shapes; the locations of the two samples in a first of the six filter shapes are (i-1, j) and (i+1, j); the locations of the two samples in a second one of the six filter shapes are (i-1, j-1) and (i+1, j+1); the positions of the two samples in a third filter shape of the six filter shapes are (i, j-1) and (i, j+1); the locations of the two samples in a fourth of the six filter shapes are (i+1, j-1) and (i-1, j+1); the locations of the two samples in a fifth of the six filter shapes are (i, j-3) and (i, j+3); the positions of the two samples in a sixth filter shape of the six filter shapes are (i, j-5) and (i, j+5); The method of claim 4.

10. The method of claim 5 , wherein the sample offset filter is the CCSO, the first component is a luma component, and the current component is a chroma component.

11. The method of claim 5 , wherein the sample offset filter is the LSO, the first component is the current component, and the reconstructed sample is a center reconstructed sample.

12. the current component is in a filtering unit in the current picture; The method includes determining the filter shape of the sample offset filter based on a position of the reconstructed samples within the filtering unit.

12. The method according to any one of claims 1 to 11.

13. The method of claim 12 , wherein the sample offset filter is a cross-component sample offset (CCSO) filter, and the filtering unit is a chroma block having 128×128 chroma samples.

14. Apparatus configured to carry out the method of any one of claims 1 to 13.

15. A program for causing a processor to carry out the method according to any one of claims 1 to 13.

Images