Method and apparatus for video coding

Abstract

Aspects of this disclosure provide methods and apparatus for video decoding. [Solution] The device's processing circuit decodes coded information from the coded video bitstream about the reconstructed sample of the current component in the current picture. The coded information indicates a sample offset filter to be applied to the reconstructed sample. The filter shape of the sample offset filter is one of a plurality of filter shapes. Each of the plurality of filter shapes contains a first reconstructed sample of a first component in the current picture. The filtered sample value of the reconstructed sample is determined based on the first reconstructed sample in the filter shape. The sample offset filter is an in-loop filter whose output value is applied to the reconstructed sample as an offset to remove coding artifacts while preserving the details of the current component in the current picture.

Description

[Technical Field]

[0001] Reference This application claims priority to U.S. Patent Application No. 17 / 495,318, “Method and apparatus for video coding,” filed on October 6, 2021, with priority to U.S. Provisional Application No. 63 / 162,956, “LOW-COMPLEXITY FILTER DESIGN FOR SAMPLE OFFSET,” filed on 18 March 2021. The entire disclosure of the prior application is incorporated herein by reference.

[0002] This disclosure generally describes embodiments related to video coding. [Background technology]

[0003] The background art description provided herein is intended to provide a general context for this disclosure. The inventors' research, to the extent that it is described in this background art section, is not explicitly or implicitly considered prior art to this disclosure, along with any other description that may not otherwise be considered prior art at the time of filing.

[0004] Video coding and decoding can be performed using interpicture prediction with motion compensation. Uncompressed digital video may contain a series of pictures, each picture having spatial dimensions of, for example, 1920×1080 luminance samples and associated chrominance samples. The series of pictures may have a fixed or variable picture rate (informally also known as frame rate), for example, 60 pictures per second or 60 Hz. Uncompressed video has specific bitrate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (1920×1080 luminance sample resolution at a frame rate of 60 Hz) requires a bandwidth of nearly 1.5 Gbit / s. One hour of such video requires more than 600 GByte of storage space.

[0005] One purpose of video coding and decoding may be to reduce the redundancy of the input video signal through compression. Compression can help reduce the aforementioned bandwidth and / or storage space requirements by more than two orders of magnitude, in some cases. Both lossless and lossy compression, and combinations thereof, can be used. Lossless compression refers to a technique that allows an exact copy of the original signal to be reconstructed from the compressed original signal. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signals is small enough to make the reconstructed signal useful for its intended purpose. For video, lossy compression is widely adopted. The amount of acceptable distortion depends on the application; for example, users of certain consumer streaming applications may tolerate higher distortion than users of television distribution applications. The achievable compression ratio can reflect that a higher tolerance for distortion leads to a higher compression ratio.

[0006] Video encoders and video decoders can utilize techniques from several broad categories, including, for example, motion compensation, transformation, quantization, and entropy coding.

[0007] Video codec techniques may include a technique known as intra coding. In intra coding, sample values ​​are represented without referencing samples or other data from a previously reconstructed reference picture. In some video codecs, the picture is spatially subdivided into blocks of samples. If all blocks of samples are coded in intra mode, the picture may be an intra picture. Intra pictures and their derived pictures, such as independent decoder refresh pictures, can be used to reset the decoder state and therefore can be used as the first picture in a coded video bitstream and video session, or as a still image. Samples in intra blocks can be transformed, and the transformation coefficients can be quantized before entropy coding. Intra prediction may be a technique to minimize the sample values ​​in the pre-transformation region. In some cases, smaller post-transformation DC values ​​and smaller AC coefficients result in fewer bits being required at a given quantization step size to represent the block after entropy coding.

[0008] For example, traditional intra-coding, such as that known from MPEG-2 generation coding techniques, does not use intra-prediction. However, some newer video compression techniques include, for example, techniques that try from surrounding sample data and / or metadata acquired during the encoding and / or decoding of data blocks that are spatially adjacent and preceding in the decoding order. Such techniques will henceforth be referred to as “intra-prediction” techniques. It should be noted that in at least some cases, intra-prediction uses reference data only from the current picture being reconstructed, and not from reference pictures.

[0009] Intra-prediction can take many different forms. If two or more of these techniques can be used in a given video coding technique, the techniques used can be coded in intra-prediction mode. In certain cases, a mode may have sub-modes and / or parameters, which can be coded individually or included in the mode's codeword. The choice of codeword for a given combination of mode, sub-mode, and / or parameters can affect the efficiency of coding via intra-prediction, and therefore the entropy coding technique used to convert the codeword into a bitstream can also have an impact.

[0010] Certain modes of intra-prediction were introduced in H.264, improved in H.265, and further refined with newer coding techniques such as Joint Search Models (JEM), Versatile Video Coding (VVC), and Benchmark Sets (BMS). Predictor blocks can be formed using neighboring sample values ​​belonging to already available samples. The sample values ​​of neighboring samples are copied to the predictor block according to direction. References to the direction used can be coded within the bitstream or predicted themselves.

[0011] Referring to Figure 1A, shown in the lower right is a subset of nine predictor directions known from the 33 possible predictor directions of H.265 (corresponding to 33 angular modes out of 35 intra-modes). The point where the arrows converge (101) represents the predicted sample. The arrows represent the direction in which the sample is predicted. For example, arrow (102) indicates that sample (101) is predicted to be to the upper right from one or more samples at an angle of 45 degrees from the horizontal. Similarly, arrow (103) indicates that sample (101) is predicted to be to the lower left from one or more samples at an angle of 22.5 degrees from the horizontal.

[0012] Referring further to FIG. 1A, in the upper left, a square block (104) of 4×4 samples (shown by thick dashed lines) is shown. The square block (104) contains 16 samples, each labeled with an "S", its position in the Y dimension (e.g., row number), and its position in the X dimension (e.g., column number). For example, sample S21 is the second sample from the top in the Y dimension and the first sample from the left in the X dimension. Similarly, sample S44 is the fourth sample within block (104) in both the Y and X dimensions. Since the block size is 4×4 samples, S44 is in the lower right. Further reference samples are shown following a similar numbering scheme. The reference samples are labeled with an "R", its Y position (e.g., row number) and X position (column number) relative to block (104). In both H.264 and H.265, the predicted samples are adjacent to the block being reconstructed, and thus negative values need not be used.

[0013] Intra-picture prediction can function by copying the reference sample values from adjacent samples as assigned by the signaled prediction direction. For example, the coded video bitstream includes signaling indicating a prediction direction that matches arrow (102) for this block, i.e., it is assumed that the samples are predicted from one or more prediction samples at a 45-degree angle from horizontal going up and to the right. In that case, samples S41, S32, S23, S14 are predicted from the same reference sample R05. Then, sample S44 is predicted from reference sample R08.

[0014] In certain cases, especially when the direction is not evenly divisible by 45 degrees, multiple reference sample values may be combined, e.g., by interpolation, to calculate the reference sample.

[0015] The number of possible directions has increased as video coding techniques have advanced. H.264 (2003) could represent nine different directions. This increased to 33 in H.265 (2013), and JEM / VVC / BMS can support up to 65 directions at the time of disclosure. Experiments have been conducted to identify the most likely directions, and using certain entropy coding techniques, a certain penalty may be accepted for less likely directions, allowing those more likely directions to be represented by a small number of bits. Furthermore, the direction itself can sometimes be predicted from adjacent directions used in adjacent, already decoded blocks.

[0016] Figure 1B shows a schematic diagram (180) illustrating the 65 intra-prediction directions by JEM to illustrate the increasing number of prediction directions over time.

[0017] The mapping of intra-predicted direction bits in a coded video bitstream representing direction can vary depending on the video coding technique, ranging from a simple direct mapping of the predicted direction to an intra-predicted mode, to a complex adaptive scheme involving codewords, the most likely mode, and similar techniques. However, in all cases, there may be certain directions that are statistically less likely to occur in video content than other particular directions. Since the purpose of video compression is to reduce redundancy, in a well-functioning video coding technique, those less likely directions will be represented with more bits than the more likely directions.

[0018] Motion compensation can be a lossy compression technique that involves a block of sample data from a previously reconstructed picture or a portion of it (a reference picture) being spatially shifted in the direction indicated by a motion vector (MV), and then used to predict the newly reconstructed picture or portion of the picture. In some cases, the reference picture may be the same as the picture currently being reconstructed. The MV can have two dimensions, X and Y, or three dimensions, where the third dimension indicates the reference picture used (the third dimension may indirectly be the time dimension).

[0019] Some video compression techniques allow predicting motion vectors (MVs) applicable to a specific area of ​​sample data from other MVs, for example, from MVs related to another area of ​​sample data that is spatially adjacent to the area being reconstructed and precedes that MV in the decoding order. This substantially reduces the amount of data required to code the MVs, thereby eliminating redundancy and increasing compression. MV prediction can work effectively because, for example, when coding an input video signal derived from a camera (known as natural video), areas larger than the area to which a single MV is applicable have a statistical likelihood of moving in a similar direction, and therefore, in some cases, can be predicted using similar motion vectors derived from the MVs of adjacent areas. This results in the found MV for a given area being similar to or identical to the MV predicted from the surrounding MVs, and this can be represented with fewer bits than would be used when directly coding the MV after entropy coding. In some cases, MV prediction can be an example of lossless compression of the signal (i.e., MV) derived from the original signal (i.e., the sample stream). In some cases, MV prediction itself may be lossy, for example, due to rounding errors when calculating the predictor from some surrounding MVs.

[0020] H.265 / HEVC (ITU-T Rec.H.265, “High Efficiency Video Coding”, December 2016) describes various MV prediction mechanisms. Of the many MV prediction mechanisms provided by H.265, the one described herein is the technique hereafter referred to as “spatial merging.”

[0021] Referring to Figure 2, the current block (201) contains a sample that, during the motion search process, the encoder detected as predictable from a previous block of the same size but spatially shifted. Instead of directly coding its MV, the MV can be derived from metadata associated with one or more reference pictures, for example, from the last reference picture (in decoding order), using an MV associated with one of the five surrounding samples represented by A0, A1, and B0, B1, B2 (202 through 206, respectively). In H.265, the MV prediction can use predictors from the same reference pictures used by adjacent blocks. [Overview of the project] [Means for solving the problem]

[0022] Aspects of this disclosure provide methods and apparatus for video coding and video decoding. In some examples, the apparatus for video decoding includes a processing circuit. The processing circuit can decode coded information from a coded video bitstream about a reconstructed sample of the current component in the current picture. The coded information indicates a sample offset filter to be applied to the reconstructed sample. The filter shape of the sample offset filter can be one of a plurality of filter shapes. Each of the plurality of filter shapes may include a first reconstructed sample of a first component in the current picture. The processing circuit can determine the output value of the sample offset filter based on the first reconstructed sample in the filter shape. Furthermore, the processing circuit can determine the filtered sample value of the reconstructed sample of the current component based on the reconstructed sample and the output value of the sample offset filter. The sample offset filter is an in-loop filter whose output value is applied to the reconstructed sample as an offset to remove coding artifacts while preserving the details of the current component in the current picture.

[0023] For each of the multiple filter shapes, the topmost sample of the first reconstructed sample is located within N rows of the central reconstructed sample within the first component, which is located in the same position as the reconstructed sample of the current component. N can be 3 or less.

[0024] In one embodiment, the number of first reconstructed samples in each of the multiple filter shapes is less than 5.

[0025] In one embodiment, each of the multiple filter shapes includes a central reconstitution sample located at the same location as the reconstitution sample of the current component, and the number of first reconstitution samples in each of the multiple filter shapes is 3. In one example, the first reconstitution samples in each of the multiple filter shapes include a central reconstitution sample and two samples that are symmetrical with respect to the central reconstitution sample.

[0026] In one example, the sample offset filter may include one of the following: a local sample offset filter (LSO), a cross-component sample offset filter (CCSO), and an adaptive sample offset filter (ASO).

[0027] In one embodiment, N is 1. In one example, each of the multiple filter shapes includes a central reconstitution sample located at the same location as the reconstitution sample of the current component, and the number of first reconstitution samples in each of the multiple filter shapes is 3.

[0028] In one example, N is 1, and the position of the central reconstructed sample in the first component of the current picture is (i,j), where i and j can be integers. Multiple filter shapes include six filter shapes. In the first of the six filter shapes, the positions of the two samples are (i-1,j) and (i+1,j). In the second of the six filter shapes, the positions of the two samples are (i-1,j-1) and (i+1,j+1). In the third of the six filter shapes, the positions of the two samples are (i,j-1) and (i,j+1). In the fourth of the six filter shapes, the positions of the two samples are (i+1,j-1) and (i-1,j+1). In the fifth of the six filter shapes, the positions of the two samples are (i,j-4) and (i,j+4). In the sixth of the six filter shapes, the positions of the two samples are (i,j-7) and (i,j+7).

[0029] In one example, N is 1, and the position of the central reconstructed sample in the first component of the current picture is (i,j), where i and j can be integers. Multiple filter shapes include six different filter shapes. The positions of the two samples in the first filter shape are (i-1,j) and (i+1,j), the positions of the two samples in the second filter shape are (i-1,j-1) and (i+1,j+1), the positions of the two samples in the third filter shape are (i,j-1) and (i,j+1), the positions of the two samples in the fourth filter shape are (i+1,j-1) and (i-1,j+1), the positions of the two samples in the fifth filter shape are (i,j-3) and (i,j+3), and the positions of the two samples in the sixth filter shape are (i,j-5) and (i,j+5).

[0030] In one example, the sample offset filter is CCSO. For instance, the first component is the luma component, and the current component is the chromatic component.

[0031] In one example, the sample offset filter is LSO, the first component is the current component, and the reconstructed sample is the central reconstructed sample.

[0032] In some examples, the current component is located within the filtering unit in the current picture. The processing circuit can determine the filter shape of the sample offset filter based on the position of the reconstructed sample within the filtering unit.

[0033] In one example, the sample offset filter is a CCSO filter, and the filtering unit is a chromatic block with 128 × 128 chromatic samples.

[0034] Aspects of the present disclosure also provide non-temporary computer-readable media that, when executed by a computer for video decoding and / or video encoding, stores instructions causing the computer to perform a method for video decoding and / or video encoding.

[0035] Further features, properties, and various advantages of the subject matter of the disclosure will become clearer from the detailed description and accompanying drawings below. [Brief explanation of the drawing]

[0036] [Figure 1A] This is a schematic diagram of an exemplary subset of intra-predictive modes. [Figure 1B] This is an example diagram of the intra-prediction direction. [Figure 2] This is a schematic diagram of the current block and its surrounding spatial merge candidates in one example. [Figure 3] This is a schematic diagram of a simplified block diagram of a communication system (300) according to one embodiment. [Figure 4] This is a schematic diagram of a simplified block diagram of a communication system (400) according to one embodiment. [Figure 5] This is a schematic diagram of a simplified block diagram of a decoder according to one embodiment. [Figure 6] This is a schematic diagram of a simplified block diagram of an encoder according to one embodiment. [Figure 7] Block diagram of an encoder according to another embodiment. [Figure 8] Block diagram of a decoder according to another embodiment. [Figure 9] This figure shows an exemplary adaptive loop filter according to an embodiment of the disclosure. [Figure 10A] This figure shows an example of subsampled locations used to calculate the vertical gradient according to an embodiment of the present disclosure. [Figure 10B] This figure shows an example of subsampled locations used to calculate the horizontal gradient according to an embodiment of the present disclosure. [Figure 10C] This figure shows an example of subsampled positions used to calculate the diagonal gradient according to an embodiment of the present disclosure. [Figure 10D]This figure shows an example of subsampled positions used to calculate the diagonal gradient according to an embodiment of the present disclosure. [Figure 11A] This figure shows the modified block classification in the virtual boundary according to an embodiment of the present disclosure. [Figure 11B] This figure shows the modified block classification in the virtual boundary according to an embodiment of the present disclosure. [Figure 12A] This figure illustrates an exemplary adaptive loop filter according to an embodiment of the present disclosure, along with a padding operation at a virtual boundary. [Figure 12B] This figure illustrates an exemplary adaptive loop filter according to an embodiment of the present disclosure, along with a padding operation at a virtual boundary. [Figure 12C] This figure illustrates an exemplary adaptive loop filter according to an embodiment of the present disclosure, along with a padding operation at a virtual boundary. [Figure 12D] This figure illustrates an exemplary adaptive loop filter according to an embodiment of the present disclosure, along with a padding operation at a virtual boundary. [Figure 12E] This figure illustrates an exemplary adaptive loop filter according to an embodiment of the present disclosure, along with a padding operation at a virtual boundary. [Figure 12F] This figure illustrates an exemplary adaptive loop filter according to an embodiment of the present disclosure, along with a padding operation at a virtual boundary. [Figure 13] This figure shows an example of a picture quadtree partitioning with maximum coding units aligned, according to one embodiment of the present disclosure. [Figure 14] This figure shows a quadtree partitioning pattern corresponding to Figure 13, according to one embodiment of the present disclosure. [Figure 15] This figure shows a cross-component filter used to generate chromatic components according to one embodiment of the present disclosure. [Figure 16] This figure shows an example of a filter according to one embodiment of the present disclosure. [Figure 17A] This figure shows an exemplary position of the chromatic sample relative to the luma sample according to embodiments of the present disclosure. [Figure 17B]This figure shows an exemplary position of the chromatic sample relative to the luma sample according to embodiments of the present disclosure. [Figure 18] This figure shows an example of block orientation search according to one embodiment of the present disclosure. [Figure 19] This figure shows an example of partial space projection according to one embodiment of the present disclosure. [Figure 20] This figure shows an example of a filter support area in a cross-component sample offset (CCSO) filter according to one embodiment of the present disclosure. [Figure 21A] This figure shows an exemplary mapping used in a CCSO filter according to one embodiment of the present disclosure. [Figure 21B] This figure shows an exemplary mapping used in a CCSO filter according to one embodiment of the present disclosure. [Figure 21C] This figure shows an exemplary mapping used in a CCSO filter according to one embodiment of the present disclosure. [Figure 22] This figure shows an exemplary implementation of a CCSO filter according to one embodiment of the present disclosure. [Figure 23] This figure shows four patterns for pixel classification in edge offset according to one embodiment of the present disclosure. [Figure 24] This figure shows an exemplary filter shape used in a sample offset filter according to one embodiment of the present disclosure. [Figure 25] This figure shows an exemplary filter shape used in a sample offset filter according to one embodiment of the present disclosure. [Figure 26] This figure shows an exemplary filter shape used in a sample offset filter according to one embodiment of the present disclosure. [Figure 27] This figure shows an exemplary filter shape used in a sample offset filter according to one embodiment of the present disclosure. [Figure 28] This figure shows an exemplary filter shape used in a sample offset filter according to one embodiment of the present disclosure. [Figure 29]This figure shows an exemplary filter shape used in a sample offset filter according to one embodiment of the present disclosure. [Figure 30] This figure shows an example in which one or more filter taps are moved according to one embodiment of the present disclosure. [Figure 31] This is a flowchart outlining the process (3100) according to one embodiment of the present disclosure. [Figure 32] This is a flowchart outlining the process (3200) according to one embodiment of the present disclosure. [Figure 33] This is a schematic diagram of a computer system according to one embodiment. [Modes for carrying out the invention]

[0037] Figure 3 shows a simplified block diagram of a communication system (300) according to one embodiment of the present disclosure. The communication system (300) includes a plurality of terminal devices that can communicate with each other, for example, via a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) interconnected via the network (350). In the example of Figure 3, the first pair of terminal devices (310) and (320) perform one-way transmission of data. For example, terminal device (310) may code video data (e.g., a stream of video pictures captured by terminal device (310)) for transmission to the other terminal device (320) via the network (350). The coded video data may be transmitted in the form of one or more coded video bitstreams. Terminal device (320) may receive coded video data from the network (350), decode the coded video data to restore video pictures, and display video pictures according to the restored video data. One-way data transmission can be common in applications such as media serving.

[0038] In another example, the communication system (300) includes a second pair of terminal devices (330) and (340) that perform bidirectional transmission of coded video data, which may occur, for example, during a video conference. For bidirectional transmission of data, in one example, each terminal device of terminal devices (330) and (340) may code video data (e.g., a stream of video pictures captured by that terminal device) for transmission to the other terminal device of terminal devices (330) and (340) via the network (350). Each terminal device of terminal devices (330) and (340) may also receive coded video data transmitted by the other terminal device of terminal devices (330) and (340), decode the coded video data to restore video pictures, and display video pictures on an accessible display device according to the restored video data.

[0039] In the example in Figure 3, terminal devices (310), (320), (330), and (340) may be represented as a server, a personal computer, and a smartphone, but the principles of this disclosure are not limited thereto. Embodiments of this disclosure apply to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network (350) represents any number of networks that transmit coded video data between terminal devices (310), (320), (330), and (340), including, for example, wired (wired connection) and / or wireless communication networks. Communication network (350) may exchange data over circuit-switched channels and / or packet-switched channels. Typical networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For the purposes of this discussion, the architecture and topology of network (350) may not be important to the operation of this disclosure unless described below herein.

[0040] Figure 4 shows an example of the application of the subject matter of disclosure, illustrating the arrangement of a video encoder and video decoder in a streaming environment. The subject matter of disclosure can be equally applied to other video-enabled applications, including, for example, video conferencing, digital television, and storage of compressed video on digital media such as CDs, DVDs, and memory sticks.

[0041] The streaming system may include an acquisition subsystem (413) which may include, for example, a digital camera that creates a video source (401), for example, a stream (402) of uncompressed video pictures. In one example, the stream (402) of video pictures includes a sample captured by the digital camera. The stream (402) of video pictures is shown in thick lines to highlight its high data volume compared to encoded video data (404) (or encoded video bitstream) and can be processed by an electronic device (420) which includes a video encoder (403) coupled to the video source (401). The video encoder (403) may include hardware, software, or a combination thereof to enable or implement aspects of the subject of disclosure as described in more detail below. The encoded video data (404) (or encoded video bitstream (404)) is shown in thin lines to highlight its low data volume compared to the stream (402) of video pictures and can be stored in a streaming server (405) for future use. One or more streaming client subsystems, such as client subsystems (406) and (408) in Figure 4, can access a streaming server (405) to obtain copies (407) and (409) of the encoded video data (404). Client subsystem (406) may include, for example, a video decoder (410) in an electronic device (430). The video decoder (410) decodes the input copy (407) of the encoded video data and creates an output stream (411) of a video picture that can be rendered on a display (412) (e.g., a display screen) or other rendering device (not shown). Some streaming systems may encode the encoded video data (404), (407), and (409) (e.g., video bitstreams) according to specific video coding / compression standards. An example of such standards is ITU-T Recommendation H.265.For example, a video coding standard under development is informally known as Multipurpose Video Coding (VVC). The subject of this disclosure may be used in the context of VVC.

[0042] It should be noted that electronic devices (420) and (430) may include other components (not shown). For example, electronic device (420) may include a video decoder (not shown), and electronic device (430) may also include a video encoder (not shown).

[0043] Figure 5 shows a block diagram of a video decoder (510) according to one embodiment of the present disclosure. The video decoder (510) can be included in an electronic device (530). The electronic device (530) can include a receiver (531) (e.g., a receiving circuit). The video decoder (510) can be used in place of the video decoder (410) in the example of Figure 4.

[0044] The receiver (531) can receive one or more coded video sequences to be decoded by the video decoder (510), and in the same or different embodiments, one coded video sequence is decoded at a time, and the decoding of each coded video sequence is independent of other coded video sequences. The coded video sequences can be received from a channel (501), which may be a hardware / software link to a storage device that stores coded video data. The receiver (531) can receive the coded video data together with other data, such as coded audio data and / or auxiliary data streams, which may be transferred to their respective user entities (not shown). The receiver (531) can isolate the coded video sequences from other data. To counteract network jitter, a buffer memory (515) may be coupled between the receiver (531) and the entropy decoder / parser (520) (hereinafter referred to as "Parser (520)"). In certain applications, the buffer memory (515) is part of the video decoder (510). In other cases, it may be located outside the video decoder (510) (not shown). In yet other cases, there may be a buffer memory (not shown) outside the video decoder (510), for example to counter network jitter, and further, there may be another buffer memory (515) inside the video decoder (510), for example to handle playback timing. When the receiver (531) is receiving data from a storage / transfer device with sufficient bandwidth and controllability, or from an isosynchronous network, the buffer memory (515) may be unnecessary or can be made small. For use in best-effort packet networks such as the Internet, the buffer memory (515) may be required, can be relatively large, can be advantageously made adaptively sized, and can be at least partially implemented in an operating system or similar element (not shown) outside the video decoder (510).

[0045] The video decoder (510) may include a parser (520) for reconstructing symbols (521) from the coded video sequence. The categories of these symbols include information used to manage the operation of the video decoder (510) and information for controlling rendering devices, such as a rendering device (512) (e.g., a display screen) which is not an integral part of the electronic device (530) but can be coupled to the electronic device (530), as shown in Figure 5. The control information for (one or more) rendering devices may be in the form of supplemental extension information (SEI messages) or video usability information (VUI) parameter set fragments (not shown). The parser (520) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may conform to video coding techniques or standards and may follow various principles, including variable-length coding, Huffman coding, context-dependent or non-context-dependent arithmetic coding, etc. The parser(520) may extract from the coded video sequence a set of at least one subgroup parameters of a subgroup of pixels in the video decoder, based on at least one parameter corresponding to a group. Subgroups may include groups of pictures (GOP), pictures, tiles, slices, macroblocks, coding units (CU), blocks, transform units (TU), and predictive units (PU). The parser(520) may also extract information from the coded video sequence such as transform coefficients, quantization parameter values, and motion vectors.

[0046] The parser (520) may perform an entropy decoding / parse operation on the video sequence received from buffer memory (515) in order to create a symbol (521).

[0047] The reconstruction of symbol (521) may include multiple different units depending on the type of coded video picture or part thereof (interpicture and intrapicture, interblock and intrablock, etc.) and other factors. Which units are included and how can be controlled by subgroup control information parsed from the video sequence coded by parser (520). The flow of such subgroup control information between parser (520) and the following multiple units is not illustrated for clarity.

[0048] Beyond the functional blocks already described, the video decoder (510) can be conceptually subdivided into several functional units, as described below. In actual implementations operating under commercial constraints, many of these units interact closely with each other and can be integrated, at least partially. However, the following conceptual subdivision into functional units is appropriate for illustrating the subject of this disclosure.

[0049] The first unit is the scaler / inverse unit (551). The scaler / inverse unit (551) receives control information from the parser (520) as one or more symbols (521), including the quantization conversion coefficients, which conversion to use, block size, quantization coefficients, and quantization scaling matrix. The scaler / inverse unit (551) can output a block containing sample values ​​that can be input to the aggregator (555).

[0050] In some cases, the output samples of the scaler / inverse transform (551) may relate to intracoded blocks, i.e., blocks that do not use predictive information from previously reconstructed pictures but can use predictive information from previously reconstructed portions of the current picture. Such predictive information can be provided by the intrapicture predictive unit (552). In some cases, the intrapicture predictive unit (552) generates a block of the same size and shape as the block being reconstructed, using surrounding already reconstructed information fetched from the current picture buffer (558). The current picture buffer (558) buffers, for example, a partially reconstructed current picture and / or a fully reconstructed current picture. The aggregator (555) may, sample by sample, add the predictive information generated by the intrapredictive unit (552) to the output sample information provided by the scaler / inverse transform unit (551).

[0051] In other cases, the output samples of the scaler / inverse unit (551) may relate to an interconnected and potentially motion-compensated block. In such cases, the motion-compensated prediction unit (553) can access the reference picture memory (557) to fetch samples to be used for prediction. After motion-compensating the fetched samples according to the symbols (521) associated with the block, these samples can be added to the output of the scaler / inverse unit (551) by the aggregator (555) to generate output sample information (in this case, called residual samples or residual signals). The addresses in the reference picture memory (557) from which the motion-compensated prediction unit (553) fetches the predicted samples can be controlled by motion vectors available to the motion-compensated prediction unit (553), for example, in the form of symbols (521) which may have X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​fetched from the reference picture memory (557) when the exact motion vectors of the subsamples are used, motion vector prediction mechanisms, etc.

[0052] The output samples of the aggregator (555) can undergo various loop filtering techniques in the loop filter unit (556). The video compression technique is controlled by parameters contained in the coded video sequence (also known as the coded video bitstream) and may include in-loop filtering techniques available to the loop filter unit (556) as symbols (521) from the parser (520), but may also respond to metadata obtained during decoding of earlier parts (in decoding order) of the coded picture or coded video sequence, and may also respond to previously reconstructed and loop-filtered sample values.

[0053] The output of the loop filter unit (556) can be a sample stream that can be output to the rendering device (512) or stored in the reference picture memory (557) for use in future interpicture prediction.

[0054] A particular coded picture, once fully reconfigured, can be used as a reference picture for future predictions. For example, once the coded picture corresponding to the current picture is fully reconfigured and the coded picture is identified as a reference picture (e.g., by the parser (520)), the current picture buffer (558) can become part of the reference picture memory (557) and can be reallocated before starting the reconfiguration of the next coded picture.

[0055] The video decoder (510) may perform decoding operations according to a specified video compression technique of the standard, such as ITU-T Rec.H.265. The coded video sequence may conform to the syntax specified by the video compression technique or standard being used, in the sense that the coded video sequence is faithful to both the syntax of the video compression technique or standard and the profile documented in the video compression technique or standard. Specifically, a profile may select certain tools from all the tools available in the video compression technique or standard as tools to be used only under that profile. Compliance may also require that the complexity of the coded video sequence be within the range defined by the level of the video compression technique or standard. In some cases, the level may limit the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limitations set by the level may, in some cases, be further limited by the virtual reference decoder (HRD) specification and metadata for HRD buffer management signaled in the coded video sequence.

[0056] In one embodiment, the receiver (531) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of one or more encoded video sequences. The additional data may be used by the video decoder (510) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may take the form of, for example, time, space, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, error correction codes, etc.

[0057] Figure 6 shows a block diagram of a video encoder (603) according to one embodiment of the present disclosure. The video encoder (603) is included in an electronic device (620). The electronic device (620) includes a transmitter (640) (e.g., a transmitting circuit). The video encoder (603) can be used in place of the video encoder (403) in the example of Figure 4.

[0058] The video encoder (603) may receive video samples from a video source (601) (not part of the electronic device (620) in the example of Figure 6) which can capture (one or more) video images to be coded by the video encoder (603). In another example, the video source (601) is part of the electronic device (620).

[0059] The video source (601) may provide a source video sequence to be coded by the video encoder (603) in the form of a digital video sample stream, which can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source (601) may be a storage device that stores previously prepared video. In a video conferencing system, the video source (601) may be a camera that captures local image information as a video sequence. The video data may be provided as a series of separate pictures that give motion when viewed sequentially. The pictures themselves may be organized as a spatial array of pixels, each pixel may contain one or more samples depending on the sampling structure, color space, etc., used. Those skilled in the art will readily understand the relationship between pixels and samples. The following description focuses on samples.

[0060] According to one embodiment, the video encoder (603) can encode and compress pictures of a source video sequence into an encoded video sequence (643) in real time or under any other time constraints required by the application. One function of the controller (650) is to enforce an appropriate coding rate. In some embodiments, the controller (650) controls and is functionally coupled to other functional units as described below. For clarity, the couplings are not illustrated. Parameters set by the controller (650) may include rate control-related parameters (such as picture skips, quantizers, lambda values ​​for rate distortion optimization techniques), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (650) may be configured to have other appropriate functions related to the video encoder (603) optimized for a particular system design.

[0061] In some embodiments, the video encoder (603) is configured to operate in a coding loop. In an overly simplified explanation, in one example, the coding loop may include a source coder (630) (for example, responsible for creating symbols such as a symbol stream based on an input picture to be coded and one or more reference pictures) and a (local) decoder (633) built into the video encoder (603). The decoder (633) reconstructs the symbols to create sample data in a similar manner to create a (remote) decoder (since any compression between symbols and coded video bitstreams is reversible in the video compression techniques considered in the subject of the disclosure). The reconstructed sample stream (sample data) is input to a reference picture memory (634). Since decoding the symbol stream yields bit-exact results regardless of the decoder's location (local or remote), the contents of the reference picture memory (634) are also bit-exact between the local encoder and the remote encoder. In other words, the predictive part of the encoder "sees" the exact same sample values ​​as the reference picture samples that the decoder will "see" when using the predictions during decoding. This fundamental principle of reference picture synchronization (and the resulting drift if synchronization cannot be maintained due to, for example, channel errors) is also used in several related technologies.

[0062] The operation of the “local” decoder (633) may be the same as that of a “remote” decoder, such as the video decoder (510), as described in detail above in relation to Figure 5. Referring briefly to Figure 5, however, since symbols are available and the encoding / decoding of symbols to the coded video sequence by the entropy coder (645) and parser (520) may be reversible, the entropy decoding portion of the video decoder (510), including the buffer memory (515) and parser (520), may not be fully implemented in the local decoder (633).

[0063] At this point, it can be said that any decoder technology present within the decoder, excluding parse / entropy decoding, must also necessarily exist in substantially the same functional form in the corresponding encoder. For this reason, the subject of the disclosure focuses on decoder operation. The description of encoder technology can be omitted as it is the inverse of the comprehensively described decoder technology. More detailed explanations are necessary only in specific areas, which are shown below.

[0064] During operation, in some examples, the source coder (630) may perform motion-compensated predictive coding, predictively coding the input picture by referencing one or more previously coded pictures from a video sequence designated as “reference pictures”. In this way, the coding engine (632) codes the difference between the pixel blocks of the input picture and the pixel blocks of the reference pictures (one or more) that may be selected as predictive references to the input picture.

[0065] The local video decoder (633) can decode the coded video data of a picture that may be designated as a reference picture based on symbols created by the source coder (630). The operation of the coding engine (632) may, advantageously, be a lossy process. If the coded video data can be decoded by a video decoder (not shown in Figure 6), the reconstructed video sequence may typically be a copy of the source video sequence with some error. The local video decoder (633) can replicate the decoding process that may be performed by the video decoder on the reference picture and have the reconstructed reference picture stored in the reference picture cache (634). In this way, the video encoder (603) can locally store a copy of the reconstructed reference picture that has content common to the reconstructed reference picture obtained by the far-end video decoder (without transmission error).

[0066] The predictor (635) may perform a predictive search for the coding engine (632). That is, for a new picture to be coded, the predictor (635) may search the reference picture memory (634) for sample data (as candidate reference pixel blocks) or specific metadata such as reference picture motion vectors and block shapes that can serve as appropriate predictive references for the new picture. The predictor (635) may work sample block by pixel block to find appropriate predictive references. In some cases, the input picture may have predictive references drawn from multiple reference pictures stored in the reference picture memory (634), as determined by the search results obtained by the predictor (635).

[0067] The controller (650) may manage the coding operations of the source coder (630), including, for example, setting parameters and subgroup parameters used to encode video data.

[0068] The outputs of all the aforementioned functional units can be entropy-coded by the entropy coder (645). The entropy coder (645) converts the symbols generated by the various functional units into coded video sequences by lossless compression of the symbols according to techniques such as Huffman coding, variable-length coding, and arithmetic coding.

[0069] The transmitter (640) may buffer (one or more) coded video sequences created by the entropy coder (645) in preparation for transmission over a communication channel (660), which may be a hardware / software link to a storage device that will store the coded video data. The transmitter (640) may merge the coded video data from the video coder (603) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0070] The controller (650) may manage the operation of the video encoder (603). During coding, the controller (650) may assign a specific coded picture type to each coded picture, and the picture type may affect the coding techniques that may be applied to each picture. For example, a picture may often be assigned as one of the following picture types:

[0071] An intra-picture (I-picture) can be a picture that can be coded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow for different types of intra-pictures, including, for example, independent decoder refresh ("IDR") pictures. Those skilled in the art will recognize these variations of I-pictures and their respective uses and characteristics.

[0072] A predictive picture (P-picture) can be a picture that can be coded and decoded using intra-prediction or inter-prediction, which predicts the sample value of each block using at most one motion vector and reference index.

[0073] A bidirectional predictive picture (B-picture) can be a picture that can be coded and decoded using intra-prediction or inter-prediction, which predicts the sample values ​​of each block using up to two motion vectors and reference indices. Similarly, a multi-predictive picture can use three or more reference pictures and associated metadata for the reconstruction of a single block.

[0074] A source picture can generally be spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each), and each block can be coded. Blocks can be predictively coded by referencing other (already coded) blocks, as determined by the coding assignment applied to each picture in the block. For example, blocks of picture I can be coded non-predictively or predictively (spatial prediction or intra-predictive prediction) by referencing already coded blocks of the same picture. Pixel blocks of picture P can be predictedly coded by spatial prediction or temporal prediction by referencing one previously coded reference picture. Blocks of picture B can be predictedly coded by spatial prediction or via temporal prediction by referencing one or two previously coded reference pictures.

[0075] The video encoder (603) may perform coding operations in accordance with a given video coding technique or standard, such as ITU-T Rec.H.265. In this operation, the video encoder (603) may perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Thus, the coded video data may conform to the syntax specified by the video coding technique or standard being used.

[0076] In one embodiment, the transmitter (640) may transmit additional data along with the encoded video. The source coder (630) may include such data as part of the encoded video sequence. The additional data may include time / space / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, and the like.

[0077] Video can be captured chronologically as multiple source pictures (video pictures). Intra-picture prediction (often abbreviated as intra-prediction) utilizes spatial correlations within a given picture, while inter-picture prediction utilizes (temporal or other) correlations between pictures. In one example, a particular picture being encoded / decoded, called the current picture, is divided into blocks. If a block in the current picture is similar to a reference block in a previously coded, still-buffered reference picture in the video, the block in the current picture can be coded by a vector called a motion vector. The motion vector points to a reference block in the reference picture and may have a third dimension to identify the reference picture if multiple reference pictures are used.

[0078] In some embodiments, a dual-prediction technique can be used in interpicture prediction. According to the dual-prediction technique, two reference pictures are used, such as a first reference picture and a second reference picture, both of which are earlier in the video in the decoding order (but may be earlier and later in the display order, respectively). A block in the current picture can be coded by a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. The block can be predicted by a combination of the first and second reference blocks.

[0079] Furthermore, merge mode techniques can be used in interpicture prediction to improve coding efficiency.

[0080] According to some embodiments of this disclosure, predictions such as interpicture prediction and intrapicture prediction are performed in block units. For example, according to the HEVC standard, pictures in a sequence of video pictures are divided into coding tree units (CTUs) for compression, and the CTUs in a picture have the same size, such as 64x64 pixels, 32x32 pixels, or 16x16 pixels. Generally, a CTU contains three coding tree blocks (CTBs), which are one luma CTB and two chroma CTBs. Each CTU can be recursively quadtree-divided into one or more coding units (CUs). For example, a 64x64 pixel CTU can be divided into one 64x64 pixel CU, or four 32x32 pixel CUs, or sixteen 16x16 pixel CUs. In one example, each CU is analyzed to determine the prediction type of the CU, such as inter-prediction type or intra-prediction type. A CU is divided into one or more prediction units (PUs) depending on its temporal and / or spatial predictability. Generally, each PU contains one chroma prediction block (PB) and two chroma PBs. In one embodiment, the prediction operation in coding (encoding / decoding) is performed in units of prediction blocks. Using a chroma prediction block as an example of a prediction block, the prediction block contains a matrix of pixel values ​​(e.g., chroma values) such as 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 pixels, etc.

[0081] Figure 7 shows a diagram of a video encoder (703) according to another embodiment of the present disclosure. The video encoder (703) is configured to receive a processing block (e.g., a prediction block) of sample values ​​in the current video picture in a sequence of video pictures, and to encode the processing block into a coded picture which is part of a coded video sequence. In one example, the video encoder (703) is used instead of the video encoder (403) in the example of Figure 4.

[0082] In the HEVC example, the video encoder (703) receives a matrix of sample values ​​for a processing block, such as an 8x8 sample prediction block. The video encoder (703) determines whether the processing block is best coded using intra-mode, inter-mode, or bi-prediction mode, for example, rate-distortion optimization. If the processing block is coded in intra-mode, the video encoder (703) encodes the processing block into a coded picture using the intra-prediction technique; if the processing block is coded in inter-mode or bi-prediction mode, the video encoder (703) may encode the processing block into a coded picture using the inter-prediction technique or the bi-prediction technique, respectively. In certain video coding techniques, the merge mode may be an inter-picture prediction submode in which the motion vector is derived from one or more motion vector predictors without benefiting from the coded motion vector components outside the predictors. In certain other video coding techniques, there may be motion vector components applicable to the target block. In one example, the video encoder (703) includes other components, such as a mode determination module (not shown) for determining the mode of the processing block.

[0083] In the example shown in Figure 7, the video encoder (703) includes an interencoder (730), an intraencoder (722), a residual calculator (723), a switch (726), a residual encoder (724), a general-purpose controller (721), and an entropy encoder (725), all coupled together as shown in Figure 7.

[0084] The interencoder (730) is configured to receive a sample of the current block (e.g., a processing block), compare that block to one or more reference blocks in the reference picture (e.g., blocks in the previous and subsequent pictures), generate interprediction information (e.g., a description of redundant information, motion vectors, and merge mode information by the intercoding technique), and compute an interprediction result (e.g., a predicted block) based on the interprediction information using any appropriate technique. In some examples, the reference picture is a decoded reference picture that has been decoded based on the encoded video information.

[0085] The intra encoder (722) is configured to receive a sample of the current block (e.g., a processing block), and optionally compare the block to an already coded block in the same picture to generate transformed quantization coefficients, and optionally also generate intra prediction information (e.g., intra prediction direction information by one or more intra coding techniques). In one example, the intra encoder (722) also calculates an intra prediction result (e.g., a predicted block) based on the intra prediction information and a reference block in the same picture.

[0086] The general-purpose controller (721) is configured to determine general-purpose control data and control other components of the video encoder (703) based on the general-purpose control data. In one example, the general-purpose controller (721) determines the mode of a block and provides control signals to the switch (726) based on the mode. For example, if the mode is intra-mode, the general-purpose controller (721) controls the switch (726) to select intra-mode results for use by the residual calculator (723), and controls the entropy encoder (725) to select intra-prediction information and include that intra-prediction information in the bitstream. If the mode is inter-mode, the general-purpose controller (721) controls the switch (726) to select inter-prediction results for use by the residual calculator (723), and controls the entropy encoder (725) to select inter-prediction information and include that inter-prediction information in the bitstream.

[0087] The residual calculator (723) is configured to calculate the difference (residual data) between the received block and the prediction result selected from the intra-encoder (722) or inter-encoder (730). The residual encoder (724) is configured to operate on the residual data to encode the residual data and generate conversion coefficients. In one example, the residual encoder (724) is configured to convert the residual data from the spatial domain to the frequency domain and generate conversion coefficients. The conversion coefficients are then quantized to obtain quantized conversion coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data can be appropriately used by the intra-encoder (722) and inter-encoder (730). For example, an interencoder (730) can generate a decoded block based on decoded residual data and interprediction information, and an intraencoder (722) can generate a decoded block based on decoded residual data and intraprediction information. The decoded block is appropriately processed to generate a decoded picture, which in some examples can be buffered in a memory circuit (not shown) and used as a reference picture.

[0088] The entropy encoder (725) is configured to format the bitstream to include the encoded blocks. The entropy encoder (725) is configured to include various information in the bitstream according to an appropriate standard such as the HEVC standard. For example, the entropy encoder (725) is configured to include general control data, selected prediction information (e.g., intra-prediction information or inter-prediction information), residual information, and other appropriate information in the bitstream. Note that, according to the subject of the disclosure, residual information is not present when coding blocks in either inter-mode or bi-prediction mode merge submodes.

[0089] Figure 8 shows a diagram of a video decoder (810) according to another embodiment of the present disclosure. The video decoder (810) is configured to receive a coded picture, which is part of a coded video sequence, and to decode the coded picture to produce a reconstructed picture. In one example, the video decoder (810) is used instead of the video decoder (410) in the example of Figure 4.

[0090] In the example shown in Figure 8, the video decoder (810) includes an entropy decoder (871), an interdecoder (880), a residual decoder (873), a reconfiguration module (874), and an intradecoder (872) coupled together as shown in Figure 8.

[0091] The entropy decoder (871) can be configured to reconstruct specific symbols from the coded picture that represent the syntactic elements that make up the coded picture. Such symbols may include, for example, the mode in which the block is coded (e.g., intra-mode, inter-mode, bi-prediction mode, the latter two being merge sub-mode or another sub-mode), prediction information (e.g., intra-prediction information or inter-prediction information) that can identify specific samples or metadata used for prediction by the intra-decoder (872) or inter-decoder (880), respectively, and residual information in the form of quantization transformation coefficients. For example, if the prediction mode is inter-mode or bi-prediction mode, inter-prediction information is provided to the inter-decoder (880), and if the prediction type is intra-prediction type, intra-prediction information is provided to the intra-decoder (872). The residual information can undergo inverse quantization and be provided to the residual decoder (873).

[0092] The interdecoder (880) is configured to receive interprediction information and generate interprediction results based on the interprediction information.

[0093] The intra decoder (872) is configured to receive intra prediction information and generate prediction results based on the intra prediction information.

[0094] The residual decoder (873) is configured to perform inverse quantization to extract inverse quantization conversion coefficients, and to process these inverse quantization conversion coefficients to convert the residual from the frequency domain to the spatial domain. The residual decoder (873) may also require certain control information (to include quantization parameters (QP)), which may be provided by the entropy decoder (871) (the data path is not illustrated as this may only involve a small amount of control information).

[0095] The reconstruction module (874) is configured to combine the residuals as output from the residual decoder (873) and the prediction results (optionally as output from the inter-prediction module or intra-prediction module) in the spatial domain to form a reconstructed block which may be part of the reconstructed picture, and the reconstructed block may be part of the reconstructed video. Note that other appropriate operations, such as deblocking operations, may be performed to improve visual quality.

[0096] It should be noted that the video encoders (403), (603), and (703), as well as the video decoders (410), (510), and (810), can be implemented using any suitable technique. In one embodiment, the video encoders (403), (603), and (703), as well as the video decoders (410), (510), and (810), can be implemented using one or more integrated circuits. In another embodiment, the video encoders (403), (603), and (603), as well as the video decoders (410), (510), and (810), can be implemented using one or more processors that execute software instructions.

[0097] This disclosure includes a set of video coding techniques, such as a low-complexity filter or filter geometry for sample offsetting. For example, a low-complexity filter geometry for a sample offset filter is disclosed.

[0098] The video coding format can include any suitable video coding format, such as the Alliance for Open Media (AOMedia) Video 1 (AV1) or next-generation AOMedia Video formats beyond AV1, or other open video coding formats designed for video transmission over the Internet. Video coding standards may include High Efficiency Video Coding (HEVC) standards and next-generation video coding beyond HEVC (e.g., Versatile Video Coding (VVC)).

[0099] To reduce artifacts, the encoder / decoder can apply an adaptive loop filter (ALF) using block-based filter adaptation. For the luma component, for example, one of several filters (e.g., 25 filters) can be selected for a 4x4 luma block based on the direction and activity of the local gradient.

[0100] ALF can have any suitable shape and size. Referring to Figure 9, ALF(910) to ALF(911) have rhombus shapes, such as a 5x5 rhombus shape for ALF(910) and a 7x7 rhombus shape for ALF(911). In ALF(910), elements (920) to (932) can be used in the filtering process to form a rhombus shape. Seven values ​​(e.g., C0 to C6) can be used for elements (920) to (932). In ALF(911), elements (940) to (964) can be used in the filtering process to form a rhombus shape. Thirteen values ​​(e.g., C0 to C12) can be used for elements (940) to (964).

[0101] Referring to Figure 9, in some examples, two ALFs (910) and (911) with rhombic filter shapes are used. The 5×5 rhombic filter (910) can be applied to the chromatic component (e.g., chromatic block, chromatic CB), and the 7×7 rhombic filter (911) can be applied to the luma component (e.g., luma block, luma CB). Other suitable shapes (one or more) and sizes (one or more) can be used in the ALF. For example, a 9×9 rhombic filter can be used.

[0102] The filter coefficients at the positions indicated by the values ​​(e.g., C0-C6 in (910) or C0-C12 in (920)) may be non-zero. Furthermore, if the ALF includes a clipping function, the clipping values ​​at those positions may be non-zero.

[0103] For the block classification of Luma components, a 4x4 block (or Luma block, Luma CB) can be classified or categorized as one of several (e.g., 25) classes. Equation (1) is used to determine the quantized values ​​of the directional parameter D and the activity value A.

number

Math

Math

Math

Math

Math

Math

[0104] To reduce the complexity of the above-described block classification, subsampled one-dimensional Laplacian calculations can be applied. Figures 10A to 10D show examples of subsampled positions used to calculate the gradients g v , g h , g d1 , and g d2 in the vertical direction (Figure 10A), horizontal direction (Figure 10B), and two diagonal directions d1 (Figure 10C) and d2 (Figure 10D), respectively. The same subsampled positions can be used for gradient calculations in different directions. In Figure 10A, the label "V" represents the vertical gradient g vThis indicates the subsampled position for calculating the horizontal gradient g. In Figure 10B, the label "H" indicates the horizontal gradient g. h This indicates the subsampled positions for calculating the diagonal gradient g. In Figure 10C, the label "D1" indicates the diagonal gradient g. d1 This indicates the subsampled position for calculating the diagonal gradient g. In Figure 10D, the label "D2" indicates the diagonal gradient g2. d2 This indicates the subsampled positions for calculating [the value].

[0105] Horizontal gradient g v and vertical gradient g h Maximum value

number

number

number

number

number

number

number

number

number

number

number

[0106] Activity value A can be calculated as follows:

number

number

[0107] Block classification is not applied to the chroma components within a picture; therefore, a single set of ALF coefficients can be applied to each chroma component.

[0108] Geometric transformations can be applied to the filter coefficients and the corresponding filter clip values ​​(also called clip values). Before filtering a block (e.g., a 4x4 Luma block), for example, the gradient values ​​calculated for the block (e.g., g v , g h , g d1 and / or g d2Depending on the filter coefficients f(k,l) and the corresponding filter clipping values ​​c(k,l), geometric transformations such as rotation, diagonal or vertical inversion can be applied. The geometric transformation applied to the filter coefficients f(k,l) and the corresponding filter clipping values ​​c(k,l) may be equivalent to applying a geometric transformation to the samples within the region supported by the filter. Geometric transformations can make different blocks to which ALF is applied more similar by aligning their respective orientations.

[0109] Three geometric transformations, including diagonal inversion, vertical inversion, and rotation, can be performed as shown in equations (9) to (11), respectively. f D (k,l)=f(l,k), c D (k,l)=c(l,k) Equation (9) f V (k,l)=f(k,Kl-1), c V (k,l)=c(k,Kl-1) Equation (10) f R (k,l)=f(Kl-1,k), c R (k,l)=c(Kl-1,k) Equation (11) In the formula, K is the size of the ALF or filter, and 0 ≤ k, l ≤ K-1 are the coordinates of the coefficients. For example, the upper left corner of the filter f or the clip value matrix (or clip matrix) c is (0,0) and the lower right corner is (K-1, K-1). The transformation can be applied to the filter coefficients f(k,l) and clip values ​​c(k,l) depending on the gradient values ​​calculated for the block. An example of the relationship between the transformation and the four gradients is summarized in Table 1.

[0110] [Table 1]

[0111] In some embodiments, such as VVC, the ALF filter parameters are signaled by a picture adaptive parameter set (APS). The APS can signal one or more sets (e.g., up to 25 sets) of chroma filter coefficients and clip value indices. In one example, one of the sets may include chroma filter coefficients and one or more clip value indices. One or more sets (e.g., up to 8 sets) of chroma filter coefficients and clip value indices can also be signaled. To reduce signaling overhead, filter coefficients for different classifications of the chroma component (e.g., having different classification indices) can be merged. The slice header can signal the index of the APS used for the current slice. In examples such as VVC, ALF signaling is CTU-based.

[0112] In one embodiment, a clip value index (also called a clipping index) can be decoded from the APS. The clip value index can be used, for example, to determine a corresponding clip value based on the relationship between the clip value index and the corresponding clip value. This relationship can be predefined and stored in the decoder. In one example, this relationship is described by tables such as a table of clip value indices and corresponding clip values ​​for the luma component (e.g., used for luma CBs) and a table of clip value indices and corresponding clip values ​​for the chroma component (e.g., used for chroma CBs). The clip value may depend on the bit depth B, which can refer to the internal bit depth, the bit depth of the reconstructed sample in the CB to be filtered, etc. In some examples, a table of clip values ​​(e.g., luma and / or chroma) is obtained using equation (12). AlfClip={for n∈[0 .. N-1] round(2 B-α*n )} Formula (12) In the formula, AlfClip is the clipping value, B is the bit depth (e.g., bitDepth), N (e.g., N=4) is the number of allowed clipping values, and α is a predefined constant value. In one example, α is equal to 2.35. n is the clipping value index (also called the clipping index or clipIdx). Table 2 shows an example table obtained using formula (12) with N=4. The clipping index n can be 0, 1, 2, and 3 in Table 2. Table 2 can be used for luma blocks or chroma blocks.

[0113] [Table 2]

[0114] The slice header of the current slice can signal one or more APS indices (e.g., up to seven APS indices) to specify the Luma filter sets available for the current slice. The filtering process can be controlled at one or more appropriate levels, such as the picture level, slice level, or CTB level. In one embodiment, the filtering process can be further controlled at the CTB level. A flag can be signaled to indicate whether the ALF is applied to the Luma CTB. The Luma CTB can select a filter set from among multiple fixed filter sets (e.g., 16 fixed filter sets) and (one or more) signaled filter sets (also called one or more signaled filter sets) that are signaled by the APS. A filter set index can be signaled to the Luma CTB to indicate the filter set to be applied (e.g., a filter set from among multiple fixed filter sets and one or more signaled filter sets). Multiple fixed filter sets can be predefined and hardcoded in the encoder and decoder and can be called predefined filter sets.

[0115] For chroma components, the APS index can be signaled in the slice header to indicate the chroma filter set that should be used for the current slice. At the CTB level, if the APS has two or more chroma filter sets, a filter set index can be signaled for each chroma CTB.

[0116] The filter coefficients can be quantized to a norm equal to 128. To reduce the complexity of multiplication, bitstream fit can be applied so that the non-centralized coefficient values ​​are within the range of -27 to 27-1. In one example, the centralized coefficient is not signaled in the bitstream and can be considered equal to 128.

[0117] In some embodiments, the syntax and meaning of the clipping index and clip value are defined as follows: alf_luma_clip_idx[ sfIdx ][ j ] can be used to specify the clipping index of the clip value to be used before multiplying by the j-th coefficient of the signaled luma filter indicated by sfIdx. Bitstream compatibility requirements may include the requirement that the value of alf_luma_clip_idx[ sfIdx ][ j ], where sfIdx = 0 to alf_luma_num_filters_signalled_minus1 and j = 0 to 11, be in the range of 0 to 3.

[0118] The luma filter clip value AlfClipL[adaptation_parameter_set_id], which has the element AlfClipL[adaptation_parameter_set_id][filtIdx][j], where filtIdx=0~NumAlfFilters-1 and j=0~11, can be derived as specified in Table 2, depending on bitDepth set to equal BitDepthY and clipIdx set to equal alf_luma_clip_idx[alf_luma_coeff_delta_idx[filtIdx]][j].

[0119] The `alf_chroma_clip_idx[altIdx][j]` can be used to specify the clipping index of the clip value that should be used before multiplying by the j-th coefficient of the alternative chroma filter having index altIdx. Bitstream compatibility requirements may include the requirement that the value of `alf_chroma_clip_idx[altIdx][j]` be in the range of 0 to 3 for altIdx=0 to alf_chroma_num_alt_filters_minus1 and j=0 to 5.

[0120] A chroma filter clip value AlfClipC[adaptation_parameter_set_id][altIdx][j], having the element AlfClipC[adaptation_parameter_set_id][altIdx], where altIdx=0~alf_chroma_num_alt_filters_minus1 and j=0~5, can be derived as specified in Table 2, depending on bitDepth set to equal BitDepthC and clipIdx set to equal alf_chroma_clip_idx[altIdx][j].

[0121] In one embodiment, the filtering process can be described as follows. On the decoder side, when ALF is enabled for CTB, the sample R(i,j) in CU (or CB) can be filtered, and the filtered sample value R'(i,j) is obtained, as shown below using the formula (13). In one example, each sample in CU is filtered.

number

[0122] The selected clipping values ​​can be coded in the "alf_data" syntax element as follows: That is, the clipping index corresponding to the selected clipping values ​​shown in Table 2 can be coded using an appropriate coding scheme (e.g., Golomb coding scheme). The coding scheme can be the same one used to code the filter set index.

[0123] In one embodiment, a virtual boundary filtering process can be used to reduce the line buffer requirements of the ALF. Thus, modified block classification and filtering can be employed for samples near the CTU boundary (e.g., horizontal CTU boundary). The virtual boundary (1130) is defined as "N" of the horizontal CTU boundary (1120), as shown in Figure 11A. samples By shifting only the samples, it can be defined as a line, N samplescan be a positive integer. For example, N samples For the luma component, it is equal to 4, N samples For the chroma component, this is equal to 2.

[0124] Referring to Figure 11A, the modified block classification can be applied to the Luma component. For example, the one-dimensional Laplacian gradient calculation for a 4x4 block (1110) above the virtual boundary (1130) uses only the samples above the virtual boundary (1130). Similarly, referring to Figure 11B, the one-dimensional Laplacian gradient calculation for a 4x4 block (1111) below the virtual boundary (1131) shifted from the CTU boundary (1121) uses only the samples below the virtual boundary (1131). Thus, the quantization of the activity value A can be scaled by taking into account the reduction in the number of samples used in the one-dimensional Laplacian gradient calculation.

[0125] For the filtering process, symmetric padding operations at the virtual boundary can be used for both the luma and chroma components. Figures 12A to 12F show examples of such modified ALF filtering for the luma component at the virtual boundary. If the sample being filtered is located below the virtual boundary, adjacent samples located above the virtual boundary can be padded. If the sample being filtered is located above the virtual boundary, adjacent samples located below the virtual boundary can be padded. Referring to Figure 12A, adjacent sample C0 can be padded with sample C2 located below the virtual boundary (1210). Referring to Figure 12B, adjacent sample C0 can be padded with sample C2 located above the virtual boundary (1220). Referring to Figure 12C, adjacent samples C1 to C3 can be padded with samples C5 to C7, respectively, located below the virtual boundary (1230). Referring to Figure 12D, adjacent samples C1 to C3 can be padded with samples C5 to C7, respectively, located above the virtual boundary (1240). Referring to Figure 12E, adjacent samples C4 to C8 can be padded with samples C10, C11, C12, C11, and C10, respectively, which are located below the virtual boundary (1250). Referring to Figure 12F, adjacent samples C4 to C8 can be padded with samples C10, C11, C12, C11, and C10, respectively, which are located above the virtual boundary (1260).

[0126] In some examples, the above explanation can be appropriately applied when the sample and adjacent samples are located to the left (or right) and right (or left) of the virtual boundary.

[0127] A maximum coding unit (LCU) aligned picture quadtree partition can be used. To improve coding efficiency, a coding unit synchronized picture quadtree-based adaptive loop filter can be used for video coding. In one example, a Luma picture is divided into multiple multilevel quadtree partitions, with each partition boundary aligned to a maximum coding unit (LCU) boundary. Each partition may have a filtering process and can therefore be called a filter unit or filtering unit (FU).

[0128] The two-pass coding flow is described as follows: In the first pass, the quadtree partitioning pattern and the best filter (or optimal filter) for each FU can be determined. During the determination process, filtering distortion can be estimated by Fast Filtering Distortion Estimation (FFDE). The reconstructed picture can be filtered according to the determined quadtree partitioning pattern and selected filter for the FU (e.g., all FUs). In the second pass, the on / off control of the CU-synchronous ALF can be performed. According to the ALF on / off result, the initially filtered picture is partially reconstructed by the reconstructed picture.

[0129] A picture can be partitioned into multilevel quadtree partitions by employing a top-down partitioning strategy and using a rate-distortion criterion. Each partition can be called a FU. The partitioning process can align the quadtree partitions with LCU boundaries, as shown in Figure 13. Figure 13 shows an LCU-aligned picture quadtree partition according to one embodiment of the present disclosure. In one example, the coding order of the FUs follows the z-scan order. For example, referring to Figure 13, the picture is partitioned into 10 FUs (e.g., FU0 to FU9), and the coding order is from FU0 to FU9, e.g., FU0, FU1, FU2, FU3, FU4, FU5, FU6, FU7, FU8, and FU9.

[0130] To indicate a picture quadtree partitioning pattern, partitioning flags can be encoded and sent in z-scan order. Figure 14 shows a quadtree partitioning pattern corresponding to Figure 13 according to one embodiment of the present disclosure. As shown in the example in Figure 14, the quadtree partitioning flags are encoded in z-scan order.

[0131] The filter for each FU can be selected from two filter sets based on a rate distortion criterion. The first set may contain newly derived 1 / 2 symmetric square and rhombic filters for the current FU. The second set may come from a time-delay filter buffer. The time-delay filter buffer can store filters previously derived for FUs in previous pictures. The filter with the minimum rate distortion cost from the two filter sets can be selected for the current FU. Similarly, if the current FU is not the smallest FU and can be further divided into four child FUs, the rate distortion costs of those four child FUs can be calculated. By recursively comparing the rate distortion costs with and without division, the picture quadtree partitioning pattern can be determined.

[0132] In some examples, the maximum quadtree partitioning level is 2, and therefore the maximum number of FUs is 16. When deciding on a quadtree partition, the correlation values ​​used to derive the Wiener coefficients of the 16 FUs at the lowest quadtree level (minimum FUs) can be reused. For the remaining FUs, the Wiener filters for the remaining FUs can be derived from the correlations of the 16 FUs at the lowest quadtree level. Therefore, in one example, there is only one framebuffer access to derive the filter coefficients for all FUs.

[0133] After the quadtree partitioning pattern is determined, CU-synchronous ALF on / off control can be performed to further reduce filtering distortion. By comparing filtering distortion with unfiltering distortion, leaf CUs can explicitly switch the ALF on / off in their corresponding local regions. Coding efficiency can be further improved by redesigning the filter coefficients according to the ALF on / off result. In one example, the redesign process requires additional framebuffer access. Therefore, in some examples, such as coding unit-synchronous picture quadtree-based adaptive loop filter (CS-PQALF) encoder designs, redesign is not required after the CU-synchronous ALF on / off decision to minimize the number of framebuffer accesses.

[0134] The cross-component filtering process can apply cross-component filters, such as cross-component adaptive loop filters (CC-ALF). These cross-component filters can refine the chroma component (e.g., chroma CB corresponding to the chroma CB) using the luma sample values ​​of the luma component (e.g., luma CB). In one example, both the luma CB and chroma CB are contained within the CU.

[0135] Figure 15 shows a cross-component filter (e.g., CC-ALF) used to generate a chroma component according to one embodiment of the present disclosure. In some examples, Figure 15 shows filtering processes for a first chroma component (e.g., first chroma CB), a second chroma component (e.g., second chroma CB), and a luma component (e.g., luma CB). The luma component can be filtered by a sample adaptive offset (SAO) filter (1510) to produce an SAO-filtered luma component (1541). The SAO-filtered luma component (1541) can be further filtered by an ALF luma filter (1516) to become a filtered luma CB (1561) (e.g., "Y").

[0136] The first chroma component can be filtered by an SAO filter (1512) and an ALF chroma filter (1518) to generate a first intermediate component (1552). Furthermore, the SAO-filtered chroma component (1541) can be filtered by a cross-component filter (e.g., CC-ALF) (1521) for the first chroma component to generate a second intermediate component (1542). Subsequently, a filtered first chroma component (1562) (e.g., "Cb") can be generated based on at least one of the second intermediate component (1542) and the first intermediate component (1552). In one example, the filtered first chroma component (1562) (e.g., "Cb") can be generated by combining the second intermediate component (1542) and the first intermediate component (1552) using an adder (1522). The cross-component adaptive loop filtering process for the first chroma component may include steps performed by a CC-ALF (1521) and steps performed by, for example, an adder (1522).

[0137] The above description can be applied to the second chroma component. The second chroma component can be filtered by an SAO filter (1514) and an ALF chroma filter (1518) to generate a third intermediate component (1553). Furthermore, the SAO-filtered chroma component (1541) can be filtered by a cross-component filter (e.g., CC-ALF) (1531) for the second chroma component to generate a fourth intermediate component (1543). Subsequently, a filtered second chroma component (1563) (e.g., "Cr") can be generated based on at least one of the fourth intermediate component (1543) and the third intermediate component (1553). In one example, the filtered second chroma component (1563) (e.g., "Cr") can be generated by combining the fourth intermediate component (1543) and the third intermediate component (1553) using an adder (1532). In one example, the cross-component adaptive loop filtering process for the second chroma component may include steps performed by a CC-ALF (1531) and steps performed by, for example, an adder (1532).

[0138] Cross-component filters (e.g., CC-ALF(1521), CC-ALF(1531)) can operate by applying a linear filter with any suitable filter shape to the luma component (or luma channel) to refine each chroma component (e.g., first chroma component, second chroma component).

[0139] Figure 16 shows an example of a filter (1600) according to one embodiment of the present disclosure. The filter (1600) may include non-zero filter coefficients and zero filter coefficients. The filter (1600) has a rhombus shape (1620) formed by filter coefficients (1610) (shown as black circles). In one example, non-zero filter coefficients in the filter (1600) are included in filter coefficients (1610), and filter coefficients not included in filter coefficients (1610) are zero. Therefore, non-zero filter coefficients in the filter (1600) are included in the rhombus shape (1620), and filter coefficients not included in the rhombus shape (1620) are zero. In one example, the number of filter coefficients in the filter (1600) is equal to the number of filter coefficients (1610), which is 18 in the example shown in Figure 14.

[0140] A CC-ALF can contain any suitable filter coefficients (also called CC-ALF filter coefficients). Returning to Figure 15, CC-ALF(1521) and CC-ALF(1531) can have the same filter shape, such as the rhombus shape (1620) shown in Figure 14, and the same number of filter coefficients. In one example, the values ​​of the filter coefficients in CC-ALF(1521) are different from the values ​​of the filter coefficients in CC-ALF(1531).

[0141] In general, filter coefficients within CC-ALF (e.g., non-zero filter coefficients) can be transmitted, for example, via APS. In one example, the filter coefficients can be multiplied by (e.g., 2) 10It can be scaled and rounded for fixed-point display. The application of CC-ALF can be controlled by a variable block size and signaled by a context-coded flag (e.g., a CC-ALF enable flag) received for each block of samples. Context-coded flags, such as the CC-ALF enable flag, can be signaled at any appropriate level, such as the block level. The block size can be received at the slice level for each chroma component along with the CC-ALF enable flag. In some examples, block sizes of 16x16, 32x32, and 64x64 (in chroma sample units) can be supported.

[0142] As an example, the syntactic changes of CC-ALF are shown in Table 3 below.

[0143] [Table 3]

[0144] The meaning of CC-ALF related syntax can be explained below.

[0145] A value of 0 for alf_ctb_cross_component_cb_idc [xCtb >> CtbLog2SizeY ][ yCtb >> CtbLog2SizeY ] may indicate that the cross-component Cb filter is not applied to the block of Cb color component samples at the Luma position (xCtb, yCtb).

[0146] A non-zero alf_cross_component_cb_idc[ xCtb >> CtbLog2SizeY ][ yCtb >> CtbLog2SizeY ] may indicate that the alf_cross_component_cb_idc[ xCtb >> CtbLog2SizeY ][ yCtb >> CtbLog2SizeY ]-th cross-component Cb filter is applied to the block of Cb color component samples at the Luma position (xCtb, yCtb).

[0147] A value of 0 for alf_ctb_cross_component_cr_idc[ xCtb >> CtbLog2SizeY ][ yCtb >> CtbLog2SizeY ] may indicate that the cross-component Cr filter is not applied to the block of Cr color component samples at the Luma position (xCtb, yCtb).

[0148] A non-zero alf_cross_component_cr_idc[ xCtb >> CtbLog2SizeY ][ yCtb >> CtbLog2SizeY ] may indicate that the alf_cross_component_cr_idc[ xCtb >> CtbLog2SizeY ][ yCtb >> CtbLog2SizeY ]-th cross-component Cr filter is applied to the block of Cr color component samples at the Luma position (xCtb, yCtb).

[0149] Examples of chroma sampling formats are described below. Generally, a luma block can correspond to one or more chroma blocks, such as two chroma blocks. The number of samples in each of the chroma blocks can be less than the number of samples in the luma block. A chroma subsampling format (also called a chroma subsampling format, specified, for example, by chroma_format_idc) can specify the chroma horizontal subsampling factor (e.g., SubWidthC) and the chroma vertical subsampling factor (e.g., SubHeightC) between each of the chroma blocks and the corresponding luma block. In one example, the chroma subsampling format is 4:2:0, and therefore, as shown in Figures 15A and 15B, the chroma horizontal subsampling factor (e.g., SubWidthC) and the chroma vertical subsampling factor (e.g., SubHeightC) are 2. In one example, the chroma subsampling format is 4:2:2, and therefore the chroma horizontal subsampling factor (e.g., SubWidthC) is 2 and the chroma vertical subsampling factor (e.g., SubHeightC) is 1. In another example, the chroma subsampling format is 4:4:4, and therefore the chroma horizontal subsampling factor (e.g., SubWidthC) and the chroma vertical subsampling factor (e.g., SubHeightC) are both 1. The chroma sample type (also called the chroma sample position) can indicate the relative position of a chroma sample within a chroma block to at least one corresponding chroma sample within that chroma block.

[0150] Figures 17A and 17B show exemplary positions of chroma samples relative to chroma samples according to embodiments of the present disclosure. Referring to Figure 17A, chroma sample (1701) is located in rows (1711) to (1718). The chroma sample (1701) shown in Figure 17A may represent a portion of a picture. In one example, a chroma block (e.g., chroma CB) contains chroma sample (1701). A chroma block may correspond to two chroma blocks having a 4:2:0 chroma subsampling format. In one example, each chroma block contains chroma sample (1703). Each chroma sample (for example, chroma sample (1703(1))) corresponds to four luma samples (for example, luma samples (1701(1)) to (1701(4))). In one example, the four luma samples are the top-left sample (1701(1)), the top-right sample (1701(2)), the bottom-left sample (1701(3)), and the bottom-right sample (1701(4)). The chroma sample (for example, (1703(1))) corresponds to the top-left sample (1701(1)) and the bottom-left sample (1701 A chroma sample type of a chroma block located in the left-center position between (3)) and having chroma sample (1703) can be called chroma sample type 0. Chroma sample type 0 indicates relative position 0, which corresponds to the left-center position midway between the upper-left sample (1701(1)) and the lower-left sample (1701(3)). Four chroma samples (for example, (1701(1)) to (1701(4))) can be called adjacent chroma samples of chroma sample (1703)(1).

[0151] In one example, each chroma block contains a chroma sample (1704). The above description of chroma sample (1703) can be applied to chroma sample (1704), and therefore, for brevity, a detailed explanation can be omitted. Each of the chroma samples (1704) can be positioned in the center of four corresponding luma samples, and the chroma sample type of a chroma block having chroma samples (1704) can be called chroma sample type 1. Chroma sample type 1 indicates relative position 1, corresponding to the center of the four luma samples (e.g., (1701(1)) to (1701(4))). For example, one of the chroma samples (1704) can be positioned in the center of luma samples (1701(1)) to (1701(4)).

[0152] In one example, each chroma block contains a chroma sample (1705). Each chroma sample (1705) can be positioned in the upper left position, which is the same location as the upper left sample of the four corresponding luma samples (1701). The chroma sample type of a chroma block containing chroma samples (1705) can be called chroma sample type 2. Thus, each chroma sample (1705) is positioned in the same location as the upper left sample of the four luma samples (1701) corresponding to each chroma sample. Chroma sample type 2 indicates relative position 2, which corresponds to the upper left position of the four luma samples (1701). For example, one of the chroma samples (1705) can be positioned in the upper left position of luma samples (1701(1)) to (1701(4)).

[0153] In one example, each chroma block contains a chroma sample (1706). Each chroma sample (1706) can be positioned in the upper-center position between the corresponding upper-left sample and the corresponding upper-right sample, and the chroma sample type of a chroma block containing chroma samples (1706) can be called chroma sample type 3. Chroma sample type 3 indicates relative position 3, which corresponds to the upper-center position between the upper-left sample and the upper-right sample. For example, one of the chroma samples (1706) can be positioned in the upper-center position between chroma samples (1701(1))~(1701(4)).

[0154] In one example, each chroma block contains a chroma sample (1707). Each chroma sample (1707) can be positioned in the lower left position, which is the same location as the lower left sample of the four corresponding luma samples (1701). The chroma sample type of a chroma block containing chroma samples (1707) can be called chroma sample type 4. Thus, each chroma sample (1707) is positioned in the same location as the lower left sample of the four luma samples (1701) corresponding to each chroma sample. Chroma sample type 4 indicates relative position 4, which corresponds to the lower left position of the four luma samples (1701). For example, one of the chroma samples (1707) can be positioned in the lower left position of luma samples (1701(1)) to (1701(4)).

[0155] In one example, each chroma block contains a chroma sample (1708). Each chroma sample (1708) is located in the lower center position between the lower left sample and the lower right sample, and the chroma sample type of a chroma block containing a chroma sample (1708) can be called chroma sample type 5. Chroma sample type 5 indicates a relative position 5 corresponding to the lower center position between the lower left sample and the lower right sample of the four luma samples (1701). For example, one of the chroma samples (1708) can be placed between the lower left sample and the lower right sample of luma samples (1701(1)) to (1701(4)).

[0156] In general, any suitable chroma sample type can be used in a chroma subsampling format. Chroma sample types 0-5 are exemplary chroma sample types described in chroma subsampling format 4:2:0. Further chroma sample types may be used in chroma subsampling format 4:2:0. In addition, other chroma sample types and / or variations of chroma sample types 0-5 can be used in other chroma subsampling formats such as 4:2:2 and 4:4:4. For example, a chroma sample type combining chroma sample (1705) and chroma sample (1707) is used in chroma subsampling format 4:2:2.

[0157] For example, a luma block is considered to have alternating rows such as (1711)~(1712) containing the top two samples of four luma samples (e.g., (1701(1))~(1701(4))) (e.g., (1701(1))~(1701)(2)) and the bottom two samples of four luma samples (e.g., (1701(3))~(1701(4))) (e.g., (1701(3))~(1701(4))). Therefore, rows (1711), (1713), (1715), and (1717) are considered to have alternating rows such as (1711)~(1712) containing the top two samples of four luma samples (e.g., (1701(1))~(1701(4))). The current row can be called the top field, and rows (1712), (1714), (1716), and (1718) can be called the next row (also called the bottom field). The four luma samples (e.g., (1701(1)) to (1701(4))) are located in the current row (e.g., (1711)) and the next row (e.g., (1712)). Relative positions 2 and 3 are located in the current row, relative positions 0 and 1 are located between each current row and its respective next row, and relative positions 4 and 5 are located in the next row.

[0158] Chroma samples (1703), (1704), (1705), (1706), (1707), or (1708) are located in rows (1751) to (1754) within each chroma block. The specific position of rows (1751) to (1754) may depend on the chroma sample type of the chroma sample. For example, for chroma samples (1703) to (1704) which have chroma sample types 0 to 1, row (1751) is located between rows (1711) and (1712). For chroma samples (1705) to (1706) which have chroma sample types 2 to 3, row (1751) is located in the same position as the current row (1711). For chromatic samples (1707)-(1708) having chromatic sample types 4-5, row (1751) is located in the same position as the following row (1712). The above explanation can be appropriately applied to rows (1752)-(1754), and for brevity, a detailed explanation is omitted.

[0159] Any suitable scanning method can be used to display, store, and / or transmit the luma blocks and corresponding (one or more) chroma blocks described above in Figure 17A. In one example, sequential scanning is used.

[0160] As shown in Figure 17B, skip scanning can be used. As mentioned above, the chroma subsampling format is 4:2:0 (e.g., chroma_format_idc is equal to 1). In one example, the variable chroma position type (e.g., ChromaLocType) indicates the current row (e.g., ChromaLocType is chroma_sample_loc_type_top_field) or the next row (e.g., ChromaLocType is chroma_sample_loc_type_bottom_field). The current rows (1711), (1713), (1715), and (1717) and the next rows (1712), (1714), (1716), and (1718) can be scanned separately. For example, the current rows (1711), (1713), (1715), and (1717) can be scanned first, followed by the next rows (1712), (1714), (1716), and (1718). The current row may contain the luma sample (1701), and the next row may contain the luma sample (1702).

[0161] Similarly, corresponding chroma blocks can be scanned by skipping them. Rows (1751) and (1753) containing chroma samples (1703), (1704), (1705), (1706), (1707), or (1708) without fill can be called the current row (or current chroma row), and rows (1752) and (1754) containing chroma samples (1703), (1704), (1705), (1706), (1707), or (1708) with gray fill can be called the next row (or next chroma row). In one example, during a skip scan, rows (1751) and (1753) are scanned first, followed by rows (1752) and (1754).

[0162] Constrained Directional Enhancement Filters (CDEFs) can be used in video coding. In-loop CDEFs can be used to remove coding artifacts while preserving image detail. In some examples, such as HEVC, the Sample Adaptive Offset (SAO) algorithm achieves a similar objective by defining signal offsets for pixels of different classes. Unlike SAOs, CDEFs are nonlinear spatial filters. In some examples, the filter design is constrained to be easily vectorizable (e.g., implementable with single-instruction multiple-data (SIMD) operations), which is not the case with other nonlinear filters such as median filters or bidirectional filters.

[0163] In one example, the design of CDEF is conceived from the following observations: In some examples, the amount of ringing artifacts in the coded image can be approximately proportional to the quantization step size. The amount of detail is a characteristic of the input image. However, the minimum detail retained in the quantized image can be proportional to the quantization step size. For a given quantization step size, the magnitude of the ringing artifact can be smaller than the magnitude of the detail.

[0164] CDEF can identify the orientation of each block. In this case, CDEF can adaptively filter along the identified orientation and along a smaller angle along the orientation rotated 45° from the identified orientation. Filter strength can be explicitly signaled, allowing for advanced control over blurring. Efficient encoder search can be designed for the filter strength. CDEF can be based on two in-loop filters, and the combined filter can be used for video coding in AV1 codecs, etc.

[0165] Direction search can operate on pixels (or samples) reconstructed after, for example, a deblocking filter. Since the reconstructed pixels are available from the decoder, direction may not require signaling. When applied to quantized images, the search can operate on blocks (e.g., 8x8 blocks) of a suitable size that is small enough to properly handle nonlinear edges and large enough to reliably estimate the direction. Having a constant direction across the 8x8 region facilitates the vectorization of the filter. For each block, the direction that best matches the pattern within the block can be determined by minimizing a difference measure such as the sum of squared differences (SSD) or RMS error for each of the quantized blocks and fully directional blocks (1830). In one example, a fully directional block (e.g., one of the (1830)) is a block where all pixels along a line in a certain direction have the same value. Figure 18 shows an example of direction search for an 8x8 block (1810) according to one embodiment of the present disclosure. In the example shown in Figure 18, the 45-degree direction (1823) is selected from the pair of directions (1820) because it minimizes the error. For example, the error in the 45-degree direction is 12, which is the smallest error in the range from 12 to 87 shown in row (1840).

[0166] The following describes nonlinear low-pass directional filters. Identifying directions can be helpful in aligning filter taps along the identified directions to reduce ringing artifacts while preserving directional edges or patterns. However, in some examples, directional filtering alone is insufficient to reduce ringing artifacts. It is desirable to use additional filter taps for pixels that do not exist along the primary direction (e.g., the identified direction). These additional filter taps can be treated more conservatively to reduce the risk of blurring. Thus, CDEFs can define primary and secondary taps. In one example, a complete two-dimensional (2D) CDEF filter is represented as follows:

number

[0167] In equation (14), D is the damping parameter, and S (p) and S (s) These are the strengths of the primary and secondary taps, respectively, and the function round(·) can round the value away from zero.

number

number

[0168] In post-deblocking video coding, such as AV1, a set of in-loop restoration techniques can be used to de-noise and improve edge quality overall beyond the deblocking operation. This set of in-loop restoration techniques can be switchable within a frame (or picture) for each appropriately sized tile. Several examples of in-loop restoration techniques are described based on a dual self-guided filter using a separable symmetric Wiener filter and subspace projection. Because content statistics can vary significantly within a frame, these tools can be integrated into a switchable framework that can trigger different tools in different areas of the frame.

[0169] A separable symmetric Wiener filter can be described below. One restoration tool can be a Wiener filter. All pixels (or samples) in a degraded frame can be reconstructed as a non-causal filtered version of the pixels in a w×w window around that pixel, where w = 2r + 1 and is an odd integer r. A 2D filter tap is given by w 2 It can be represented as a column vectorized vector F with 1 element, and by simple linear least mean squared error (LMMSE) optimization, F = H -1 The filter parameters given by M can be obtained, where H is E[XX]. T ] is equal to the autocovariance of x, and w in the w×w window around the pixel. 2 This is a column vectorized version of the individual samples, E[YX T M, which is equal to ], is the cross-correlation between x and the scalar source sample y that should be estimated. The encoder can estimate H and M from the realized values ​​in the deblocked frame and source and send the resulting filter F to the decoder. However, in some examples, w 2 Sending individual taps can incur significant bitrate costs. Furthermore, inseparable filtering can make decoding extremely complex. Therefore, several additional constraints can be imposed on the properties of F. For example, F is constrained to be separable so that filtering can be implemented as separate horizontal and vertical w-tap convolutions. In one example, the horizontal and vertical filters are constrained to be symmetric. Furthermore, in one example, it can be assumed that the sum of the horizontal filter coefficients and the vertical filter coefficients is 1.

[0170] The following describes dual self-inductive filtering using subspace projection. In some examples, inductive filtering can be used in image filtering where a local linear model is used to calculate the filtered output from an unfiltered sample x. The local linear model can be written as follows: Equation (15): y = Fx + G In the formula, F and G can be determined based on the degraded image and a guide image (also called a guide image) of the neighborhood of the filtered pixels. If the guide image is identical to the degraded image, the resulting self-inductive filtering may have the effect of edge-preserving smoothing. According to some aspects of this disclosure, certain forms of self-inductive filtering may depend on two parameters, namely the radius r and the noise parameter e, which can be enumerated as follows:

[0171] 1. Mean μ and variance σ of each pixel within a (2r+1)×(2r+1) window around it. 2 Obtain the mean μ and variance σ of the pixels. 2 The acquisition of these parameters can be efficiently performed by box filtering based on integral imaging.

[0172] 2. Calculate the parameters f and g for each pixel based on equation (16). f=σ 2 / (σ 2 +e);g=(1-f)μ Equation (16)

[0173] 3. Calculate F and G for each pixel as the average value of the parameter f and parameter g within a 3x3 window around the pixel being used.

[0174] Filtering can be controlled by the radius r and the noise parameter e, where a larger radius r means higher spatial variance, and a higher noise parameter e may mean greater range variance.

[0175] Figure 19 shows an example of a subspace projection according to one embodiment of the present disclosure. In the example shown in Figure 19, the subspace projection uses readily available restored values ​​X1 and X2 to obtain a final restored value X that is closer to the source Y. fThis can generate the following. Even if the readily available restored values ​​X1 and X2 are not close to the source Y, if the readily available restored values ​​X1 and X2 move in the correct direction, an appropriate multiplier {α,β} can bring the readily available restored values ​​X1 and X2 considerably closer to the source Y. In one example, the final restored value X f This is obtained based on equation (17) below. X f =X+α(X1-X)+β(X2-X) Equation (17)

[0176] Distortion in reconstructed samples (also called reconstructed samples) can be reduced using a loop filtering method called a cross-component sample offset (CCSO) filter. In the CCSO filtering process, a nonlinear mapping can be used to determine the output offset based on the processed input reconstructed sample of the first color component. The output offset can then be added to the reconstructed sample of the second color component in the CCSO filtering process.

[0177] The input reconstructed sample may be from a first color component located in the filter support area, as shown in Figure 20. Figure 20 shows an example of a filter support area in a CCSO filter according to one embodiment of the present disclosure. The filter support area may include four reconstructed samples, namely p0, p1, p2, and p3. The four input reconstructed samples may trace a cross shape in the vertical and horizontal directions. In one example, the central sample (represented by c) in the first color component and the sample to be filtered (represented by f) in the second color component are located in the same place. When processing the input reconstructed sample, the following steps may be applied.

[0178] Step 1: The delta values ​​(e.g., differences) between the four reconstructed samples, namely p0, p1, p2, and p3, and the central sample c are calculated and expressed as m0, m1, m2, and m3, respectively. For example, the delta value between p0 and c is m0.

[0179] Step 2: The delta values ​​m0 to m3 can be further quantized. The quantized values ​​can be represented as d0, d1, d2, and d3. In one example, the quantized values ​​are -1, 0, or 1 based on the following quantization process. If mi < -N, then di = -1 (Equation 18) If -N ≤ mi ≤ N, then di = 0 (Equation 19) If mi > N, then di = 1 (Equation 20) In the formula, N is the quantization step size, with examples of N values ​​being 4, 8, 12, 16, etc., di and mi refer to the respective quantization value and delta value, and i is 0, 1, 2, or 3.

[0180] The quantization values ​​d0 to d3 can be used to identify combinations of nonlinear mappings. In the example shown in Figure 20, the CCSO filter has four filter inputs d0 to d3, and each filter input can have one of three quantization values ​​(e.g., -1, 0, and 1), so the total number of combinations is 81 (e.g., 3 4 Figures 21A to 21C show an example of 81 combinations according to one embodiment of the present disclosure. The last column may represent the output offset value for each combination. The output offset value can be an integer such as 0, 1, -1, 3, -3, 5, -5, -7, etc.

[0181] The final filtering process for the CCSO filter can be applied as follows. f'=clip(f+s) Equation (21) In the formula, f is the reconstructed sample to be filtered, and s is the output offset value taken from, for example, the table in Figures 21A to 21C. In the example shown in formula (21), the filtered sample value f' of the reconstructed sample f to be filtered can be further clipped to a range associated with the bit depth.

[0182] The Local Sample Offset (LSO) method or LSO filtering process can be used in video coding. LSO allows the application of filtering techniques similar to those used in CCSO. However, the output offset value can be applied to the color component, which is the same color component as the input reconstructed sample used in the filtering process. Therefore, in LSO, the input reconstructed samples used in the filtering process (e.g., p0-p3 and c) and the reconstructed sample to be filtered (e.g., f) are the same component, e.g., the luma component, the chroma component, or any other suitable component. LSO can have a filter shape similar to or identical to that of CCSO (e.g., as shown in Figure 20).

[0183] A less complex 3-tap CCSO design can be used for video coding. Figure 22 shows an exemplary implementation of CCSO according to one embodiment of the present disclosure. Eight different filter shapes can be defined. Each filter shape can define the positions of three reconstructed samples (also called three taps) in a first component (also called the first color component). The three reconstructed samples may include a central sample (represented as c). In one example, the reconstructed sample in the second color component to be filtered is located in the same place as the central sample c. For clarity, the reconstructed sample in the second color component to be filtered is not shown in Figure 22.

[0184] SAO filters can be used in video coding. In examples such as HEVC, an SAO filter or SAO filtering process can be applied to the reconstructed signal after deblocking filtering, for example, by using an offset value in the slice header. In the case of luma samples, the encoder can determine whether an SAO filter is applied to the current slice. If an SAO filter is enabled, the current picture can be recursively divided into four sub-regions, as shown in Table 4, and one of six SAO types (e.g., SAO types 1-6) can be selected for each sub-region. The SAO filter can reduce distortion by classifying the reconstructed pixels into multiple categories and adding an offset to the pixels in each category within the current sub-region. Edge characteristics can be used for pixel classification in SAO types 1-4, and pixel intensity can be used for pixel classification in SAO types 5-6.

[0185] [Table 4]

[0186] Using band offsets (BOs), pixels in a subregion (e.g., all pixels) can be classified into multiple bands, each containing pixels with the same intensity interval. The intensity range can be evenly divided into multiple intervals (e.g., 32 intervals) from the minimum intensity value (e.g., 0) to the maximum intensity value (e.g., 255 for 8-bit pixels), each interval potentially having an offset. Subsequently, multiple intervals or bands (e.g., 32 bands) can be divided into two groups. One group may contain 16 central bands, and the other group may contain the remaining 16 bands. In one example, only the offset of one group is sent. For pixel classification operations in BOs, the most significant 5 bits of each pixel can be used directly as the band index.

[0187] Edge offset (EO) can use four one-dimensional three-pixel patterns for pixel classification that take edge direction information into account, as shown in Figure 23. Figure 23 shows examples of four one-dimensional three-pixel patterns for pixel classification in EO. The four one-dimensional three-pixel patterns, from left to right, are the one-dimensional 0-degree pattern (2310), the one-dimensional 90-degree pattern (2320), the one-dimensional 135-degree pattern (2330), and the one-dimensional 45-degree pattern (2340), respectively. For each sub-region of the picture (e.g., the current picture), one of the four patterns can be selected, and pixels can be classified into multiple categories by comparing each pixel with its two adjacent pixels. The selection can be sent as side information in a bitstream. Table 5 shows the pixel classification rules for EO.

[0188] [Table 5]

[0189] In one example, it is desirable for the decoder-side SAO to operate independently of the LCU to conserve line buffers. To enable SAO to operate independently of the LCU, in one example, when the 90-degree, 135-degree, and 45-degree classification patterns are selected, the top and bottom rows of pixels within each LCU are not SAO-processed, and when the 0-degree, 135-degree, and 45-degree patterns are selected, the leftmost and rightmost columns of pixels within each LCU are not SAO-processed.

[0190] Table 6 below describes the syntax that can be used to signal a CTU when a parameter is not merged from an adjacent CTU.

[0191] [Table 6]

[0192] Certain filters or filter shapes used in the CCSO filtering process may require access to (one or more) adjacent samples that are relatively far from the central sample located in the same position as the reconstructed sample to be filtered (for example, filter shapes 5, 7, and 8 shown in Figure 22, which require access to (one or more) adjacent samples four rows above the central sample). In some cases, accessing (one or more) adjacent samples that are relatively far from the central sample may be too costly to implement in hardware the CCSO filter. Therefore, sample offset filters (e.g., CCSO or LSO) that use samples relatively close to the central sample can be used, as shown in Figures 24-28. In some cases, accessing rows that are below the central sample and relatively far from the central sample does not increase implementation costs as significantly as accessing rows that are above the central sample and relatively far from the central sample. Therefore, in some cases, constraints are imposed to access a limited number of rows above the central sample, and sample offset filters such as those shown in Figure 29 can be used.

[0193] According to aspects of this disclosure, a sample offset filter (or sample offset filtering process) may refer to a filter or filtering process in which a reconstructed sample of a first component (also called a first color component) is used as input to the sample offset filter (or sample offset filtering process), and the output of the sample offset filter (or sample offset filtering process) may be applied to a reconstructed sample (to be filtered) of a second component (or second color component). As shown in equation (21), a reconstructed sample of a second component may be filtered based on the output of the sample offset filter. The sample offset filter may be an in-loop filter in which the output (also called the output value) is applied to the reconstructed sample as an offset to remove coding artifacts while preserving the details of the second component. The sample offset filter may include one of LSO, CCSO, and adaptive sample offset filter (ASO).

[0194] The components, such as the first component and the second component, can be any appropriate components in the color space used for video coding (e.g., YCrCB, RGB, etc.), including the lumens, chromians, Y components, Cb components, Cr components, R components, G components, and B components. The second component can also be called the current component, as the reconstructed sample to be filtered will be processed by the encoder and / or decoder.

[0195] In one embodiment, the first component (or the first color component) is different from the second component (or the second color component), and the sample offset filter is referred to as a CCSO filter. More specifically, CCSO can refer to a filter or a filtering process in which the reconstructed samples of the first component are used as an input to the CCSO filter (or the CCSO filtering process), and the output of the CCSO filter (or the CCSO filtering process) can be applied to the reconstructed samples of the second component (or the second color component) to be filtered. As shown in Equation (21), the reconstructed samples of the second component can be filtered based on the output of the CCSO filter. In one example, in the case of CCSO, the first component is a luma component and the second component is a chroma component. In one example, in the case of CCSO, the first component and the second component are two different chroma components.

[0196] In one embodiment, the first component of the sample offset filter is the second component of the sample offset filter, and the sample offset filter can be referred to as an LSO filter. The LSO filter can be a filter or a filtering process in which the reconstructed samples of the first component (or the first color component) (e.g., Y, Cb, or Cr) are used as an input, and the output can be applied to the same first component, for example, to filter the reconstructed samples of the first component.

[0197] When a sample offset filter is applied to a reconstructed sample (to be filtered) of a current component (or current color component) in the current picture, the filter shape of the sample offset filter can be one of a plurality of filter shapes (or a plurality of filters). Each of the plurality of filter shapes can include a first reconstructed sample of a first component (or first color component) in the current picture. According to an aspect of the present disclosure, for each of the plurality of filter shapes, the top sample of the first reconstructed sample is located within N rows (or lines) from the central sample of the first component (also referred to as the central reconstructed sample) that is located at the same location as the reconstructed sample (to be filtered) of the current component. The reconstructed sample to be filtered is in the current component.

[0198] In some embodiments, the central sample of the first component is at the same position as the reconstructed sample (to be filtered) of the current component. In one example, the first component is a luma component and the current component is a chroma component. Referring to FIG. 17A, the central sample is a luma sample (1701(1)), the reconstructed sample to be filtered is a chroma sample 1705, and the central sample (e.g., (1701(1))) and the reconstructed sample to be filtered (e.g., (1705)) are at the same position.

[0199] In some embodiments, the central sample of the first component is located slightly differently from the reconstructed sample (to be filtered) of the current component. Referring to Figure 17A, the central sample is the chroma sample (1701(1)) and the reconstructed sample to be filtered is the chroma sample (1703(1)), and the central sample (e.g., (1701(1))) and the reconstructed sample to be filtered (e.g., (1703(1))) are located slightly differently (e.g., the difference along the vertical is less than one row). For example, referring to Figure 17A, the offset along the vertical in the current picture is 0.5 rows. The central sample (e.g., (1701(1))) and the reconstructed sample to be filtered (e.g., (1703(1))) are located in the same place.

[0200] In some embodiments, for each of the multiple filter shapes, the top sample of the first reconstructed sample lies within N rows (or lines) of the reconstructed sample to be filtered. N can be any suitable positive integer, such as in the range of 1 to 1024. For example, N can be constrained to a relatively small integer to reduce the hardware implementation cost of sample offset filters (e.g., CCSO, LSO). In some examples, when N is constrained to a relatively small integer, only adjacent samples that are within a constrained number of rows or lines from the central sample are used. In one example, N is 3 or less.

[0201] In one example, the number of first reconstruction samples in each of the multiple filter shapes is less than 5.

[0202] In one example, each of the multiple filter shapes has an equal number of M first reconstruction samples of the first component, where M is a positive integer. A filter or filter shape having several M first reconstruction samples can be called an M-tap filter or a filter with M taps.

[0203] In one example, N is 1. In this example, when N is 1, the number of first reconstruction samples in each of the multiple filter shapes is 3.

[0204] In one embodiment, each of the multiple filter shapes includes a central sample located at the same location as the reconstituted sample (to be filtered) of the current component. The first reconstituted sample includes the central sample. The topmost sample of the first reconstituted sample is located within N rows (or lines) from the central sample. As described above, N can be 3 or less. In one example, N is 1.

[0205] Next, the output value of the sample offset filter can be determined based on the first reconstructed sample within the filter shape. Furthermore, as shown in equation (21), the filtered sample value of the reconstructed sample for the current component can be determined based on the output values ​​of the reconstructed sample and the sample offset filter.

[0206] In one embodiment, each of the multiple filter shapes includes a central sample located in the same place as the reconstituted sample of the current component, and the number of first reconstituted samples in each of the multiple filter shapes is 3. The remaining first reconstituted samples (different from the central sample) in each of the multiple filter shapes can be arranged symmetrically or asymmetrically with respect to the central sample. Furthermore, in one example, the first reconstituted samples in each of the multiple filter shapes include the central sample and two samples that are symmetrical with respect to the central sample. In one example, N is 1.

[0207] In one example, the multiple filter shapes (or multiple filters) used in a sample offset filter (e.g., CCSO or LSO) include one or more of the following seven different 3-tap filter shapes, as shown in Figure 24. Figure 24 shows an example of the multiple filter shapes (or multiple filters) used in a sample offset filter (e.g., CCSO or LSO) according to one embodiment of the present disclosure. The multiple filter shapes include filter shapes (or multiple filters) 1 to 7. Each of the multiple filter shapes may include three first reconstructed samples, including a central sample (represented as c) and two samples symmetrical with respect to the central sample. N is 1. For example, in the case of filter shapes 1, 2, 4, 6, and 7, the top sample of the first reconstructed sample (e.g., each of samples (2401) to (2405)) is located within one row (e.g., one row above) of the reconstructed sample to be filtered (located in the same place as the central sample c). For example, in the case of filter shapes 3 and 5, the top sample of the first reconstructed sample is located within one row (e.g., on the same row) of the reconstructed sample to be filtered (located in the same place as the central sample c).

[0208] More specifically, referring to Figure 24, the position of the central sample c in the first component of the current picture is (i,j), where i and j are integers. Multiple filter shapes include seven filter shapes 1 to 7. The positions of the two samples in the first filter shape of the seven filter shapes are (i-1,j) and (i+1,j). The positions of the two samples in the second filter shape of the seven filter shapes are (i-1,j-1) and (i+1,j+1). The positions of the two samples in the third filter shape of the seven filter shapes are (i,j-1) and (i,j+1). The positions of the two samples in the fourth filter shape of the seven filter shapes are (i+1,j-1) and (i-1,j+1). The positions of the two samples in the fifth filter shape of the seven filter shapes are (i,j-4) and (i,j+4). In the sixth of the seven filter shapes, the positions of the two samples are (i-1, j-7) and (i+1, j+7). In the seventh of the seven filter shapes, the positions of the two samples are (i+1, j-7) and (i-1, j+7). A subset of the filter shapes in Figure 24 can be used in other embodiments. For example, multiple filter shapes (or multiple filters) used in a sample offset filter (e.g., CCSO or LSO) include one or more of the filter shapes 1-7 in Figure 24.

[0209] In one example, the multiple filter shapes (or multiple filters) used in a sample offset filter (e.g., CCSO or LSO) include one or more of the following six different 3-tap filter shapes, as shown in Figure 25. Figure 25 shows an example of the multiple filter shapes (or multiple filters) used in a sample offset filter (e.g., CCSO or LSO) according to one embodiment of the present disclosure. The multiple filter shapes include filter shapes (or multiple filters) 1 to 6. Each of the multiple filter shapes may include three first reconstructed samples, including a central sample (represented as c) and two samples symmetrical with respect to the central sample. N is 1. For example, in the case of filter shapes 1, 2, and 4, the top sample of the first reconstructed sample (e.g., each of samples (2501) to (2503)) is located within one row (e.g., one row above) of the reconstructed sample to be filtered (located in the same place as the central sample c). For example, in the case of filter shapes 3, 5, and 6, the top sample of the first reconstructed sample is located within one row (e.g., on the same row) of the reconstructed sample to be filtered (located in the same place as the central sample c).

[0210] More specifically, referring to Figure 25, the position of the central sample c in the first component of the current picture is (i,j), where i and j are integers. Multiple filter shapes include seven filter shapes 1 to 6. The positions of the two samples in the first filter shape of the six filter shapes are (i-1,j) and (i+1,j). The positions of the two samples in the second filter shape of the six filter shapes are (i-1,j-1) and (i+1,j+1). The positions of the two samples in the third filter shape of the six filter shapes are (i,j-1) and (i,j+1). The positions of the two samples in the fourth filter shape of the six filter shapes are (i+1,j-1) and (i-1,j+1). The positions of the two samples in the fifth filter shape of the six filter shapes are (i,j-4) and (i,j+4). In the sixth of the six filter shapes, the positions of the two samples are (i,j-7) and (i,j+7). A subset of these filter shapes can be used in other embodiments. For example, multiple filter shapes (or multiple filters) used in a sample offset filter (e.g., CCSO or LSO) include one or more of the filter shapes 1-6 in Figure 25.

[0211] In one example, the multiple filter shapes (or multiple filters) used in a sample offset filter (e.g., CCSO or LSO) include one or more of the following seven different 3-tap filter shapes, as shown in Figure 26. Figure 26 shows an example of the multiple filter shapes (or multiple filters) used in a sample offset filter (e.g., CCSO or LSO) according to one embodiment of the present disclosure. The multiple filter shapes include filter shapes (or multiple filters) 1 to 7. Each of the multiple filter shapes may include three first reconstructed samples, including a central sample (represented as c) and two samples symmetrical with respect to the central sample. N is 1. For example, in the case of filter shapes 1, 2, and 4-7, the top sample of the first reconstructed samples (e.g., each of samples (2601) to (2606)) is located within one row (e.g., one row above) of the reconstructed sample to be filtered (located in the same place as the central sample c). For example, in the case of filter shape 3, the topmost sample of the first reconstructed sample is located within one row (e.g., on the same row) of the reconstructed sample to be filtered (located in the same place as the central sample c).

[0212] More specifically, referring to Figure 26, the position of the central sample c in the first component of the current picture is (i,j), where i and j are integers. Multiple filter shapes include seven filter shapes 1 to 7. The positions of the two samples in the first filter shape of the seven filter shapes are (i-1,j) and (i+1,j). The positions of the two samples in the second filter shape of the seven filter shapes are (i-1,j-1) and (i+1,j+1). The positions of the two samples in the third filter shape of the seven filter shapes are (i,j-1) and (i,j+1). The positions of the two samples in the fourth filter shape of the seven filter shapes are (i+1,j-1) and (i-1,j+1). The positions of the two samples in the fifth filter shape of the seven filter shapes are (i-1,j-3) and (i+1,j+3). In the sixth of the seven filter shapes, the positions of the two samples are (i+1, j-5) and (i-1, j+5). In the seventh of the seven filter shapes, the positions of the two samples are (i-1, j-7) and (i+1, j+7). A subset of these filter shapes can be used in other embodiments. For example, multiple filter shapes (or multiple filters) used in a sample offset filter (e.g., CCSO or LSO) include one or more of the filter shapes 1 to 7 in Figure 26.

[0213] In one example, the multiple filter shapes (or multiple filters) used in a sample offset filter (e.g., CCSO or LSO) include one or more of the following seven different 3-tap filter shapes, as shown in Figure 27. Figure 27 shows an example of the multiple filter shapes (or multiple filters) used in a sample offset filter (e.g., CCSO or LSO) according to one embodiment of the present disclosure. The multiple filter shapes include filter shapes (or multiple filters) 1 to 7. Each of the multiple filter shapes may include three first reconstructed samples, including a central sample (represented as c) and two samples symmetrical with respect to the central sample. N is 1. For example, in the case of filter shapes 1, 2, and 4 to 7, the top sample of the first reconstructed samples (e.g., each of samples (2701) to (2706)) is located within one row (e.g., one row above) of the reconstructed sample to be filtered (located in the same place as the central sample c). For example, in the case of filter shape 3, the topmost sample of the first reconstructed sample is located within one row (e.g., on the same row) of the reconstructed sample to be filtered (located in the same place as the central sample c).

[0214] More specifically, referring to Figure 27, the position of the central sample c in the first component of the current picture is (i,j), where i and j are integers. Multiple filter shapes include seven filter shapes 1 to 7. The positions of the two samples in the first filter shape of the seven filter shapes are (i-1,j) and (i+1,j). The positions of the two samples in the second filter shape of the seven filter shapes are (i-1,j-1) and (i+1,j+1). The positions of the two samples in the third filter shape of the seven filter shapes are (i,j-1) and (i,j+1). The positions of the two samples in the fourth filter shape of the seven filter shapes are (i+1,j-1) and (i-1,j+1). The positions of the two samples in the fifth filter shape of the seven filter shapes are (i+1,j-3) and (i-1,j+3). The positions of the six samples in the sixth of the seven filter shapes are (i-1, j-5) and (i+1, j+5). The positions of the two samples in the seventh of the seven filter shapes are (i+1, j-7) and (i-1, j+7). A subset of these filter shapes can be used in other embodiments. For example, multiple filter shapes (or multiple filters) used in a sample offset filter (e.g., CCSO or LSO) include one or more of the filter shapes 1-7 in Figure 27.

[0215] In one example, the multiple filter shapes (or multiple filters) used in a sample offset filter (e.g., CCSO or LSO) include one or more of the following seven different 3-tap filter shapes (filter shapes or filters 1-7), as shown in Figure 28. Figure 28 shows an example of the multiple filter shapes (or multiple filters) used in a sample offset filter (e.g., CCSO or LSO) according to one embodiment of the present disclosure. Each of filter shapes 1-7 may include three first reconstructed samples, each including a central sample (represented as c) and two samples symmetrical with respect to the central sample. N is 1. For example, in filter shapes 1, 2, and 4, the top sample of the first reconstructed sample (e.g., each of samples (2801)-(2803)) is located within one row (e.g., one row above) of the reconstructed sample to be filtered (located in the same place as the central sample c). For example, in filter shapes 3 and 5-7, the top sample of the first reconstructed sample is located within one row (e.g., on the same row) of the reconstructed sample to be filtered (located in the same place as the central sample c).

[0216] More specifically, referring to Figure 28, the position of the central sample c in the first component of the current picture is (i,j), where i and j are integers. The positions of the two samples in the first of the seven filter shapes are (i-1,j) and (i+1,j). The positions of the two samples in the second of the seven filter shapes are (i-1,j-1) and (i+1,j+1). The positions of the two samples in the third of the seven filter shapes are (i,j-1) and (i,j+1). The positions of the two samples in the fourth of the seven filter shapes are (i+1,j-1) and (i-1,j+1). The positions of the two samples in the fifth of the seven filter shapes are (i,j-3) and (i,j+3). In the sixth of the seven filter shapes, the positions of the two samples are (i,j-5) and (i,j+5). In the seventh of the seven filter shapes, the positions of the two samples are (i,j-7) and (i,j+7). In one example, the multiple filter shapes (or multiple filters) used in a sample offset filter (e.g., CCSO or LSO) include one or more of the filter shapes 1 to 7 in Figure 28. According to one embodiment, the multiple filter shapes include filters 1 to 6 in Figure 28.

[0217] The filter or filter shape used for a sample offset filter (e.g., CCSO or LSO) may not be symmetric. In one example, the maximum number of the topmost row(s) (e.g., the row(s) above the central sample of the first component located at the same position as the reconstruction sample to be filtered) used as input to the sample offset filter (e.g., CCSO or LSO) and the maximum number of the bottommost row(s) (e.g., the row(s) below the central sample located at the same position as the reconstruction sample to be filtered) are different. FIG. 29 shows an example of filter shapes (or filters) 1 to 4 and 6 to 8 according to an embodiment of the present disclosure. Each of the filter shapes (or filters) 1 to 4 and 6 to 8 is a 3-tap filter including the central sample c of the first component and two other samples. The filter shapes 1 to 4 and 6 are symmetric, and the two other samples are symmetrically located with respect to the central sample c. The filter shapes 7 and 8 are asymmetric, and the two other samples are not symmetrically arranged with respect to the central sample c. In one example, when an asymmetric filter (e.g., filter 7 or filter 8 in FIG. 29) is used for a sample offset filter (e.g., CCSO or LSO), the maximum number of rows (or lines) of the topmost sample is limited to N with respect to the current sample to be filtered (e.g., the current reconstruction sample within the current component). In one example, the current sample is located at the same position as the central sample c of the first component. As described above, N can be any suitable positive integer. Examples of values of N may include, but are not limited to, 1 to 1024. In one example, N is 3 or less. In one example, N is 1.

[0218] According to some aspects of the present disclosure, the filter (e.g., the position(s) of the filter taps within the filter) or the filter shape used for a sample offset filter (e.g., CCSO or LSO) may depend on the relative position of the reconstruction sample (also referred to as the current sample) to be filtered within the FU (e.g., a 128×128 chroma block).

[0219] In one embodiment, coded information about the reconstructed sample of the current component in the filtering unit can be decoded from the coded video bitstream. The coded information may indicate a sample offset filter to be applied to the reconstructed sample to be filtered. The filter shape of the sample offset filter may include a first reconstructed sample of a first component in the current picture. The FU is located in the current picture. As described above, the filter shape can be determined based on the position (or relative position) of the reconstructed sample in the FU. The reconstructed sample may be located in the same place as the central sample of the filter shape. Subsequently, the output value of the sample offset filter can be determined based on the first reconstructed sample in the filter shape. Then, based on the reconstructed sample and the output value of the sample offset filter, the filtered sample value of the reconstructed sample of the current component can be determined.

[0220] In one embodiment, the current component is located within the filtering unit in the current picture. The filter shape of the sample offset filter can be determined based on the position of the reconstructed sample within the filtering unit.

[0221] In one embodiment, when applying a sample offset filter (e.g., CCSO or LSO), the sample offset filter is not permitted if it has one or more filter taps that reach beyond the top K rows with respect to the current sample to be filtered (or the central sample located at the same location as the current sample). In one example, the sample offset filter is replaced by another sample offset filter that has filter taps within the top K rows with respect to the current sample to be filtered. K can be a positive integer. In one example, K is equal to N.

[0222] In one embodiment, when applying a sample offset filter (e.g., CCSO or LSO), if the sample offset filter has one or more filter taps that reach beyond the top L rows relative to the current FU (or CTU) to be filtered, those one or more filter taps can be moved to the nearest neighbor sample that is in the same column but is located within the top L rows relative to the current FU (or CTU) to be filtered or otherwise padded. The top L rows may be relative to the samples in the current FU (or CTU). In one example, the top L rows are relative to the top boundary of the current FU (or CTU). For example, if the sample offset filter has one or more filter taps that reach beyond the top L rows relative to the top boundary of the current FU (or CTU) to be filtered, those one or more filter taps can be moved to the nearest neighbor sample that is in the same column but is located within the top L rows relative to the top boundary of the current FU (or CTU) to be filtered or otherwise padded. L can be an integer. In one example, L is equal to N.

[0223] Figure 30 shows an example of one or more filter taps (e.g., filter taps 7 and 8) being moved according to one embodiment of the present disclosure. Figure 30 shows seven filters (or filter shapes) 1-4 and 6-8. Filter taps 1-4 and 6 are located within the top four rows of the current FU (or CTU), such as the top boundary of the current FU, while two filter taps 7-8 (indicated by (3001)-(3002)) are located beyond the top four rows from the current FU (or CTU). Filter taps 7-8 are then moved to new positions indicated by (3003)-(3004). The new positions of filter taps 7-8 can be located within the top four rows from the current FU (or CTU), such as within the top one row of the current FU (or CTU).

[0224] Examples of filters, such as those shown in Figures 20, 22, and 24-30, include a central sample c located at the same location as the reconstructed sample (or current sample) to be filtered. In general, a sample offset filter can include or exclude the central sample.

[0225] Figure 31 shows a flowchart illustrating the process (3100) according to one embodiment of the present disclosure. Process (3100) can be used to filter (one or more) samples in the current picture, such as (one or more) samples in the FU of the current picture. In various embodiments, process (3100) is performed by processing circuits such as processing circuits in terminal devices (310), (330), and (340), processing circuits that perform the functions of a video encoder (403), processing circuits that perform the functions of a video decoder (410), processing circuits that perform the functions of a video decoder (510), and processing circuits that perform the functions of a video encoder (603). In some embodiments, process (3100) is implemented by software instructions, so that when a processing circuit executes a software instruction, the processing circuit executes process (3100). The process starts at (S3101) and proceeds to (S3110).

[0226] In (S3110), coded information about the reconstructed sample of the current component in the current picture can be decoded from the coded video bitstream. The coded information may indicate a sample offset filter to be applied to the reconstructed sample. The filter shape of the sample offset filter can be one of a plurality of filter shapes. Each of the plurality of filter shapes can contain a first reconstructed sample of a first component in the current picture. In one embodiment, for each of the plurality of filter shapes, the top sample of the first reconstructed sample can be located within N rows (or lines) from the central reconstructed sample (also called the central sample) in the first component, which is located in the same place as the reconstructed sample of the current component. N can be 3 or less.

[0227] In one example, the number of first reconstruction samples in each of the multiple filter shapes is less than 5.

[0228] In one example, each of the multiple filter shapes includes a central reconstitution sample located at the same location as the reconstitution sample of the current component, and the number of first reconstitution samples in each of the multiple filter shapes is 3. The first reconstitution samples in each of the multiple filter shapes may include the central reconstitution sample and two samples that are symmetrical with respect to the central reconstitution sample.

[0229] In one example, N is 1.

[0230] In one example, the position of the centrally reconstructed sample in the first component of the current picture is (i,j), where i and j are integers, and the multiple filter shapes include six filter shapes. The positions of the two samples in the first filter shape of the six filter shapes are (i-1,j) and (i+1,j). The positions of the two samples in the second filter shape of the six filter shapes are (i-1,j-1) and (i+1,j+1). The positions of the two samples in the third filter shape of the six filter shapes are (i,j-1) and (i,j+1). The positions of the two samples in the fourth filter shape of the six filter shapes are (i+1,j-1) and (i-1,j+1). The positions of the two samples in the fifth filter shape of the six filter shapes are (i,j-4) and (i,j+4). In the sixth of the six filter shapes, the positions of the two samples are (i,j-7) and (i,j+7).

[0231] In one example, the position of the central reconstructed sample in the first component of the current picture is (i,j), where i and j are integers, and the multiple filter shapes include six different filter shapes. The positions of the two samples in the first filter shape are (i-1,j) and (i+1,j). The positions of the two samples in the second filter shape are (i-1,j-1) and (i+1,j+1). The positions of the two samples in the third filter shape are (i,j-1) and (i,j+1). The positions of the two samples in the fourth filter shape are (i+1,j-1) and (i-1,j+1). The positions of the two samples in the fifth filter shape are (i,j-3) and (i,j+3). The positions of the two samples in the sixth filter shape are (i,j-5) and (i,j+5).

[0232] The sample offset filter can be either a CCSO filter or an LSO filter.

[0233] In one example, the first component is the second component. In another example, the first component is different from the second component. The first component can be the luma component, and the current component can be the chroma component.

[0234] In one example, the sample offset filter is an LSO filter, the first component is the current component, and the reconstructed sample within the current component is the central reconstructed sample.

[0235] In (S3120), the output value of the sample offset filter can be determined based on the first reconstructed sample within the filter shape.

[0236] In (S3130), the filtered sample value of the reconstructed sample for the current component can be determined based on the output values ​​of the reconstructed sample and the sample offset filter. In various examples, the sample offset filter is an in-loop filter whose output value is applied to the reconstructed sample as an offset to remove coding artifacts while preserving the details of the current component in the current picture. The sample offset filter may include one of LSO, CCSO, and Adaptive Sample Offset Filter (ASO). The process proceeds to (S3199) and terminates.

[0237] Process (3100) can be appropriately adapted. One or more steps of process (3100) can be modified and / or omitted. One or more additional steps can be added. Any suitable order of execution can be used. In one example, the current component is in the filtering unit in the current picture. The filter shape of the sample offset filter can be determined based on the position of the reconstituted sample in the filtering unit. The sample offset filter is a CCSO filter, and the filtering unit is a chromatic block with a 128 × 128 chromatic sample.

[0238] Figure 32 shows a flowchart illustrating the process (3200) according to one embodiment of the present disclosure. Process (3200) can be used to filter (one or more) samples in the current picture, such as (one or more) samples in the FU of the current picture. In various embodiments, process (3200) is performed by processing circuits such as processing circuits in terminal devices (310), (330), and (340), processing circuits that perform the functions of a video encoder (403), processing circuits that perform the functions of a video decoder (410), processing circuits that perform the functions of a video decoder (510), and processing circuits that perform the functions of a video encoder (603). In some embodiments, process (3200) is implemented by software instructions, so that when a processing circuit executes a software instruction, the processing circuit executes process (3200). The process starts at (S3201) and proceeds to (S3210).

[0239] In (S3210), coded information about the reconstructed sample of the current component in the filtering unit can be decoded from the coded video bitstream. The coded information can indicate a sample offset filter to be applied to the reconstructed sample. The filter shape of the sample offset filter can include the first reconstructed sample of the first component in the current picture, and the filtering unit is located in the current picture.

[0240] In one example, the filtering unit is a chromatic block with 128 × 128 chromatic samples.

[0241] In one example, the sample offset filter is a cross-component sample offset (CCSO) filter.

[0242] In one example, a sample offset filter is a local sample offset (LSO) filter.

[0243] As explained in (S3220) with reference to Figure 30, the filter shape (e.g., the position of each of the filter taps) can be determined based on the position of the reconstituted sample within the filtering unit.

[0244] In (S3230), the output value of the sample offset filter can be determined based on the first reconstructed sample within the filter shape.

[0245] In (S3240), the filtered sample value of the reconstituted sample for the current component can be determined based on the output values ​​of the reconstituted sample and the sample offset filter. The process proceeds to (S3299) and terminates.

[0246] Process (3200) can be appropriately adapted. One or more steps of process (3200) can be modified and / or omitted. One or more additional steps can be added. Any appropriate order of implementation can be used.

[0247] The embodiments of this disclosure may be used separately or combined in any order. Furthermore, each of the methods (or embodiments), encoders, and decoders may be implemented by processing circuits (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored on a non-temporary computer-readable medium. The embodiments of this disclosure may be applied to luma blocks or chroma blocks.

[0248] The aforementioned techniques can be implemented as computer software physically stored on one or more computer-readable media, using computer-readable instructions. For example, Figure 33 shows a computer system (3300) suitable for implementing a particular embodiment of the subject matter of the disclosure.

[0249] Computer software can be coded using any suitable machine code or computer language that can be subjected to mechanisms such as assembly, compilation, and linking to create code that contains instructions that can be executed directly or via interpretation, microcode execution, etc., by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc.

[0250] Instructions can be executed on various types of computers or their components, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, and Internet of Things devices.

[0251] The components shown in Figure 33 for the computer system (3300) are essentially illustrative and are not intended to imply any limitation on the scope of use or functionality of computer software implementing embodiments of the present disclosure. The configuration of the components should not be construed as having any dependencies or requirements on any one or combination of components shown in the exemplary embodiments of the computer system (3300).

[0252] The computer system (3300) may include certain human interface input devices, such as those that can respond to input from one or more human users via, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voices, applause), visual input (e.g., gestures), and olfactory input (not shown). The human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., voices, music, ambient sounds), images (e.g., scanned images, photographic images taken from a still image camera), and video (e.g., two-dimensional images, three-dimensional images including stereoscopic images).

[0253] The input human interface device may include one or more of the following (only one of each shown): a keyboard (3301), a mouse (3302), a trackpad (3303), a touchscreen (3310), a data glove (not shown), a joystick (3305), a microphone (3306), a scanner (3307), and a camera (3308).

[0254] The computer system (3300) may also include certain human interface output devices. Such human interface output devices may stimulate the senses of one or more human users, for example, by tactile output, sound, light, and smell / taste. Such human interface output devices may include tactile output devices (e.g., tactile feedback by touchscreen (3310), data glove (not shown), or joystick (3305), although there may also be tactile feedback devices that do not function as input devices), audio output devices (e.g., speakers (3309), headphones (not shown)), visual output devices (e.g., screens (3310), including CRT screens, LCD screens, plasma screens, and OLED screens, each with or without touchscreen input functionality, each with or without tactile feedback functionality, some of which may be capable of two-dimensional visual output, or four-dimensional or more output by means such as stereoscopic image output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0255] The computer system (3300) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (3320) with media such as CD / DVD (3321), thumb drives (3322), removable hard drives or solid-state drives (3323), legacy magnetic media such as tapes and floppy disks (not shown), and dedicated ROM / ASIC / PLD-based devices such as security dongles (not shown).

[0256] Those skilled in the art will also understand that the term “computer-readable medium” as used in relation to the subject matter of this disclosure does not include transmission media, carrier waves, or other transient signals.

[0257] The computer system (3300) may also include an interface (3354) to one or more communication networks (3355). The networks may be, for example, wireless, wired, or optical. The networks may further be local, wide-area, metropolitan, vehicle and industrial, real-time, or latency-tolerant. Examples of networks include local area networks such as Ethernet and Wi-Fi; cellular networks such as GSM, 3G, 4G, 5G, and LTE; wired or wireless wide-area digital television networks such as cable television, satellite television, and terrestrial television; and vehicle and industrial networks such as CANbus. Certain networks generally require an external network interface adapter attached to a specific general-purpose data port or peripheral bus (3349) (e.g., a USB port on the computer system (3300)). Other networks are generally integrated into the core of the computer system (3300) by attachment to a system bus, as described later (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (3300) can communicate with other entities. Such communications can be, for example, one-way receive only (e.g., television broadcasting), one-way transmit only (e.g., CANbus to a specific CANbus device), or two-way, to other computer systems using a local or wide-area digital network. Specific protocols and protocol stacks can be used for each of the aforementioned networks and network interfaces.

[0258] The aforementioned human interface device, human-accessible storage device, and network interface can be mounted on the core (3340) of the computer system (3300).

[0259] The core (3340) may include one or more central processing units (CPUs) (3341), graphics processing units (GPUs) (3342), dedicated programmable processing units in the form of field-programmable gate areas (FPGAs) (3343), hardware accelerators for specific tasks (3344), graphics adapters (3350), and the like. These devices, along with read-only memory (ROM) (3345), random access memory (3346), and internal mass storage such as hard drives and SSDs (3347) that are not accessible to the internal user, may be connected via a system bus (3348). In some computer systems, the system bus (3348) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be connected directly to the core's system bus (3348) or via a peripheral bus (3349). For example, a screen (3310) may be connected to the graphics adapter (3350). Peripheral bus architectures include PCI, USB, and others.

[0260] The CPU (3341), GPU (3342), FPGA (3343), and accelerator (3344) can execute specific instructions that can be combined to form the aforementioned computer code. This computer code can be stored in ROM (3345) or RAM (3346). Transition data can also be stored in RAM (3346), and persistent data can be stored, for example, in internal mass storage (3347). High-speed storage and retrieval to any of the memory devices can be made possible by using cache memory that can be closely associated with one or more CPUs (3341), GPUs (3342), mass storage (3347), ROM (3345), RAM (3346), etc.

[0261] Computer-readable media may contain computer code for performing various computer implementation operations. The media and computer code may be specifically designed and constructed for the purposes of this disclosure, or they may be of a type that is well known and available to those skilled in the computer software technology.

[0262] For example, but not limited to, a computer system having an architecture (3300), particularly a core (3340), can provide functionality as a result of (one or more) processors (including CPUs, GPUs, FPGAs, accelerators, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be user-accessible mass storage as described above, as well as media associated with specific storage of the core (3340) that are non-transient in nature, such as core internal mass storage (3347) or ROM (3345). Software implementing various embodiments of this disclosure can be stored in such devices and executed by the core (3340). The computer-readable media can include one or more memory devices or chips, depending on the specific needs. The software can cause the core (3340), specifically the processors (including CPUs, GPUs, FPGAs, etc.) therein, to execute specific processes or specific parts of specific processes described herein, including defining data structures stored in RAM (3346) and modifying such data structures according to processes defined by the software. In addition, or as an alternative, a computer system may also provide functionality as a result of logic wired or otherwise embodied in circuits (e.g., accelerators (3344)) that can perform specific processes or specific parts of specific processes described herein, instead of or in conjunction with software. Where we refer to software, it may include logic where appropriate, and vice versa. Where we refer to computer-readable media, it may include circuits (such as integrated circuits (ICs)) that house software for execution, circuits that embody logic for execution, or both, where appropriate. This disclosure encompasses any appropriate combination of hardware and software.

[0263] Note A: Acronym JEM: Collaborative Search Model VVC: Versatile Video Coding BMS: Benchmark Set MV: Motion Vector HEVC: High-Efficiency Video Coding SEI: Supplemental Enhancement Information VUI: Video Usability Information GOP: Group of Pictures TU: Conversion Unit PU: Prediction Unit CTU: Coding Tree Unit CTB: Coding Tree Block PB: Prediction Block HRD: Virtual Reference Decoder SNR: Signal-to-Noise Ratio CPU: Central Processing Unit GPU: Graphics Processing Unit CRT: Cathode Ray Tube LCD: Liquid crystal display OLED: Organic Light-Emitting Diode CD: Compact Disc DVD: Digital Video Disc ROM: Read-Only Memory RAM: Random Access Memory ASIC: Application-Specific Integrated Circuit PLD: Programmable Logic Device LAN: Local Area Network GSM: Global Mobile Communications System LTE: Long-Term Evolution CANBus: Controller Area Network Bus USB: Universal Serial Bus PCI: Peripheral device interconnection FPGA: Field-Programmable Gate Area SSD: Solid State Drive IC: Integrated Circuit HDR: High Dynamic Range SDR: Standard Dynamic Range JVET: Joint Video Exploration Team MPM: Most Probable Mode WAIP: Wide-angle intra-prediction CU: Coding Unit PU: Prediction Unit TU: Conversion Unit CTU: Coding Tree Unit PDPC: Location-dependent prediction combination ISP: Intra Subpartition SPS: Sequence Parameter Settings PPS: Picture Parameter Set APS: Adaptive Parameter Set VPS: Video Parameter Set DPS: Decryption parameter set ALF: Adaptive Loop Filter SAO: Sample Adaptive Offset CC-ALF: Cross-Component Adaptive Loop Filter CDEF: Constrained Directional Enhancement Filter CCSO: Cross-component sample offset LSO: Local Sample Offset LR: Loop Restoration Filter AV1: AOMedia Video 1 AV2:AOMedia Video 2

[0264] While this disclosure describes several exemplary embodiments, there are many modifications, substitutions, and alternative equivalents that fall within the scope of this disclosure. Therefore, those skilled in the art will understand that numerous systems and methods embodying the principles of this disclosure, and thus falling within the spirit and scope of this disclosure, can be devised, although these are not explicitly illustrated or described herein. [Explanation of symbols]

[0265] 101 samples 102 Arrow 103 Arrow 104 square blocks 180 Schematic Diagram 201 Current Block 202 surrounding samples 203 Surrounding samples 204 Surrounding samples 205 surrounding samples 206 surrounding samples 300 Communication Systems 310 Terminal device 320 Terminal devices 330 Terminal devices 340 Terminal devices 350 Networks 400 Communication Systems 401 Video Source 402 Video Picture Stream 403 Video Encoder 404 encoded video data 405 Streaming Server 406 Client Subsystem 407 Copy of encoded video data 408 Client Subsystem 409 Copies of encoded video data 410 Video Decoder 411 Video picture output stream 412 displays 413 Ingestion subsystem 420 Electronic equipment 430 Electronic equipment 501 Channel 510 Video Decoder 512 Rendering device 515 buffer memory 520 Entropy Decoder / Parser 521 Symbols 530 Electronic equipment 531 Receiver 551 Scaler / Inverse Conversion Unit 552 Intrapicture Prediction Units 553 Motion Compensation Prediction Unit 555 Aggregator 556 Loop Filter Unit 557 Reference Picture Memory 558 Current picture buffer 601 Video Sources 603 Video Encoder 620 Electronic equipment 630 Source Coder 632 Coding Engine 633 (Local) Decoder 634 Reference Picture Memory 635 Predictor 640 Transmitter 643 coded video sequence 645 Entropy Coder 650 Controller 660 communication channels 703 Video Encoder 721 General-purpose controller 722 Intra Encoders 723 Residual Calculator 724 Residual Encoder 725 Entropy Encoder 726 switches 728 Residual Decoder 730 Interencoder 810 Video Decoder 871 Entropy Decoder 872 Intra Decoder 873 Residual Decoder 874 Reconfiguration Module 880 Interdecoder 910 ALF 911 ALF 920 elements 921 elements 922 elements 923 elements 924 elements 925 elements 926 elements 927 elements 928 elements 929 elements 930 elements 931 elements 932 elements 941 elements 942 elements 943 elements 944 elements 945 elements 946 elements 947 elements 948 elements 949 elements 950 elements 951 elements 952 elements 953 elements 954 elements 955 elements 956 elements 957 elements 958 elements 959 elements 960 elements 961 elements 962 elements 963 elements 964 elements C0 value C1 value C2 value C3 value C4 value C5 value C6 value C7 value C8 value C9 value C10 value C11 Value C12 value H horizontal direction V vertical direction D1 Diagonal Direction D2 Diagonal 1110 4x4 block 1120 Horizontal CTU Boundary 1130 Virtual Boundary 1111 4x4 block 1121 CTU boundary 1131 Virtual Boundary 1210 Virtual Boundary 1220 Virtual Boundary 1230 Virtual Boundary 1240 virtual boundary 1250 virtual boundary 1260 virtual boundary 1510 Sample Adaptive Offset (SAO) Filter 1512 SAO filter 1514 SAO filter 1516 ALF Luma Filter 1518 ALF Chroma Filter 1521 Cross-Component Filter 1522 Adder 1531 Cross-Component Filter 1532 Adder 1541 SAO filtered luma component 1542 Second intermediate component 1543 Fourth intermediate component 1552 First intermediate component 1553 Third intermediate component 1561 Filtered Luma CB 1562 Filtered first chromatic component 1563 Filtered second chromatic component 1600 filters 1610 filter coefficients 1620 Diamond shape 1701 Luma Sample 1701(1) Top left Luma sample 1701(2) Upper right Luma sample 1701(3) Lower left Luma sample 1701(4) Lower right Luma sample 1702 Luma Sample 1703 Chroma Sample 1703(1) Chroma sample 1704 Chroma Sample 1705 Chroma Sample 1706 Chroma Sample 1707 Chroma Sample 1708 Chroma Sample Line 1711 Line 1712 Line 1713 Line 1714 Line 1715 1716 lines Line 1717 Line 1718 Line 1751 1752 lines 1753 lines 1754 lines 1810 8x8 blocks 1820 1 pair of directions 1823 45 degree direction 1830 Fully Directional Block 1840 lines p0 sample p1 Sample p2 Sample p3 Sample 2310 1D 0-degree pattern 2320 1D 90-degree pattern 2330 1D 135-degree pattern 2340 1D 45-degree pattern 2401 Top Sample 2402 Top Sample 2403 Top Sample 2404 Top Sample 2405 Top Sample 2501 Top Sample 2502 Top Sample 2503 Top Sample 2601 Top Sample 2602 Top Sample 2603 Top Sample 2604 Top Sample 2605 Top Sample 2606 Top Sample 2701 Top Sample 2702 Top Sample 2703 Top Sample 2704 Top Sample 2705 ​​Top Sample 2706 Top Sample 2801 Top Sample 2802 Top Sample 2803 Top Sample 3001 Filter tap position 3002 Filter tap position 3003 Filter tap position 3004 Filter tap position 3100 processes 3200 processes 3300 Computer Systems 3301 Keyboard 3302 Mouse 3303 Trackpad 3305 Joystick 3306 Microphone 3307 Scanner 3308 Camera 3309 Speaker 3310 Touchscreen 3320 CD / DVD ROM / RW 3321 CD / DVD and other media 3322 Thumb Drive 3323 Removable hard drive or solid state drive 3340 cores 3341 Central Processing Unit (CPU) 3342 Graphics Processing Units (GPUs) 3343 Field-Programmable Gate Area (FPGA) 3344 Hardware accelerators for specific tasks 3345 Read-Only Memory (ROM) 3346 Random Access Memory 3347 Cores, Large Capacity Internal Storage 3348 System Bus 3349 General-purpose data port or peripheral bus 3350 Graphics Adapter 3354 Interface 3355 Communication Network

Claims

[Claim 1] A method for video decoding in a decoder, A step of decoding coded information from a coded video bitstream about a reconstructed sample of a current component in the current picture, wherein the coded information indicates a sample offset filter to be applied to the reconstructed sample, the filter shape of the sample offset filter is one of a plurality of filter shapes, and each of the plurality of filter shapes includes a first reconstructed sample of a first component in the current picture, The steps include determining the output value of the sample offset filter based on the first reconstructed sample within the filter shape, A step of determining the filtered sample value of the current component of the reconstructed sample based on the reconstructed sample and the output value of the sample offset filter. A method comprising the following, wherein the sample offset filter is an in-loop filter to which the output value is applied as an offset to remove coding artifacts while preserving the details of the current component in the current picture.

Images