Filtering method, device, apparatus and readable medium for video decoding

Abstract

Aspects of the disclosure provide methods and apparatuses for video coding / decoding. In some examples, a device for video decoding includes processing circuitry. For example, the processing circuitry determines a first combination from reconstructed samples having a first color component within a filter support region. Then, the processing circuitry determines a first offset value associated with the first combination based on a mapping that associates offset values with combinations of possible reconstructed sample values, and applies the first offset value to a to-be-filtered sample having a second color component in the filter support region to determine a filtered sample having the second color component.

Description

[0001] Cross-referencing

[0002] This application claims the benefit of priority to U.S. Patent Application No. 17 / 240,812, filed April 26, 2021, entitled “METHOD AND APPARATUS FOR OFFSET IN VIDEO FILTERING,” which in turn claims the benefit of priority to U.S. Provisional Application No. 63 / 086,548, filed October 1, 2020, entitled “CROSS-COMPONENTSAMPLE OFFSET,” and U.S. Provisional Application No. 63 / 121,081, filed December 3, 2020, entitled “LOCAL SAMPLE OFFSET.” The entire disclosure of the earlier applications is incorporated herein by reference. Technical Field

[0003] This disclosure relates to video encoding and decoding technologies. In particular, this disclosure relates to a filtering method, apparatus, and device for video decoding, as well as a non-transitory computer-readable medium. Background Technology

[0004] The background description provided herein is for the purpose of presenting the overall context of this disclosure. To the extent that the work described in this background section is intended, neither the work of the currently identified inventors nor any aspect of the description which at the time of filing is otherwise limited to prior art is expressly or implicitly acknowledged as prior art to this disclosure.

[0005] Video encoding and decoding can be performed using inter-frame picture prediction with motion compensation. Uncompressed digital video can comprise a series of pictures, each with a spatial dimension of, for example, 1920×1080 luminance samples and associated chrominance samples. This series of pictures can have a fixed or variable picture rate (also informally referred to as frame rate), for example, 60 pictures per second or 60Hz. Uncompressed video has specific bitrate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (1920×1080 luminance sample resolution at a 60Hz frame rate) requires approximately 1.5 Gbit / s of bandwidth. One hour of such video would require more than 600 gigabytes (GByte) of storage space.

[0006] One purpose of video encoding and decoding is to reduce redundancy in the input video signal through compression. Compression can help reduce the aforementioned bandwidth and / or storage space requirements, in some cases by two orders of magnitude or more. Lossless compression, lossy compression, and combinations thereof can be used. Lossless compression refers to the technique of reconstructing an exact copy of the original signal from the compressed original signal. When using lossy compression, the reconstructed signal may differ from the original signal, but the distortion between the original and reconstructed signals is small enough that the reconstructed signal can be used for the target application. In the case of video, lossy compression is widely used. The amount of distortion tolerated depends on the application; for example, users of some consumer streaming applications may tolerate higher distortion compared to users of television distribution applications. The achievable compression ratio reflects this: higher allowable / tolerable distortion can result in a higher compression ratio.

[0007] Video encoders and decoders can utilize techniques from several broad categories, including motion compensation, transform, quantization, and entropy coding.

[0008] Video codec techniques can include techniques called intra-frame coding. In intra-frame coding, sample values ​​are represented without reference to samples or other data from a previously reconstructed reference picture. In some video codecs, a picture is spatially subdivided into sample blocks. This picture can be an intra-frame picture when all sample blocks are encoded in intra-frame mode. Intra-frame pictures and their derivatives, such as independent decoder refresh pictures, can be used to reset the decoder state and therefore can be used as the first picture in the encoded video bitstream and video session or as a still image. Transforms can be applied to the samples of an intra-frame block, and the transform coefficients can be quantized before entropy coding. Intra-frame prediction can be a technique that minimizes the sample values ​​in the pre-transform domain. In some cases, the smaller the DC value and AC coefficients after the transform, the fewer bits are needed to represent the entropy-coded block at a given quantization step size.

[0009] Traditional intra-frame coding techniques, such as those known from MPEG-2 generation coding technologies, do not use intra-frame prediction. However, some newer video compression techniques include attempts to use metadata obtained during the encoding / decoding of, for example, spatially adjacent data blocks that are first in the decoding order, along with / or surrounding sample data. Such techniques are hereinafter referred to as "intra-frame prediction" techniques. Note that in at least some cases, intra-frame prediction uses only reference data from the current image being reconstructed, and not reference data from a reference image.

[0010] Many different forms of intra-prediction can exist. When more than one such technique can be used for a given video coding technique, the technique used can be encoded within an intra-prediction mode. In some cases, a mode can have sub-modes and / or parameters, and these sub-modes and / or parameters can be encoded individually or included in the mode codeword. The choice of codewords used for a given combination of mode / sub-mode / parameters can affect the coding efficiency gain through intra-prediction, and therefore can affect the entropy coding technique used to convert the codewords into a bitstream.

[0011] Some patterns of intra-frame prediction were introduced in H.264, refined in H.265, and further refined in newer coding techniques such as the Joint Exploration Model (JEM), Versatile Video Coding (VVC), and Benchmark Sets (BMS). Predictor blocks can be formed using neighboring sample values ​​belonging to already available samples. The sample values ​​of neighboring samples are copied into the predictor block according to the orientation. The reference to the orientation used can be encoded in the bitstream, or it can be predicted itself.

[0012] Referring to Figure 1A, a subset of nine predictor directions known from the 33 possible predictor directions of H.265 (corresponding to 33 angular modes out of 35 intra-frame modes) is depicted in the lower right. The point (101) where the arrows converge represents the sample being predicted. The arrows indicate the direction in which the sample is predicted. For example, arrow (102) indicates that sample (101) is predicted based on one or more samples at a 45-degree angle to the horizontal in the upper right. Similarly, arrow (103) indicates that sample (101) is predicted based on one or more samples at a 22.5-degree angle to the horizontal in the lower left of sample (101).

[0013] Referring again to Figure 1A, a 4×4 square block (104) of samples is depicted in the upper left (indicated by a bold dashed line). The square block (104) comprises 16 samples, each labeled with “S”, its position in the Y dimension (e.g., row index), and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (from top) and the first sample in the X dimension (from left). Similarly, sample S44 is the fourth sample in block (104) in both the Y and X dimensions. Since the block size is 4×4 samples, S44 is in the lower right. Also shown are reference samples following a similar numbering scheme. Reference samples are labeled with R, their Y position (e.g., row index) relative to block (104), and their X position (column index). In both H.264 and H.265, the predicted samples are adjacent to the blocks being reconstructed; therefore, negative values ​​are not required.

[0014] Intra-frame image prediction can work by appropriately copying reference sample values ​​from neighboring samples according to the prediction direction indicated by a signal. For example, suppose the encoded video bitstream includes signaling for that block, the signaling indicating a prediction direction consistent with arrow (102)—that is, predicting samples based on one or more prediction samples at a 45-degree angle to the horizontal from the upper right. In this case, samples S41, S32, S23, and S14 are predicted based on the same reference sample R05. Then sample S44 is predicted based on reference sample R08.

[0015] In some cases, the values ​​of multiple reference samples can be combined, for example, by interpolation, to calculate the reference sample; especially when the direction is not divisible by 45 degrees.

[0016] As video coding technology has developed, the number of possible directions has also increased. In H.264 (2003), nine different directions could be represented. This increased to 33 in H.265 (2013), and JEM / VVC / BMS could support up to 65 directions at the time of publication. Experiments have been conducted to identify the most likely directions, and certain techniques in entropy coding have been used to represent these possible directions with a small number of bits, thus incurring some penalty for less likely directions. Furthermore, the direction itself can sometimes be predicted based on neighboring directions used in nearby decoded blocks.

[0017] Figure 1B shows a schematic diagram (180) depicting 65 intra-frame prediction directions according to JEM to illustrate how the number of prediction directions increases over time.

[0018] The mapping of intra-predicted direction bits representing direction in a encoded video bitstream can vary from video coding technique to video coding technique; and the range can be, for example, from a simple direct mapping of predicted direction to intra-predicted mode to codewords, to complex adaptive schemes involving the most probable mode and similar techniques. However, in all cases, there may be some directions that are statistically less likely to appear in the video content than some other directions. Since the goal of video compression is to reduce redundancy, in well-functioning video coding techniques, those less probable directions will be represented by a larger number of bits than the more probable directions.

[0019] Motion compensation can be a lossy compression technique and can involve predicting a newly reconstructed image or a portion thereof using blocks of sample data from a previously reconstructed image or a portion thereof (the reference image) after spatial shifting in the direction indicated by the motion vector (hereinafter referred to as MV). In some cases, the reference image may be the same as the image currently being reconstructed. MV may have two dimensions, X and Y, or three dimensions, with the third dimension being an indication of the reference image in use (the third dimension may indirectly be a temporal dimension).

[0020] In some video compression techniques, an MV applicable to a specific region of sample data can be predicted based on other MVs. For example, the MV can be predicted based on another region of sample data that is spatially adjacent to the region being reconstructed and precedes that MV in the decoding order. Doing so can significantly reduce the amount of data required to encode the MV, thereby eliminating redundancy and increasing compression. MV prediction can work efficiently, for example, because when encoding the input video signal (called natural video) from the camera device, there is a statistical probability that a larger region than the region applicable to a single MV moves in a similar direction. Therefore, in some cases, similar motion vectors derived from MVs of neighboring regions can be used for prediction. This makes the MV found for a given region similar to or the same as the MV predicted based on the surrounding MVs, and can be represented after entropy encoding with fewer bits than would be used if the MV were encoded directly. In some cases, MV prediction can be an example of lossless compression of the signal (i.e., the MV) derived from the original signal (i.e., the sample stream). In other cases, MV prediction itself can be lossy, for example, due to rounding errors when calculating the predictor based on several surrounding MVs.

[0021] Various MV prediction mechanisms are described in H.265 / HEVC (ITU-T H.265 Recommendation, “High Efficiency Video Coding”, December 2016). Among the various MV prediction mechanisms provided by H.265, the one described here is the technique referred to below as “spatial combining”.

[0022] Referring to Figure 2, the current block (201) includes samples that have been discovered by the encoder during the motion search process and can be predicted based on previous blocks of the same size that have been spatially shifted. Instead of directly encoding this MV, the MV can be derived from metadata associated with one or more reference images, for example, from the most recent (in decoding order) reference image, using the MV associated with any of the five surrounding samples denoted as A0, A1 and B0, B1, B2 (202 to 206, respectively). In H.265, MV prediction can use a predictor from the same reference image being used by neighboring blocks.

[0023] In existing technologies, relevant CCF examples are based on the following linear assumptions: the luminance reconstructed sample value has a linear mapping relationship with the incremental value between the original chrominance sample and the chrominance reconstructed sample; or the color component reconstructed sample value has a linear mapping relationship with the incremental value between the original sample with that color component and the reconstructed sample with that color component. Furthermore, the reconstructed samples obtained based on existing filtering techniques have high distortion. Summary of the Invention

[0024] This disclosure provides a method and apparatus for video encoding / decoding. In some examples, an apparatus for video decoding includes a processing circuitry system. For example, the processing circuitry determines a first combination based on reconstructed samples having a first color component within a filter support region. The processing circuitry then determines a first offset value associated with the first combination based on a mapping that associates an offset value determined from the processing results of the reconstructed samples having the first color component with possible combinations of reconstructed sample values; and applies the first offset value to a sample to be filtered having a second color component within the filter support region to determine a filtered sample having the second color component.

[0025] In some implementations, the mapping is a non-linear mapping defined in the form of a lookup table. The second color component is different from the first color component.

[0026] In some implementations, the processing circuitry determines a first combination based on reconstructed samples with a first color component within the filter support region, using at least one of linear operations and quantization operations. In some examples, the processing circuitry calculates the differences between each pair of reconstructed samples with the first color component and quantizes these differences to determine the first combination. In one example, the processing circuitry calculates the difference between surrounding reconstructed samples and juxtaposed samples with the first color component within the reconstructed samples. These juxtaposed samples are located at the same position as the sample to be filtered, which has a second color component. The processing circuitry then quantizes the differences based on a quantization step size to determine the first combination.

[0027] In one implementation, the mapping associates the offset value with a combination of full domains of reconstructed samples having a first color component within the filter support region. In another implementation, the mapping associates the offset value with a combination of subset domains of reconstructed samples having a first color component within the filter support region.

[0028] In some implementations, the mapping is carried in at least one of the video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), adaptive parameter set (APS), slice header, and tile header of the encoded video bitstream.

[0029] In an implementation, the processing circuitry derives the offset value of the mapping based on a cost metric, and includes the mapping in at least one of the video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), adaptive parameter set (APS), slice header, and tile header of the encoded video bitstream.

[0030] In some implementations, the processing circuit adds a first offset value to a sample to be filtered that has a second color component to determine an intermediate filtered sample; and limits the intermediate filtered sample based on the maximum and minimum values ​​to determine a filtered sample that has a second color component.

[0031] This disclosure also provides a filtering apparatus for video decoding. The apparatus includes: a first determining module configured to determine a first combination based on reconstructed samples having a first color component within a filter support region; a second determining module configured to determine a first offset value associated with the first combination based on a mapping that associates an offset value determined from processing the reconstructed samples having the first color component with possible combinations of reconstructed sample values; and a third determining module configured to apply the first offset value to a sample to be filtered having a second color component within a filter support region to determine a filtered sample for the second color component.

[0032] This disclosure also provides a non-transitory computer-readable medium storing instructions that, when executed by a computer for video decoding, cause the computer to perform any of the methods for video decoding.

[0033] The technical solution according to the present invention can achieve the following technical effects: On the encoder side, the encoding device can derive a mapping between the reconstructed sample with a first color component in the filter support region and the offset to be added to the reconstructed sample with a second color component; then, a filtering process is applied on the encoder side and / or the decoder side based on this mapping. CCSO and LSO, used as loop filters, can reduce the distortion of the reconstructed samples. Furthermore, CCSO and LSO do not rely on the linear mapping assumption used in the relevant example CCF. Attached Figure Description

[0034] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0035] Figure 1A is a schematic illustration of an exemplary subset of intra-prediction modes.

[0036] Figure 1B is an illustration of an exemplary intra-frame prediction direction.

[0037] Figure 2 is a schematic illustration of the current block and its surrounding space merge candidates in an example.

[0038] Figure 3 It is a schematic diagram of a simplified block diagram of a communication system according to an embodiment.

[0039] Figure 4 It is a schematic diagram of a simplified block diagram of a communication system according to an embodiment.

[0040] Figure 5 This is a schematic diagram of a simplified block diagram of a decoder according to an embodiment.

[0041] Figure 6 This is a schematic diagram of a simplified block diagram of an encoder according to an embodiment.

[0042] Figure 7 A block diagram of an encoder according to another embodiment is shown.

[0043] Figure 8 A block diagram of a decoder according to another embodiment is shown.

[0044] Figure 9 An example of a filter shape according to an embodiment of this disclosure is shown.

[0045] Figures 10A to 10DAn example of a subsampling location for calculating a gradient according to an embodiment of this disclosure is shown.

[0046] Figures 11A to 11B An example of a virtual boundary filtering process according to an embodiment of this disclosure is shown.

[0047] Figures 12A to 12F An example of a symmetrical filling operation at a virtual boundary according to an embodiment of this disclosure is shown.

[0048] Figure 13 The images shown are examples of partitions according to some embodiments of this disclosure.

[0049] Figure 14 The quadtree segmentation patterns used for images are shown in some examples.

[0050] Figure 15 A cross component filter according to an embodiment of the present disclosure is shown.

[0051] Figure 16 An example of a filter shape according to an embodiment of this disclosure is shown.

[0052] Figure 17 Syntax examples for cross-component filters according to some embodiments of this disclosure are shown.

[0053] Figures 18A to 18B An exemplary position of a chromaticity sample relative to a luminance sample according to an embodiment of this disclosure is shown.

[0054] Figure 19 An example of directional search according to an implementation of this disclosure is shown.

[0055] Figure 20 An example of subspace projection is shown, illustrating some examples.

[0056] Figure 21 A table of multiple sample adaptive offset (SAO) types according to an embodiment of this disclosure is shown.

[0057] Figure 22 Examples of patterns used for pixel classification in edge offsets are shown in some examples.

[0058] Figure 23 A table showing pixel classification rules used for edge offsets in some examples is provided.

[0059] Figure 24 An example of a syntax that can be used for signal notification is shown.

[0060] Figure 25Examples of filter support regions according to some embodiments of this disclosure are shown.

[0061] Figure 26 An example of another filter support region according to some embodiments of this disclosure is shown.

[0062] Figures 27A to 27C A table showing 81 combinations of embodiments according to this disclosure is provided.

[0063] Figure 28 A lookup table is shown according to an embodiment of this disclosure.

[0064] Figure 29 A flowchart outlining the process of implementing this disclosure is shown.

[0065] Figure 30 It is a schematic diagram of a computer system according to an implementation method. Detailed Implementation

[0066] Figure 3 A simplified block diagram of a communication system (300) according to an embodiment of the present disclosure is shown. The communication system (300) includes a plurality of terminal devices that can communicate with each other via, for example, a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) interconnected via the network (350). Figure 3 In the example, the first pair of terminal devices (310) and (320) perform one-way data transmission. For example, terminal device (310) may encode video data (e.g., a video image stream captured by terminal device (310)) for transmission via network (350) to another terminal device (320). The encoded video data may be transmitted as one or more encoded video bitstreams. Terminal device (320) may receive the encoded video data from network (350), decode the encoded video data to recover the video images, and display the video images based on the recovered video data. One-way data transmission can be common in media service applications, etc.

[0067] In another example, the communication system (300) includes a second pair of terminal devices (330) and (340) that perform bidirectional transmission of encoded video data, which may occur, for example, during a video conference. For the bidirectional transmission of data, in this example, each of the terminal devices (330) and (340) may encode video data (e.g., a stream of video images captured by the terminal device) for transmission via a network (350) to the other terminal device (330) and (340). Each of the terminal devices (330) and (340) may also receive encoded video data transmitted by the other terminal device (330) and (340), and may decode the encoded video data to recover video images, and may display the video images at an accessible display device based on the recovered video data.

[0068] exist Figure 3 In the examples, terminal devices (310), (320), (330), and (340) may be shown as servers, personal computers, and smartphones, but the principles of this disclosure are not limited thereto. Embodiments of this disclosure can be used in laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network (350) refers to any number of networks that transmit encoded video data between terminal devices (310), (320), (330), and (340), including, for example, wired (connected) and / or wireless communication networks. Communication networks (350) may exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet. For the purposes of this discussion, unless otherwise stated below, the architecture and topology of the network (350) may be of little importance to the operation of this disclosure.

[0069] As an example of the application of the disclosed topic, Figure 4 The placement of a video encoder and video decoder in a streaming environment is illustrated. The disclosed subject matter is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, and storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0070] The streaming system may include a capture subsystem (413), which may include a video source (401), such as a digital camera device, which creates, for example, an uncompressed video picture stream (402). In the example, the video picture stream (402) includes samples captured by the digital camera device. The video picture stream (402) is depicted as a thick line to emphasize the high data volume when compared with encoded video data (404) (or encoded video bitstream), which may be processed by an electronic device (420) including a video encoder (403) coupled to the video source (401). The video encoder (403) may include hardware, software, or a combination thereof to implement or carry out aspects of the disclosed subject matter as described in more detail below. Encoded video data (404) (or encoded video bitstream (404)) is depicted as a thin line to emphasize the lower data volume when compared to the video picture stream (402). The encoded video data (404) can be stored on a streaming server (405) for future use. One or more streaming client subsystems, such as Figure 4 Client subsystems (406) and (408) can access a streaming server (405) to retrieve copies (407) and (409) of encoded video data (404). Client subsystem (406) may include, for example, a video decoder (410) in an electronic device (430). The video decoder (410) decodes the incoming copy (407) of the encoded video data and creates an outgoing video picture stream (411) that can be displayed on a display (412) (e.g., a screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (404), (407), and (409) (e.g., video bitstreams) may be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T Recommendation H.265. In this example, the video coding standard under development is informally referred to as Versatile Video Coding (VVC). The disclosed topics can be used in the context of VVC.

[0071] Note that electronic devices (420) and (430) may include other components (not shown). For example, electronic device (420) may include a video decoder (not shown), and electronic device (430) may also include a video encoder (not shown).

[0072] Figure 5A block diagram of a video decoder (510) according to an embodiment of the present disclosure is shown. The video decoder (510) may be included in an electronic device (530). The electronic device (530) may include a receiver (531) (e.g., a receiving circuitry system). The video decoder (510) may be used in place of Figure 4 The video decoder (410) in the example.

[0073] The receiver (531) can receive one or more encoded video sequences to be decoded by the video decoder (510); in the same or another embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the other encoded video sequences. The encoded video sequences can be received from a channel (501), which can be a hardware / software link to a storage device storing the encoded video data. The receiver (531) can receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that can be forwarded to their respective user entities (not depicted). The receiver (531) can separate the encoded video sequences from other data. To prevent network jitter, a buffer memory (515) can be coupled between the receiver (531) and the entropy decoder / parser (520) (hereinafter referred to as "parser (520)"). In some applications, the buffer memory (515) is part of the video decoder (510). In other applications, the buffer memory (515) can be external to the video decoder (510) (not depicted). In some other applications, a buffer memory (not depicted) may be present outside the video decoder (510) to prevent network jitter, for example, and another buffer memory (515) may be present inside the video decoder (510) for example, to handle broadcast timing. The buffer memory (515) may not be necessary, or may be small, when the receiver (531) receives data from a store / forward device with sufficient bandwidth and controllability, or from an isosynchronous network. For use on optimal packet networks such as the Internet, a buffer memory (515) may be required; the buffer memory (515) may be relatively large and advantageously adaptive in size, and may be implemented at least partially in the operating system or in a similar element (not depicted) outside the video decoder (510).

[0074] The video decoder (510) may include a parser (520) to reconstruct symbols (521) from the encoded video sequence. These symbols include: information for managing the operation of the video decoder (510); and potential information for controlling a presentation device such as a presentation device (512) (e.g., a display screen), which is not part of the electronic device (530) but may be coupled to it, such as... Figure 5 As shown. Control information for one or more presentation devices may be in the form of Supplemental Enhancement Information (SEI) messages or Video Usability Information (VUI) parameter set fragments (not depicted). The parser (520) may parse / decode the received encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or video coding standards and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (520) may extract a subgroup parameter set from the encoded video sequence for at least one subgroup of pixels in the subgroups for use in the video decoder, based on at least one parameter corresponding to a group. Subgroups may include Group of Picture (GOP), picture, tile, slice, macroblock, coding unit (CU), block, transform unit (TU), prediction unit (PU), etc. The parser (520) can also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from the encoded video sequence.

[0075] The parser (520) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (515) to create symbols (521).

[0076] Depending on the type of encoded video picture or a portion thereof (e.g., inter-frame picture and intra-frame picture, inter-frame block and intra-frame block) and other factors, the reconstruction of the symbol (521) may involve multiple different units. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the encoded video sequence by the parser (520). For the sake of brevity, the flow of such subgroup control information between the parser (520) and the following multiple units is not described.

[0077] In addition to the functional blocks already mentioned, the video decoder (510) can be conceptually subdivided into several functional units as described below. In a practical implementation operating under commercial constraints, many of these units interact closely with each other and can be at least partially integrated with each other. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into the following functional units.

[0078] The first unit is the scaler / inverse transform unit (551). The scaler / inverse transform unit (551) receives from the parser (520) quantization transform coefficients as symbols (521) and control information, including which transform to use, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (551) can output a block containing sample values ​​that can be input to the aggregator (555).

[0079] In some cases, the output samples of the scaler / inverse transform unit (551) may belong to intra-coded blocks; that is, blocks that do not use predictive information from previously reconstructed images but can use predictive information from previously reconstructed portions of the current image. Such predictive information may be provided by the intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) generates blocks of the same size and shape as the blocks in the reconstruction using surrounding reconstructed information obtained from the current image buffer (558). For example, the current image buffer (558) buffers partially reconstructed and / or fully reconstructed current images. In some cases, the aggregator (555) adds the predictive information already generated by the intra-picture prediction unit (552) to the output sample information provided by the scaler / inverse transform unit (551) on a per-sample basis.

[0080] In other cases, the output samples of the scaler / inverse transform unit (551) may belong to inter-frame coded and possibly motion-compensated blocks. In such cases, the motion compensation prediction unit (553) can access the reference image memory (557) to obtain samples for prediction. After motion compensation of the obtained samples according to the symbols (521) belonging to the block, these samples can be added to the output of the scaler / inverse transform unit (551) by the aggregator (555) (in this case, referred to as residual samples or residual signals) to generate output sample information. The address in the reference image memory (557) from which the motion compensation prediction unit (553) obtains the predicted samples can be controlled by motion vectors, which are provided to the motion compensation prediction unit (553) in the form of symbols (521), which may have, for example, X components, Y components, and reference image components. Motion compensation may also include interpolation of sample values ​​obtained from the reference image memory (557) when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0081] The output samples of the aggregator (555) can undergo various loop filtering techniques in the loop filter unit (556). The video compression technique may include an in-loop filtering technique controlled by parameters included in the encoded video sequence (also referred to as the encoded video bitstream), which are available as symbols (521) from the parser (520) to the loop filter unit (556). However, the video compression technique may also respond to metadata obtained during the decoding of a previous (in decoding order) portion of the encoded picture or encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0082] The loop filter unit (556) can output a sample stream, which can be output to the presentation device (512) and stored in the reference image memory (557) for future inter-frame image prediction.

[0083] Once fully reconstructed, some of the encoded images can be used as reference images for future predictions. For example, once the encoded image corresponding to the current image has been fully reconstructed and that encoded image (by, for example, the parser (520)) is identified as the reference image, the current image buffer (558) can become part of the reference image memory (557), and a new current image buffer can be reallocated before the reconstruction of subsequent encoded images begins.

[0084] The video decoder (510) can perform decoding operations according to a predetermined video compression technique as specified in a standard such as ITU-T Recommendation H.265. An encoded video sequence can conform to the syntax specified by the video compression technique or standard used, in the sense that the encoded video sequence follows the syntax of the video compression technique or standard and the profile documented in the video compression technique or standard. Specifically, the profile can select certain tools from all available tools in the video compression technique or standard as tools usable only under said profile. For compliance, the complexity of the encoded video sequence is also required to be within the range defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (measured in megasamples per second, for example), maximum reference picture size, etc. In some cases, the limitations set by the hierarchy can be further limited by the Hypothetical Reference Decoder (HRD) specification and metadata managed by the HRD buffer, which is signaled in the encoded video sequence.

[0085] In this implementation, the receiver (531) may receive supplemental (redundant) data along with the encoded video. The supplemental data may be included as part of one or more encoded video sequences. The supplemental data may be used by the video decoder (510) to properly decode the data and / or more accurately reconstruct the original video data. The supplemental data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0086] Figure 6 A block diagram of a video encoder (603) according to an embodiment of the present disclosure is shown. The video encoder (603) is included in an electronic device (620). The electronic device (620) includes a transmitter (640) (e.g., a transmission circuitry system). The video encoder (603) can be used in place of... Figure 4 The video encoder (403) in the example.

[0087] The video encoder (603) can obtain data from the video source (601) (the video source (601) is not...). Figure 6 In one example, a portion of the electronic device (620) receives video samples, and a video source (601) can capture one or more video images to be encoded by a video encoder (603). In another example, the video source (601) is a portion of the electronic device (620).

[0088] A video source (601) can provide a sequence of source video samples in the form of a digital video sample stream to be encoded by a video encoder (603). This digital video sample stream can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit…), any color space (e.g., BT.601YCrCB, RGB…), and any suitable sampling structure (e.g., YCrCb4:2:0, YCrCb4:4:4). In a media service system, the video source (601) can be a storage device storing previously prepared video. In a video conferencing system, the video source (601) can be a camera device capturing local image information as a video sequence. Video data can be provided as multiple individual pictures that are given motion when viewed sequentially. The pictures themselves can be organized as spatial pixel arrays, where each pixel can include one or more samples, depending on the sampling structure, color space, etc., used. Those skilled in the art will readily understand the relationship between pixels and samples. The following description focuses on samples.

[0089] According to the implementation, the video encoder (603) can encode and compress images of the source video sequence into an encoded video sequence (643) in real time or under any other time constraints required by the application. Implementing an appropriate encoding speed is a function of the controller (650). In some implementations, the controller (650) controls and is functionally coupled to other functional units as described below. For simplicity, the coupling is not depicted. Parameters set by the controller (650) may include rate control related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, group of images (GOP) layout, maximum motion vector search range, etc. The controller (650) can be configured to have other suitable functions belonging to the video encoder (603) optimized for certain system designs.

[0090] In some implementations, the video encoder (603) is configured to operate within an encoding loop. As a hypersimplified description, in this example, the encoding loop may include a source encoder (630) (e.g., responsible for creating symbols such as a symbol stream based on the input image to be encoded and one or more reference images) and a (local) decoder (633) embedded within the video encoder (603). The decoder (633) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data (since any compression between the symbols and the encoded video bitstream is lossless in the video compression techniques considered in the disclosed subject matter). The reconstructed sample stream (sample data) is input to a reference image memory (634). Since decoding of the symbol stream produces bit-accurate results regardless of the decoder's location (local or remote), the contents of the reference image memory (634) are also bit-accurate between the local and remote encoders. In other words, the reference image samples "seen" by the encoder's prediction portion are exactly the same sample values ​​that the decoder will "see" when using prediction during decoding. The basic principles of synchronization (and the drift that occurs when synchronization cannot be maintained, for example, due to channel errors) of this reference image are also used in some related fields.

[0091] The operation of the "local" decoder (633) can be combined with that already mentioned above. Figure 5 The operation of the "remote" decoder described in detail is the same as that of the video decoder (510). However, a brief reference is also provided. Figure 5 Since symbols are available and the entropy encoder (645) and parser (520) can encode / decode symbols into an encoded video sequence in a lossless manner, the entropy decoding portion of the video decoder (510), which includes the buffer memory (515) and the parser (520), can be implemented in the local decoder (633) without being fully integrated into the local decoder (633).

[0092] It can be observed that any decoder technique other than parsing / entropy decoding, which exists in the decoder, must also exist in the corresponding encoder in essentially the same functional form. For this reason, the disclosed subject focuses on decoder operation. Since the encoder technique is the inverse of the fully described decoder technique, the description of the encoder technique can be simplified. Only certain areas require more detailed description, which will be provided below.

[0093] During operation, in some examples, the source encoder (630) may perform motion-compensated predictive coding, which predictively codes the input image with reference to one or more previously encoded images designated as "reference images" from the video sequence. In this way, the encoding engine (632) encodes the differences between pixel blocks of the input image and pixel blocks of one or more reference images (which may be selected as prediction references to the input image).

[0094] The local video decoder (633) can decode encoded video data of a picture that can be designated as a reference picture based on symbols created by the source encoder (630). The operation of the encoding engine (632) can advantageously be a lossy process. When the encoded video data can be decoded by the video decoder (633), Figure 6 When decoded at (not shown), the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (633) replicates the decoding process performed by the video decoder on the reference image and can store the reconstructed reference image in the reference image memory (634). In this way, the video encoder (603) can locally store a copy of the reconstructed reference image that shares the same content (no transmission errors) as the reconstructed reference image to be obtained by the remote video decoder.

[0095] The predictor (635) can perform a prediction search against the encoding engine (632). That is, for a new image to be encoded, the predictor (635) can search in the reference image memory (634) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can be used as appropriate prediction references for the new image. The predictor (635) can operate pixel-by-pixel based on the sample blocks to find appropriate prediction references. In some cases, as determined by the search results obtained by the predictor (635), the input image may have prediction references obtained from multiple reference images stored in the reference image memory (634).

[0096] The controller (650) can manage the encoding operations of the source encoder (630), including, for example, setting parameters and subgroup parameters for encoding video data.

[0097] The outputs of all the functional units mentioned above can undergo entropy encoding in the entropy encoder (645). The entropy encoder (645) converts the symbols generated by the various functional units into an encoded video sequence by lossless compression of the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0098] The transmitter (640) can buffer one or more encoded video sequences created by the entropy encoder (645) in preparation for transmission via a communication channel (660), which may be a hardware / software link to a storage device storing the encoded video data. The transmitter (640) can combine the encoded video data from the video encoder (603) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (sources not shown).

[0099] The controller (650) can manage the operation of the video encoder (603). During encoding, the controller (650) can assign a specific encoded image type to each encoded image, which may affect the encoding techniques that can be applied to the corresponding image. For example, images can typically be assigned to one of the following image types:

[0100] An intra-frame picture (I-picture) can be a picture that is encoded and decoded without using any other pictures in the sequence as a prediction source. Some video codecs allow different types of intra-frame pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art will understand those variations of I-pictures and their corresponding applications and characteristics.

[0101] A predictive image (P-image) can be an image encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most one motion vector and a reference index to predict sample values ​​for each block.

[0102] Bidirectional predictive images (B-images) can be images encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most two motion vectors and a reference index to predict sample values ​​for each block. Similarly, multiple predictive images can use more than two reference images and associated metadata for the reconstruction of a single block.

[0103] Source images are typically spatially subdivided into multiple sample blocks (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples), and encoded block by block. Predictive coding can be performed on these blocks with reference to other (already encoded) blocks, determined by coding assignments applied to the corresponding images of the blocks. For example, blocks of image I can be non-predictively coded, or blocks of image I can be predictively coded (spatial or intra-frame prediction) with reference to already encoded blocks of the same image. Pixel blocks of image P can be predictively coded with reference to a previously encoded reference image via spatial or temporal prediction. Blocks of image B can be predictively coded with reference to one or two previously encoded reference images via spatial or temporal prediction.

[0104] The video encoder (603) can perform encoding operations according to a predetermined video coding technique or standard such as ITU-T H.265 Recommendation. In its operation, the video encoder (603) can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0105] In this implementation, the transmitter (640) may transmit additional data along with the encoded video. The source encoder (630) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant images and slices, SEI messages, VUI parameter set fragments, etc.

[0106] Video can be captured in a time series as multiple source images (video images). Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In the example, a specific image during encoding / decoding—referred to as the current image—is divided into blocks. When a block in the current image resembles a reference block in a previously encoded and still cached reference image in the video, the block in the current image can be encoded using a vector called a motion vector. The motion vector points to the reference block in the reference image, and in the case of multiple reference images, the motion vector can have a third dimension that identifies the reference image.

[0107] In some implementations, bidirectional prediction techniques can be used for inter-frame image prediction. According to bidirectional prediction, two reference images are used, such as a first reference image and a second reference image that both precede the current image in the video in decoding order (but may be past and future in display order, respectively). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. The block can be predicted using a combination of the first and second reference blocks.

[0108] In addition, merging mode techniques can be used for inter-frame image prediction to improve coding efficiency.

[0109] According to some embodiments of this disclosure, predictions such as inter-frame image prediction and intra-frame image prediction are performed on a block-by-block basis. For example, according to the HEVC standard, images in a video image sequence are divided into coding tree units (CTUs) for compression. The CTUs in the images have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU comprises three coding tree blocks (CTBs): one luma CTB and two chroma CTBs. Each CTU can be recursively partitioned into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be partitioned into one 64×64 pixel CU, four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In the example, each CU is analyzed to determine the prediction type used for the CU, such as inter-frame prediction or intra-frame prediction. Depending on temporal and / or spatial predictability, a CU is partitioned into one or more prediction units (PUs). Typically, each PU includes a luminance prediction block (PB) and two chrominance PBs. In implementations, prediction operations in decoding (encoding / decoding) are performed on a block-by-block basis. Using a luminance prediction block as an example, a prediction block comprises a matrix of pixel values ​​(e.g., luminance values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0110] Figure 7 A diagram of a video encoder (703) according to another embodiment of the present disclosure is shown. The video encoder (703) is configured to receive processing blocks (e.g., prediction blocks) of sample values ​​within a current video image in a video image sequence, and to encode the processing blocks into an encoded image that is part of an encoded video sequence. In the example, the video encoder (703) is used instead of Figure 4 The video encoder (403) in the example.

[0111] In the HEVC example, the video encoder (703) receives a matrix of sample values ​​for processing blocks, such as prediction blocks of 8×8 samples. The video encoder (703) uses, for example, rate-distortion optimization to determine whether to best encode the processing block using intra-frame mode, inter-frame mode, or bidirectional prediction mode. When encoding the processing block in intra-frame mode, the video encoder (703) can encode the processing block into the encoded picture using intra-frame prediction techniques; while when encoding the processing block in inter-frame mode or bidirectional prediction mode, the video encoder (703) can encode the processing block into the encoded picture using inter-frame prediction or bidirectional prediction techniques, respectively. In some video coding techniques, the merging mode can be an inter-frame picture prediction sub-mode, in which motion vectors are derived from the predicted vectors without the aid of one or more externally encoded motion vector components. In some other video coding techniques, motion vector components applicable to the subject block may exist. In the example, the video encoder (703) includes other components, such as a mode decision module (not shown) for determining the mode of the processing block.

[0112] exist Figure 7 In the example, the video encoder (703) includes, as follows: Figure 7 The inter-frame encoder (730), intra-frame encoder (722), residual calculator (723), switch (726), residual encoder (724), general controller (721) and entropy encoder (725) are shown coupled together.

[0113] The inter-frame encoder (730) is configured to receive samples of the current block (e.g., the processing block), compare the block with one or more reference blocks in a reference image (e.g., blocks in previous and subsequent images), generate inter-frame prediction information (e.g., redundancy descriptions based on inter-frame coding techniques, motion vectors, merging mode information), and compute inter-frame prediction results (e.g., predicted blocks) based on the inter-frame prediction information using any suitable technique. In some examples, the reference image is a decoded reference image based on encoded video information.

[0114] The intra encoder (722) is configured to receive samples of the current block (e.g., the processing block), in some cases compare the block with already encoded blocks in the same image, generate quantization coefficients after transformation, and in some cases also generate intra prediction information (e.g., intra prediction direction information based on one or more intra coding techniques). In the example, the intra encoder (722) also calculates intra prediction results (e.g., predicted blocks) based on the intra prediction information and a reference block in the same image.

[0115] A general-purpose controller (721) is configured to determine general-purpose control data and, based on this data, control other components of the video encoder (703). In this example, the general-purpose controller (721) determines the mode of the block and, based on this mode, provides control signals to the switch (726). For example, when the mode is intra-frame mode, the general-purpose controller (721) controls the switch (726) to select the intra-frame mode result for use by the residual calculator (723) and controls the entropy encoder (725) to select intra-frame prediction information and include it in the bitstream; and when the mode is inter-frame mode, the general-purpose controller (721) controls the switch (726) to select the inter-frame prediction result for use by the residual calculator (723) and controls the entropy encoder (725) to select inter-frame prediction information and include it in the bitstream.

[0116] A residual calculator (723) is configured to calculate the difference (residual data) between the received block and the prediction result selected from an intra encoder (722) or an inter encoder (730). A residual encoder (724) is configured to operate based on the residual data to encode the residual data to generate transform coefficients. In an example, the residual encoder (724) is configured to transform the residual data from the spatial domain to the frequency domain and generate transform coefficients. The transform coefficients are then quantized to obtain quantized transform coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data can be appropriately used by the intra encoder (722) and the inter encoder (730). For example, the inter encoder (730) can generate a decoded block based on the decoded residual data and inter-frame prediction information, and the intra encoder (722) can generate a decoded block based on the decoded residual data and intra-frame prediction information. Decoded blocks are processed appropriately to generate decoded images, and in some examples these decoded images can be cached in memory circuitry (not shown) and used as reference images.

[0117] An entropy encoder (725) is configured to format the bitstream to include encoded blocks. The entropy encoder (725) is configured to include various information according to a suitable standard such as the HEVC standard. In this example, the entropy encoder (725) is configured to include general control data, selected prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information in the bitstream. Note that, according to the disclosed subject matter, residual information is not present when blocks are encoded in inter-frame mode or in a merged sub-mode of dual prediction mode.

[0118] Figure 8A diagram of a video decoder (810) according to another embodiment of the present disclosure is shown. The video decoder (810) is configured to receive an encoded image as part of an encoded video sequence and decode the encoded image to generate a reconstructed image. In this example, the video decoder (810) is used instead of Figure 4 The video decoder (410) in the example.

[0119] exist Figure 8 In the example, the video decoder (810) includes, for example, Figure 8 The entropy decoder (871), inter-frame decoder (880), residual decoder (873), reconstruction module (874), and intra-frame decoder (872) are shown coupled together.

[0120] The entropy decoder (871) can be configured to reconstruct certain symbols representing the syntax elements constituting the encoded image from the encoded image. Such symbols may include, for example, the mode encoding the block (e.g., intra-frame mode, inter-frame mode, dual prediction mode, a combined sub-mode of the latter two, or another sub-mode), prediction information (such as, for example, intra-frame prediction information or inter-frame prediction information) that can respectively identify certain samples or metadata used for prediction by the intra-frame decoder (872) or the inter-frame decoder (880), residual information in the form of, for example, quantized transform coefficients, etc. In the example, when the prediction mode is inter-frame mode or dual prediction mode, the inter-frame prediction information is provided to the inter-frame decoder (880); and when the prediction type is intra-frame prediction type, the intra-frame prediction information is provided to the intra-frame decoder (872). The residual information may be inversely quantized and provided to the residual decoder (873).

[0121] The inter-frame decoder (880) is configured to receive inter-frame prediction information and generate inter-frame prediction results based on the inter-frame prediction information.

[0122] The intra-frame decoder (872) is configured to receive intra-frame prediction information and generate prediction results based on the intra-frame prediction information.

[0123] The residual decoder (873) is configured to perform inverse quantization to extract the dequantized transform coefficients and process the dequantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (873) may also require some control information (including quantizer parameters (QP)), and this information can be provided by the entropy decoder (871) (the data path is not depicted because this is only low-level control information).

[0124] The reconstruction module (874) is configured to combine the residual output by the residual decoder (873) with the prediction result (output by the inter-frame prediction module or the intra-frame prediction module, as appropriate) in the spatial domain to form a reconstructed block, which can be a part of a reconstructed image, which in turn can be a part of a reconstructed video. Note that other suitable operations, such as deblocking, can be performed to improve visual quality.

[0125] Note that any suitable technology can be used to implement the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810). In one implementation, one or more integrated circuits can be used to implement the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810). In another implementation, one or more processors that execute software instructions can be used to implement the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810).

[0126] This disclosure provides various aspects of filtering techniques for video encoding / decoding.

[0127] The encoder / decoder can apply an adaptive loop filter (ALF) with block-based filter adaptation to reduce artifacts. For the luma component, one of several filters (e.g., 25 filters) can be selected for a 4×4 luma block, for example, based on the direction and activity of the local gradient.

[0128] ALFs can have any suitable shape and size. (See reference) Figure 9 ALF(910) to (911) have a rhombus shape, such as a 5×5 rhombus for ALF(910) and a 7×7 rhombus for ALF(911). In ALF(910), elements (920) to (932) form a rhombus shape and can be used in the filtering process. Seven values ​​(e.g., C0 to C6) can be used for elements (920) to (932). In ALF(911), elements (940) to (964) form a rhombus shape and can be used in the filtering process. Thirteen values ​​(e.g., C0 to C12) can be used for elements (940) to (964).

[0129] Reference Figure 9In some examples, two ALFs (910) to (911) with diamond filter shapes are used. A 5×5 diamond-shaped filter (910) can be applied to the chroma component (e.g., chroma block, chroma CB), while a 7×7 diamond-shaped filter (911) can be applied to the luma component (e.g., luma block, luma CB). Other suitable shapes and sizes can be used in the ALF. For example, a 9×9 diamond-shaped filter can be used.

[0130] The filter coefficients at the locations indicated by values ​​(e.g., C0 to C6 in (910) or C0 to C12 in (920)) can be nonzero. Furthermore, when the ALF includes a clipping function, the clipping value at those locations can be nonzero.

[0131] For block classification of the luminance component, a 4×4 block (or luminance block, luminance CB) can be classified or categorized into one of several (e.g., 25) classes. The classification index C can be based on the quantized values ​​of the directional parameter D and the activity value A. The result is obtained using equation (1).

[0132]

[0133] To calculate the directional parameter D and the quantization value The gradient g in the vertical, horizontal, and two diagonal directions (e.g., d1 and d2) v g h g d1 and g d2 The following 1-dimensional Laplace can be used for calculation.

[0134]

[0135]

[0136]

[0137]

[0138] Here, indices i and j refer to the coordinates of the top-left sample within the 4×4 block, and R(k,l) indicates the reconstructed sample at coordinates (k,l). Directions (e.g., d1 and d2) can refer to the two diagonal directions.

[0139] To reduce the complexity of the block classification described above, a 1-dimensional Laplace calculation based on subsampling can be applied. Figures 10A to 10D The following diagrams show the methods for calculating vertical ( Figure 10A ),level( Figure 10B ) and the two diagonal directions d1( Figure 10C ) and d2( Figure 10D The gradient g of )v g h g d1 and g d2 Examples of subsampling positions. The same subsampling position can be used for gradient calculations in different directions. Figure 10A In the text, the label "V" indicates the symbol used to calculate the vertical gradient g. v The subsampling location. Figure 10B In the text, the label "H" indicates the symbol used to calculate the horizontal gradient g. h The subsampling location. Figure 10C In the text, the label "D1" indicates the value used to calculate the diagonal gradient g of d1. d1 The subsampling location. Figure 10D In the text, the label "D2" indicates the value used to calculate the diagonal gradient g of d2. d2 The sub-sampling position.

[0140] The gradients g in the horizontal and vertical directions v and g h maximum value and minimum value It can be set as:

[0141]

[0142] gradient g in the two diagonal directions d1 and g d2 maximum value and minimum value It can be set to:

[0143]

[0144] The directional parameter D can be derived based on the above value and the following two thresholds t1 and t2.

[0145] Step 1. If (1) and (2) If true, then set D to 0.

[0146] Step 2. If If yes, proceed to step 3; otherwise, proceed to step 4.

[0147] Step 3. If If so, set D to 2; otherwise, set D to 1.

[0148] Step 4. If If so, set D to 4; otherwise, set D to 3.

[0149] Activity value A can be calculated as:

[0150]

[0151] A can be further quantized to a range of 0 to 4 (inclusive), and the quantized value is represented as

[0152] For the chromaticity components in the image, block classification is not applied, and therefore a single set of ALF coefficients can be applied to each chromaticity component.

[0153] Geometric transformations can be applied to filter coefficients and the corresponding filter limiting values ​​(also known as limiting values). Before filtering a block (e.g., a 4×4 brightness block), the limiting factor depends on, for example, the gradient value calculated for the block (e.g., g). v g h g D1 and / or g d2 Geometric transformations, such as rotation or diagonal and vertical flips, can be applied to the filter coefficients f(k, l) and the corresponding filter threshold c(k, l). Applying a geometric transformation to the filter coefficients f(k, l) and the corresponding filter threshold c(k, l) is equivalent to applying the geometric transformation to samples within the region supported by the filter. By aligning their respective orientations, geometric transformations can make different blocks applying ALF more similar.

[0154] The three geometric transformations, including diagonal flip, vertical flip, and rotation, can be performed as described in equations (9) to (11), respectively.

[0155] f D (k, l) = f(l, k), c D (k, l) = c(l, k), Equation (9)

[0156] f V (k, l) = f(k, Kl-1), c V (k, l) = c(k, Kl-1) Equation (10)

[0157] f R (k, l) = f(Kl-1, k), c R (k, l) = c(Kl-1, k) Equation (11)

[0158] Where K is the size of the ALF or filter, and 0 ≤ k, 1 ≤ K⁻¹ are the coordinates of the coefficients. For example, position (0, 0) is at the top left corner, while position (K⁻¹, K⁻¹) is at the bottom right corner of the filter f or the limiting matrix (or limiting matrix) c. Depending on the gradient values ​​computed for the block, transformations can be applied to the filter coefficients f(k, l) and the limiting value c(k, l). Table 1 summarizes examples of the relationship between the transformations and the four gradients.

[0159] Table 1: Mapping of gradients and transformations for block computation

[0160]

[0161]

[0162] In some implementations, ALF filter parameters are signaled in the Adaptation Parameter Set (APS) for the image. In the APS, one or more sets (e.g., up to 25 sets) of luminance filter coefficients and limiting indexes can be signaled. In an example, one of the sets may include luminance filter coefficients and one or more limiting indexes. One or more sets (e.g., up to 8 sets) of chrominance filter coefficients and limiting indexes can be signaled. To reduce signaling overhead, filter coefficients for different classifications of the luminance components (e.g., with different classification indices) can be merged. In the slice header, the index of the APS used for the current slice can be signaled.

[0163] In an implementation, the limiting index (also called the limiting index) can be decoded according to the APS. The limiting index can be used, for example, to determine the corresponding limiting value based on the relationship between the limiting index and the corresponding limiting value. This relationship can be predefined and stored in the decoder. In an example, this relationship is described by tables such as a luminance table (e.g., for luminance CB) of limiting index and corresponding limiting value, and a chrominance table (e.g., for chrominance CB) of limiting index and corresponding limiting value. The limiting value can depend on the bit depth B. The bit depth B can refer to the internal bit depth, the bit depth of the reconstructed sample in the CB to be filtered, etc. In some examples, Equation (12) is used to obtain the tables (e.g., luminance table, chrominance table).

[0164]

[0165] Where AlfClip is the clipping value, B is the bit depth (e.g., bitDepth), N (e.g., N=4) is the number of allowed clipping values, and (n-1) is the clipping value index (also called the clipping index or clipIdx). Table 2 shows an example of a table obtained using equation (12), where N=4. The clipping index (n-1) can be 0, 1, 2, and 3 in Table 2, and n can be 1, 2, 3, and 4, respectively. Table 2 can be used for luma blocks or chroma blocks.

[0166] Table 2 – AlfClip can depend on bit depth B and clipIdx

[0167]

[0168]

[0169] In the slice header for the current slice, one or more APS indices (e.g., up to seven APS indices) can be signaled to specify the luma filter banks that can be used for the current slice. The filtering process can be controlled at one or more suitable levels such as picture level, slice level, CTB level, etc. In an implementation, the filtering process can be further controlled at the CTB level. Signaled flags can be used to indicate whether an ALF is applied to the luma CTB. The luma CTB can select a filter bank from multiple fixed filter banks (e.g., 16 fixed filter banks) and one or more filter banks signaled in the APS (also referred to as one or more signaled filter banks). Filter bank indices can be signaled to the luma CTB to indicate the filter banks to be applied (e.g., filter banks among multiple fixed filter banks and one or more signaled filter banks). Multiple fixed filter banks can be predefined and hard-coded in the encoder and decoder and can be referred to as predefined filter banks.

[0170] For chroma components, the APS index can be signaled in the slice header to indicate the chroma filter bank to be used for the current slice. At the CTB level, if there is more than one chroma filter bank in the APS, the filter bank index can be signaled for each chroma CTB.

[0171] Filter coefficients can be quantized using a norm equal to 128. To reduce multiplication complexity, bitstream consistency can be applied so that coefficient values ​​at non-center locations can be in the range of -27 to 27-1 (inclusive). In the example, the coefficients at the center location are not signaled in the bitstream and can be considered equal to 128.

[0172] In some implementations, the syntax and semantics of the limiting index and the limiting value are defined as follows:

[0173] `alf_luma_clip_idx[sfIdx][j]` can be used to specify the limiting index of the limiting value to be used before multiplying by the j-th coefficient of the luminance filter indicated by `sfIdx`. Bitstream consistency requirements may include that the value of `alf_luma_clip_idx[sfIdx][j]`, where `sfIdx` = 0 to `alf_luma_num_filters_signalled_minus1` and `j` = 0 to 11, should be in the range of 0 to 3 (inclusive).

[0174] The luminance filter limiting value AlfClipL[adaptation_parameter_set_id], which has elements where filtIdx = 0 to NumAlfFilters-1 and j = 0 to 11, can be obtained according to the specifications in Table 2.

[0175] `alf_chroma_clip_idx[altIdx][j]` can be used to specify the limiting index to be used before multiplying the j-th coefficient of the alternative chroma filter with index `altIdx`. Bitstream consistency requirements may include that the value of `alf_chroma_clip_idx[altIdx][j]`, where `altIdx` = 0 to `alf_chroma_num_alt_filters_minus1` and `j` = 0 to 5, should be in the range of 0 to 3 (inclusive).

[0176] Based on bitDepth set to BitDepthC and clipIdx set to alf_chroma_clip_idx[altIdx][j], the chroma filter limiting value AlfClipC[adaptation_parameter_set_id][altIdx][j] with altIdx = 0 to alf_chroma_num_alt_filters_minus1 and j = 0 to 5 can be obtained according to the specifications in Table 2.

[0177] In the implementation, the filtering process can be described as follows. At the decoder end, when ALF is enabled for CTB, the samples R(i,j) in the CU (or CB) can be filtered to produce the filtered sample value R'(i,j) as shown below using equation (13). In the example, each sample in the CU is filtered.

[0178]

[0179] Here, f(k,l) represents the decoded filter coefficients, K(x,y) is the clipping function, and c(k,l) represents the decoded clipping parameter (or clipping value). The variables k and l can vary between -L / 2 and L / 2, where L represents the filter length. The clipping function K(x,y) = min(y,max(-y,x)) corresponds to the clipping function Clip3(-y,y,x). By incorporating the clipping function K(x,y), the loop filtering method (e.g., ALF) becomes a nonlinear process and can be called nonlinear ALF.

[0180] In nonlinear ALF, multiple sets of limiting values ​​can be provided as shown in Table 3. In the example, the luma group includes four limiting values ​​{1024, 181, 32, 6}, and the chroma group includes four limiting values ​​{1024, 161, 25, 4}. The four limiting values ​​in the luma group can be selected by approximately equally dividing the full range (e.g., 1024) of sample values ​​for the luma block in the logarithmic domain (encoded in 10 bits). The range for the chroma group can be from 4 to 1024.

[0181] Table 3 – Examples of Amplitude Limits

[0182]

[0183] The selected limiting value can be encoded in the "alf_data" syntax element as follows: A suitable encoding scheme (e.g., the Columbus encoding scheme) can be used to encode the limiting index corresponding to the selected limiting value, such as that shown in Table 3. The encoding scheme can be the same as that used to encode the filter bank index.

[0184] In this implementation, a virtual boundary filtering process can be used to reduce the row buffer requirements of the ALF. Therefore, modified block classification and filtering can be applied to samples near the CTU boundary (e.g., the horizontal CTU boundary). The virtual boundary (1130) can be defined by shifting the horizontal CTU boundary (1120) by "N". 样本 "A line formed from a sample, such as" Figure 11A As shown, where N 样本 It can be a positive integer. In the example, for the luminance component, N 样本 It equals 4, while for the chromaticity component, N 样本 It equals 2.

[0185] Reference Figure 11A Modified block classification can be applied to the luminance components. In the example, for the calculation of the 1D Laplacian gradient of the 4×4 block (1110) above the virtual boundary (1130), only samples above the virtual boundary (1130) are used. Similarly, refer to Figure 11BFor the 1D Laplace gradient calculation of the 4×4 block (1111) below the virtual boundary (1131) shifted from the CTU boundary (1121), only the samples below the virtual boundary (1131) are used. The quantization of the active value A can be scaled accordingly by considering reducing the number of samples used in the 1D Laplace gradient calculation.

[0186] For filtering, the symmetrical fill operation at the virtual boundary can be used for both the luminance and chrominance components. Figures 12A to 12F An example of this modified ALF filtering for the luminance component at a virtual boundary is shown. When the filtered sample is below the virtual boundary, adjacent samples above the virtual boundary are filled. When the filtered sample is above the virtual boundary, adjacent samples below the virtual boundary are filled. (See reference...) Figure 12A Adjacent sample C0 can be filled with sample C2 located below the virtual boundary (1210). (See reference...) Figure 12B Adjacent sample C0 can be filled with sample C2 located above the virtual boundary (1220). (See reference...) Figure 12C Adjacent samples C1 to C3 can be filled with samples C5 to C7 located below the virtual boundary (1230), respectively. (See reference...) Figure 12D Adjacent samples C1 to C3 can be filled with samples C5 to C7 located above the virtual boundary (1240), respectively. (See reference...) Figure 12E Adjacent samples C4 to C8 can be filled with samples C10, C11, C12, C11, and C10 located below the virtual boundary (1250), respectively. (See reference...) Figure 12F Adjacent samples C4 to C8 can be filled with samples C10, C11, C12, C11 and C10 located above the virtual boundary (1260), respectively.

[0187] In some examples, the above description may be adjusted appropriately when (one or more) samples and (one or more) adjacent samples are located to the left (or right) and right (or left) of the virtual boundary.

[0188] According to aspects of this disclosure, to improve coding efficiency, images can be partitioned based on a filtering process. In some examples, the CTU is also referred to as the largest coding unit (LCU). In examples, the CTU or LCU can have a size of 64×64 pixels. In some implementations, LCU-aligned image quadtree segmentation can be used for filtering-based partitioning. In some examples, an adaptive loop filter based on the coding unit-synchronized image quadtree can be used. For example, a luminance image can be partitioned into several multi-level quadtree partitions, with each partition boundary aligned with the boundary of the LCU. Each partition has its own filtering process and is therefore referred to as a filter unit (FU).

[0189] In some examples, a two-pass encoding process can be used. In the first pass of the two-pass encoding process, the quadtree segmentation pattern of the image and the optimal filter for each function element (FU) can be determined. In some implementations, the quadtree segmentation pattern of the image and the optimal filter for each FU are determined based on filtering distortion. During the determination process, filtering distortion can be estimated using a fast filtering distortion estimation (FFDE) technique. The image is partitioned using quadtree partitioning. Based on the determined quadtree segmentation pattern and the selected filters for all FUs, the reconstructed image can be filtered.

[0190] In the second pass of the two-pass encoding process, CU synchronous ALF on / off control is executed. Based on the ALF on / off result, the filtered first image is partially recovered from the reconstructed image.

[0191] Specifically, in some examples, a top-down segmentation strategy is employed to divide the image into multi-level quadtree partitions using a rate-distortion criterion. Each partition is called a filter unit (FU). The segmentation process aligns the quadtree partitions with the LCU boundaries. The encoding order of the FUs follows the z-scan order.

[0192] Figure 13 Examples of partitioning according to some embodiments of this disclosure are shown. Figure 13 In the image (1300), the image is divided into 10 FUs, and the encoding order is FU0, FU1, FU2, FU3, FU4, FU5, FU6, FU7, FU8 and FU9.

[0193] Figure 14 The quadtree segmentation pattern (1400) for image (1300) is shown. Figure 14In the examples, a partition flag can be used to indicate the image partitioning pattern. For example, "1" indicates a quadtree partitioning of the block; while "0" indicates no further partitioning of the block. In some examples, the minimum FU has an LCU size, and the minimum FU does not require a partition flag. The partition flag is encoded and transmitted in z-order, such as... Figure 14 As shown.

[0194] In some examples, the filter for each FU is selected from two filter sets based on a rate-distortion criterion. The first set contains filters of 1 / 2 symmetric square and diagonal square shapes derived for the current FU. The second set comes from a delay filter buffer; the delay filter buffer stores filters previously derived for FUs of an existing image. The filter with the minimum rate-distortion cost from these two sets can be selected for the current FU. Similarly, if the current FU is not the minimum FU and can be further segmented into 4 sub-FUs, the rate-distortion cost of the 4 sub-FUs is calculated. By recursively comparing the rate-distortion costs of the segmented and unsegmented cases, the quadtree segmentation mode for the image can be determined.

[0195] In some examples, the maximum number of Function Elements (FUs) can be limited using the maximum quadtree split level. In this example, when the maximum quadtree split level is 2, the maximum number of FUs is 16. Furthermore, during quadtree split determination, the correlation values ​​of the Wiener coefficients used to derive the 16 FUs (minimum FUs) at the bottom quadtree level can be reused. The remaining FUs can then have their Wiener filters derived from the correlations of the 16 FUs at the bottom quadtree level. Therefore, in this example, only one frame buffer access is performed to derive the filter coefficients for all FUs.

[0196] After determining the quadtree partitioning mode, to further reduce filtering distortion, CU-synchronized ALF on / off control can be implemented. By comparing the filtering distortion and non-filtering distortion at each leaf CU, the leaf CU can explicitly turn ALF on / off within its local region. In some examples, coding efficiency can be further improved by redesigning the filter coefficients based on the ALF on / off results.

[0197] Cross-component filtering can be applied to a cross-component filter, such as a cross-component adaptive loop filter (CC-ALF). A cross-component filter can refine a chromaticity component (e.g., the chromaticity CB corresponding to the luminance CB) using luminance sample values ​​of the luminance component (e.g., luminance CB). In the example, both the luminance CB and chromaticity CB are included in the CU.

[0198] Figure 15A cross-component filter (e.g., CC-ALF) for generating chromaticity components is illustrated according to embodiments of this disclosure. In some examples, Figure 15 The filtering process for a first chromaticity component (e.g., first chromaticity CB), a second chromaticity component (e.g., second chromaticity CB), and a luminance component (e.g., luminance CB) is illustrated. The luminance component can be filtered by a sample adaptive offset (SAO) filter (1510) to generate a SAO-filtered luminance component (1541). The SAO-filtered luminance component (1541) can be further filtered by an ALF luminance filter (1516) to become a filtered luminance CB (1561) (e.g., "Y").

[0199] The first chromaticity component can be filtered by a SAO filter (1512) and an ALF chromaticity filter (1518) to generate a first intermediate component (1552). Furthermore, the SAO-filtered luminance component (1541) can be filtered by a cross-component filter (e.g., CC-ALF) (1521) for the first chromaticity component to generate a second intermediate component (1542). Subsequently, a filtered first chromaticity component (1562) (e.g., 'Cb') can be generated based on at least one of the second intermediate component (1542) and the first intermediate component (1552). In the example, the filtered first chromaticity component (1562) (e.g., 'Cb') can be generated by combining the second intermediate component (1542) and the first intermediate component (1552) with an adder (1522). The cross-component adaptive loop filtering process for the first chromaticity component can include steps performed by the CC-ALF (1521) and steps performed by, for example, the adder (1522).

[0200] The above description can be adapted to the second chromaticity component. The second chromaticity component can be filtered by a SAO filter (1514) and an ALF chromaticity filter (1518) to generate a third intermediate component (1553). Furthermore, the SAO-filtered luminance component (1541) can be filtered by a cross-component filter (e.g., CC-ALF) (1531) for the second chromaticity component to generate a fourth intermediate component (1543). Subsequently, a filtered second chromaticity component (1563) (e.g., 'Cr') can be generated based on at least one of the fourth intermediate component (1543) and the third intermediate component (1553). In the example, the filtered second chromaticity component (1563) (e.g., 'Cr') can be generated by combining the fourth intermediate component (1543) and the third intermediate component (1553) with an adder (1532). In the example, the cross-component adaptive loop filtering process for the second chromaticity component can include steps performed by the CC-ALF (1531) and steps performed by, for example, the adder (1532).

[0201] Cross-component filters (e.g., CC-ALF(1521), CC-ALF(1531)) can be operated by applying a linear filter with any suitable filter shape to the luminance component (or luminance channel) to refine each chrominance component (e.g., first chrominance component, second chrominance component).

[0202] Figure 16 An example of a filter (1600) according to an embodiment of the present disclosure is shown. The filter (1600) may include non-zero filter coefficients and zero filter coefficients. The filter (1600) has a rhombus shape (1620) formed by filter coefficients (1610) (indicated by circles with black fill). In the example, the non-zero filter coefficients in the filter (1600) are included in the filter coefficients (1610), and the filter coefficients not included in the filter coefficients (1610) are zero. Therefore, the non-zero filter coefficients in the filter (1600) are included in the rhombus shape (1620), and the filter coefficients not included in the rhombus shape (1620) are zero. In the example, the number of filter coefficients of the filter (1600) is equal to the number of filter coefficients (1610), which is... Figure 16 The example shown is 18.

[0203] CC-ALF can include any suitable filter coefficients (also known as CC-ALF filter coefficients). Return to reference Figure 15 CC-ALF(1521) and CC-ALF(1531) can have the same filter shape, such as Figure 16The diamond shape (1620) and the same number of filter coefficients are shown. In the example, the values ​​of the filter coefficients in CC-ALF (1521) are different from the values ​​of the filter coefficients in CC-ALF (1531).

[0204] Typically, filter coefficients in CC-ALF (e.g., non-zero filter coefficients) can be transferred, for example, in APS. In the example, filter coefficients can be factored (e.g., 2). 10 Scaling is performed, and rounding can be done for fixed-point representations. CC-ALF application can be controlled on variable block sizes and is signaled via context-coded flags received for each sample block (e.g., a CC-ALF enable flag). Context-coded flags such as the CC-ALF enable flag can be signaled at any suitable level, such as the block level. For each chroma component, the block size and the CC-ALF enable flag can be received at the slice level. In some examples, block sizes of 16×16, 32×32, and 64×64 (in chroma samples) can be supported.

[0205] Figure 17 Examples of syntax for CC-ALF according to some embodiments of this disclosure are shown. Figure 17 In the example, alf_ctb_cross_component_cb_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] is an index used to indicate whether a cross component Cb filter is used, and if so, the index of the cross component Cb filter. For example, when alf_ctb_cross_component_cb_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] equals 0, the cross-component Cb filter is not applicable to the Cb color component sample block at the luminance position (xCtb, yCtb); when alf_ctb_cross_component_cb_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] is not equal to 0, alf_ctb_cross_component_cb_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] is an index for the filter to be applied. For example, the alf_ctb_cross_component_cb_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] cross component Cb filter is applied to the Cb color component sample block at the luminance position (xCtb, yCtb).

[0206] In addition, Figure 17 In the example, alf_ctb_cross_component_cr_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] is used to indicate whether a cross component Cr filter is used, and if so, the index of the cross component Cr filter. For example, when alf_ctb_cross_component_cr_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] equals 0, the cross-component Cr filter is not applicable to the Cr color component sample block at the luminance position (xCtb, yCtb); when alf_ctb_cross_component_cr_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] is not equal to 0, alf_ctb_cross_component_cr_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] is the index of the cross-component Cr filter. For example, the alf_cross_component_cr_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] cross component Cr filter can be applied to the Cr color component sample block at the luminance position (xCtb, yCtb).

[0207] In some examples, chroma subsampling is used, so the number of samples in each of the chroma blocks(s) can be less than the number of samples in the luma blocks. The chroma subsampling format (also known as the chroma subsampling format specified, for example, by chroma_format_idc) can indicate the chroma horizontal subsampling factor (e.g., SubWidthC) and chroma vertical subsampling factor (e.g., SubHeightC) between each of the chroma blocks(s) and the corresponding luma block. In the example, the chroma subsampling format is 4:2:0, and therefore the chroma horizontal subsampling factor (e.g., SubWidthC) and the chroma vertical subsampling factor (e.g., SubHeightC) are both 2, as shown below. Figures 18A to 18BAs shown. In the example, the chroma subsampling format is 4:2:2, and therefore the chroma horizontal subsampling factor (e.g., SubWidthC) is 2, and the chroma vertical subsampling factor (e.g., SubHeightC) is 1. In the example, the chroma subsampling format is 4:4:4, and therefore the chroma horizontal subsampling factor (e.g., SubWidthC) and the chroma vertical subsampling factor (e.g., SubHeightC) are both 1. The chroma sample type (also known as the chroma sample position) can indicate the relative position of a chroma sample in a chroma block with respect to at least one corresponding luminance sample in a luminance block.

[0208] Figures 18A to 18B An exemplary position of a chromaticity sample relative to a luminance sample according to an embodiment of this disclosure is shown. (Refer to...) Figure 18A The brightness sample (1801) is located in rows (1811) to (1818). Figure 18A The luminance sample (1801) shown can represent a portion of the image. In the example, a luminance block (e.g., luminance CB) includes luminance sample (1801). A luminance block can correspond to two chroma blocks with a chroma subsampling format of 4:2:0. In the example, each chroma block includes chroma sample (1803). Each chroma sample (e.g., chroma sample (1803(1))) corresponds to four luminance samples (e.g., luminance samples (1801(1)) to (1801(4))). In the example, the four luminance samples are the top left sample (1801(1)), the top right sample (1801((2)), the bottom left sample (1801(3)), and the bottom right sample (1801(4)). The chroma sample (e.g., (1803(1))) is located between the top left sample (1801(1)) and the bottom left sample (1801(4)). The chromaticity sample type of the chromaticity block with chromaticity sample (1803) located at the left center position between (1801(1)) can be called chromaticity sample type 0. Chromaticity sample type 0 indicates the relative position 0 between the left center position corresponding to the middle of the upper left sample (1801(1)) and the lower left sample (1801(3)). These four luminance samples (e.g., (1801(1)) to (1801(4))) can be called adjacent luminance samples of chromaticity sample (1803)(1).

[0209] In the example, each chroma block includes a chroma sample (1804). The above description referring to chroma sample (1803) can be applied to chroma sample (1804), and therefore, for the sake of brevity, a detailed description can be omitted. Each of the chroma samples (1804) may be located at the center of four corresponding luminance samples, and the chroma sample type of a chroma block having chroma samples (1804) may be referred to as chroma sample type 1. Chroma sample type 1 indicates the relative position 1 corresponding to the center of the four luminance samples (e.g., (1801(1)) to (1801(4))). For example, one of the chroma samples (1804) may be located at the center portion of luminance samples (1801(1)) to (1801(4)).

[0210] In the example, each chroma block includes a chroma sample (1805). Each of the chroma samples (1805) may be located at an upper-left position co-located with the upper-left samples of the four corresponding luminance samples (1801), and the chroma sample type of the chroma block having chroma samples (1805) may be called chroma sample type 2. Thus, each of the chroma samples (1805) is co-located with the upper-left samples of the four luminance samples (1801) corresponding to the corresponding chroma sample. Chroma sample type 2 indicates the relative position 2 corresponding to the upper-left position of the four luminance samples (1801). For example, one of the chroma samples (1805) may be located at the upper-left position of luminance samples (1801(1)) to (1801(4)).

[0211] In the example, each chroma block includes a chroma sample (1806). Each of the chroma samples (1806) may be located at the top center position between the corresponding top-left sample and the corresponding top-right sample, and the chroma sample type of the chroma block having chroma samples (1806) may be referred to as chroma sample type 3. Chroma sample type 3 indicates the relative position 3 corresponding to the top center position between the top-left sample and the top-right sample. For example, one of the chroma samples (1806) may be located at the top center position of the luminance samples (1801(1)) to (1801(4)).

[0212] In the example, each chroma block includes a chroma sample (1807). Each of the chroma samples (1807) may be located at a lower-left position co-located with the lower-left samples of the four corresponding luminance samples (1801), and the chroma sample type of the chroma block having chroma samples (1807) may be called chroma sample type 4. Thus, each of the chroma samples (1807) is co-located with the lower-left samples of the four luminance samples (1801) corresponding to the corresponding chroma sample. Chroma sample type 4 indicates the relative position 4 corresponding to the lower-left position of the four luminance samples (1801). For example, one of the chroma samples (1807) may be located at the lower-left position of luminance samples (1801(1)) to (1801(4)).

[0213] In the example, each chroma block includes a chroma sample (1808). Each of the chroma samples (1808) is located at the bottom center position between the bottom left sample and the bottom right sample, and the chroma sample type of the chroma block having chroma samples (1808) can be called chroma sample type 5. Chroma sample type 5 indicates the relative position 5 of the bottom center position between the bottom left and bottom right samples corresponding to the four luminance samples (1801). For example, one of the chroma samples (1808) can be located between the bottom left and bottom right samples of luminance samples (1801(1)) to (1801(4)).

[0214] Generally, any suitable chroma sample type can be used for a chroma subsampling format. Chroma sample types 0 through 5 are exemplary chroma sample types described using the chroma subsampling format 4:2:0. Additional chroma sample types can be used for the chroma subsampling format 4:2:0. Furthermore, other chroma sample types and / or variations of chroma sample types 0 through 5 can be used for other chroma subsampling formats, such as 4:2:2, 4:4:4, etc. In the example, a chroma sample type combining chroma samples (1805) and (1807) is used for the chroma subsampling format 4:2:2.

[0215] In the example, the luminance blocks are considered to have alternating rows, such as rows (1811) to (1812) which respectively include the top two samples (e.g., (1801(1)) to (1801(4))) of four luminance samples (e.g., (1801(1)) to (1801(4))) and the bottom two samples (e.g., (1801(3)) to (1801(4))) of four luminance samples (e.g., (1801(1)) to (1801(4))). Therefore, rows (1811), (1813), (1801(4)) are considered to have alternating rows (e.g., (1801(1)) to (1801(4))). (15) and (1817) can be referred to as the current row (also known as the top field), and rows (1812), (1814), (1816), and (1818) can be referred to as the next row (also known as the bottom field). Four luminance samples (e.g., (1801(1)) to (1801(4))) are located at the current row (e.g., (1811)) and the next row (e.g., (1812)). Relative positions 2 to 3 are located in the current row, relative positions 0 to 1 are located between the current row and the corresponding next row, and relative positions 4 to 5 are located in the next row.

[0216] The chroma samples (1803), (1804), (1805), (1806), (1807), or (1808) are located in rows (1851) through (1854) within each chroma block. The specific position of rows (1851) through (1854) can depend on the chroma sample type of the chroma sample. For example, for chroma samples (1803) through (1804) with corresponding chroma sample types 0 through 1, row (1851) is located between rows (1811) and (1812). For chroma samples (1805) through (1806) with corresponding chroma sample types 2 through 3, row (1851) is located in the same position as the current row (1811). For chroma samples (1807) through (1808) with corresponding chroma sample types 4 through 5, row (1851) is located in the same position as the next row (1812). The above description can be appropriately applied to lines (1852) to (1854), and detailed descriptions have been omitted for the sake of brevity.

[0217] The above can be displayed, stored, and / or transmitted using any suitable scanning method. Figure 18A The text describes the luminance blocks and (one or more) corresponding chrominance blocks. In the example, progressive scan is used.

[0218] Interlaced scanning can be used, such as Figure 18BAs shown. As mentioned above, the chroma subsampling format is 4:2:0 (e.g., chroma_format_idc equals 1). In the example, the variable chroma position type (e.g., ChromaLocType) indicates the current line (e.g., ChromaLocType is chroma_sample_loc_type_top_field) or the next line (e.g., ChromaLocType is chroma_sample_loc_type_bottom_field). The current lines (1811), (1813), (1815), and (1817) and the next lines (1812), (1814), (1816), and (1818) can be scanned individually. For example, the current lines (1811), (1813), (1815), and (1817) can be scanned first, and then the next lines (1812), (1814), (1816), and (1818) can be scanned. The current line may include a luminance sample (1801), and the next line may include a luminance sample (1802).

[0219] Similarly, the corresponding chroma blocks can be interlaced. Lines (1851) and (1853) containing chroma samples (1803), (1804), (1805), (1806), (1807), or (1808) without fill can be called the current line (or current chroma line), and lines (1852) and (1854) containing chroma samples (1803), (1804), (1805), (1806), (1807), or (1808) with gray fill can be called the next line (or next chroma line). In the example, during interlaced scanning, lines (1851) and (1853) are scanned first, followed by lines (1852) and (1854).

[0220] In some examples, constrained directional enhancement filtering (CDEF) techniques can be used. Using an in-loop constrained directional enhancement filter (CDEF) can filter out coded artifacts while preserving image details. In examples (e.g., HEVC), the Sample Adaptive Offset (SAO) algorithm can achieve a similar goal by constraining signal offsets for pixels of different classes. Unlike SAO, CDEF is a nonlinear spatial filter. In some examples, CDEF can be constrained to be easily vectorized (i.e., achievable via single instruction multiple data (SIMD) operations). Note that other nonlinear filters such as median filters and bilateral filters cannot be handled in the same way.

[0221] In some cases, the amount of ringing artifact in an encoded image is often roughly proportional to the quantization step size. The amount of detail is a property of the input image, but the smallest detail preserved in a quantized image is also often proportional to the quantization step size. For a given quantization step size, the amplitude of ringing is typically smaller than the amplitude of detail.

[0222] CDEF can be used to identify the orientation of each block, and then adaptively filter along the identified orientation and further filter to a lesser extent along an orientation rotated 45 degrees from the identified orientation. In some examples, the encoder can search for filter strength and can explicitly signal the filter strength, which allows for a high degree of control over ambiguity.

[0223] Specifically, in some examples, a direction search is performed on the reconstructed pixels immediately after the deblocking filter. Since these pixels are available to the decoder, the direction can be searched by the decoder, and therefore, in these examples, direction signaling is not required. In some examples, the direction search can operate on specific block sizes, such as 8×8 blocks, which are small enough to adequately handle non-linear edges, yet large enough to reliably estimate the direction when applied to a quantized image. Furthermore, having a constant direction over an 8×8 region makes vectorization of the filter easier. In some examples, each block (e.g., 8×8) can be compared to a fully oriented block to determine the difference. A fully oriented block is a block where all pixels along a line in one direction have the same value. In these examples, a difference metric, such as the sum of squared differences (SSD) or root mean square (RMS) error, can be computed between each fully oriented block and the block. The fully oriented block with the minimum difference (e.g., minimum SSD, minimum RMS, etc.) can then be determined, and the direction of the determined fully oriented block can be the direction that best matches the pattern in the block.

[0224] Figure 19 An example of directional search according to an embodiment of this disclosure is shown. In the example, block (1910) is a reconstructed 8×8 block and is output from the deblocking filter. Figure 19 In the example, direction search can determine the direction for block (1910) from the eight directions shown in (1920). Eight fully oriented blocks (1930) are formed corresponding to the eight directions (1920). A fully oriented block corresponding to a direction is a block where pixels along the line of that direction have the same value. Furthermore, a difference metric, such as SSD, RMS error, etc., can be calculated between each of the fully oriented blocks (1930) and block (1910). Figure 19In the example, the RMS error is shown by (1940). As shown by (1943), the RMS error of block (1910) and fully oriented block (1933) is the smallest, so the direction (1923) is the direction that best matches the pattern in block (1910).

[0225] After identifying the block orientation, a nonlinear low-pass directional filter can be determined. For example, the filter taps of the nonlinear low-pass directional filter can be aligned along the identified orientation to reduce ringing while preserving directional edges or patterns. However, in some examples, directional filtering alone is sometimes insufficient to reduce ringing. In the example, additional filter taps are also used for pixels that are not aligned with the identified orientation. To reduce the risk of blurring, the additional filter taps are handled more conservatively. For this purpose, CDEF includes primary filter taps and secondary filter taps. In the example, the complete 2D CDEF filter can be expressed as Equation (14):

[0226]

[0227] Where D represents the damping parameter, S (p) S represents the strength of the tap in the primary filter. (s) The function represents the strength of the secondary filter taps, `round(·)` rounds the relationship to zero, `w` represents the filter weights, and `f(d,S,D)` is a constraint function that operates on the difference between the filtered pixel and each of its neighbors. In the example, for small differences, the function `f(d,S,D)` equals `D`, which makes the filter behave like a linear filter; when the difference is large, the function `f(d,S,D)` equals `0`, which effectively ignores the filter taps.

[0228] In some examples, in addition to deblocking operations, an in-loop restoration scheme is used in post-encoded deblocking to generally denoise and improve edge quality. In these examples, the in-loop restoration scheme is switchable within each frame for appropriately sized tiles. The in-loop restoration scheme is based on a separable symmetric Wiener filter, a dual self-guided filter with subspace projection, and a domain transform recursive filter. Because content statistics can vary significantly within a frame, the in-loop restoration scheme is integrated within a switchable framework, where different schemes can be triggered in different regions of the frame.

[0229] A separable symmetric Wiener filter can be one of the in-loop recovery schemes. In some examples, each pixel in the degraded frame can be reconstructed as a noncausal filtered version of the pixels within a w×w window surrounding it, where w = 2r + 1 is odd for integers r. If the 2D filter taps are in column vectorized form aw 2 If the element vector F represents a ×1 element, then direct LMMSE optimization causes the filter parameters to change from F = H.-1 M is given, where H = E[XX] T [x] is the autocovariance x, and the w×w window around the pixel. 2 A column vectorized version of each sample, and M = E[YX] T ] is the cross-correlation between x and the scalar source sample y to be estimated. In the example, the encoder can estimate H and M based on the implementation in the deblocked frame and the source, and can send the resulting filter F to the decoder. However, this will not only affect the transmission w 2 Each tap incurs a considerable bit rate cost, and non-separable filtering makes decoding overly complex. In some implementations, several additional constraints are imposed on the properties of F. For the first constraint, F is constrained to be separable, such that filtering can be implemented as separable horizontal and vertical w-tap convolutions. For the second constraint, each of the horizontal and vertical filters is constrained to be symmetric. For the third constraint, it is assumed that the sum of the horizontal and vertical filter coefficients is 1.

[0230] Dual self-guided filtering with subspace projection can be one of the in-loop recovery schemes. Guided filtering is an image filtering technique in which the local linear model shown in equation (15) is used.

[0231] y = Fx + G (Equation 15)

[0232] The filtered output y is calculated based on the unfiltered sample x, where F and G are determined based on statistics of the degraded image and the guiding image near the filtered pixels. If the guiding image is identical to the degraded image, the resulting so-called self-guided filtering has the effect of preserving smooth edges. In the example, a specific form of self-guided filtering can be used. The specific form of self-guided filtering depends on two parameters: the radius r and the noise parameter e, and is enumerated as follows:

[0233] 1. Obtain the mean μ and variance σ of the pixels within a (2r+1)×(2r+1) window surrounding each pixel. 2 This step can be effectively achieved through box filtering based on panoramic imaging.

[0234] 2. Calculate for each pixel: f = σ 2 / (σ 2 +e); g=(1-f)μ

[0235] 3. Calculate F and G for each pixel, and use them as the average of the f and g values ​​in a 3×3 window around the pixel.

[0236] The specific form of a self-guided filter is controlled by r and e, where a higher r implies a higher spatial variance, and a higher e implies a higher range variance.

[0237] Figure 20 Examples of subspace projections are shown, illustrating some examples. For example... Figure 20 As shown, even if the recovered X1 and X2 are not close to the source Y, as long as they move in the correct direction, the appropriate multiplier {α,β} can make them closer to the source Y.

[0238] In some examples (e.g., HEVC), a filtering technique called Sample Adaptive Offset (SAO) can be used. In some examples, SAO is applied to the reconstructed signal after the deblocking filter. SAO can use the offset value given in the slice header. In some examples, for luminance samples, the encoder can decide whether to apply (enable) SAO to the slice. When SAO is enabled, the current image allows the coding unit to be recursively segmented into four sub-regions, and each sub-region can select a SAO type from multiple SAO types based on features within the sub-region.

[0239] Figure 21 A table (2100) showing multiple SAO types according to embodiments of this disclosure is illustrated. Table (2100) shows SAO types 0 to 6. Note that SAO type 0 is used to indicate that no SAO is applied. Furthermore, each SAO type from SAO type 1 to SAO type 6 includes multiple categories. SAO can classify reconstructed pixels in a sub-region and reduce distortion by adding an offset to pixels of each category in the sub-region. In some examples, edge attributes can be used for pixel classification in SAO types 1 to 4, and pixel intensity can be used for pixel classification in SAO types 5 to 6.

[0240] Specifically, in implementations such as SAO types 5 to 6, a band offset (BO) can be used to classify all pixels in a sub-region into multiple bands. Each of the multiple bands includes pixels with the same intensity interval. In some examples, the intensity range is divided equally into multiple intervals, such as 32 intervals from zero to the maximum intensity value (e.g., 255 for an 8-bit pixel), and each interval is associated with an offset. Furthermore, in an example, the 32 bands are divided into two groups, such as a first group and a second group. The first group includes the central 16 bands (e.g., the 16 intervals in the middle of the intensity range), while the second group includes the remaining 16 bands (e.g., the 8 intervals on the lower side of the intensity range and the 8 intervals on the higher side of the intensity range). In the example, only the offset of one of the two groups is transmitted. In some implementations, when using the pixel classification operation in the BO, the five most significant bits of each pixel can be directly used as the band index.

[0241] Furthermore, in implementations such as SAO types 1 to 4, edge offset (EO) can be used for pixel classification and offset determination. For example, pixel classification can be determined based on a 1D 3-pixel pattern that takes edge orientation information into account.

[0242] Figure 22 An example of a 3-pixel pattern used for pixel classification in edge offsets is shown in some examples. Figure 22 In the example, the first mode (2210) (as shown in 3 gray pixels) is called the 0-degree mode (associated horizontally with the 0-degree mode), the second mode (2220) (as shown in 3 gray pixels) is called the 90-degree mode (associated vertically with the 90-degree mode), the third mode (2230) (as shown in 3 gray pixels) is called the 135-degree mode (associated diagonally with the 135-degree mode), and the fourth mode (2240) (as shown in 3 gray pixels) is called the 45-degree mode (associated diagonally with the 45-degree mode). In the example, edge orientation information for sub-regions can be considered for selection. Figure 22 One of the four orientation modes shown. In this example, the selection can be sent as side information in the encoded video bitstream. Then, by comparing each pixel with its two neighboring pixels in the direction associated with the orientation mode, pixels in the sub-region can be classified into multiple categories.

[0243] Figure 23 Table (2300) shows the pixel classification rules used for edge offsets in some examples. Specifically, pixel c (also...) Figure 22 (shown in each pattern) with two adjacent pixels (in) Figure 22 Each pattern is also shown in gray for comparison, and the comparison can be based on... Figure 23 The pixel classification rule shown classifies pixel c into one of categories 0 to 4.

[0244] In some implementations, the SAO on the decoder side can operate independently of the maximum coding unit (LCU) (e.g., CTU), thereby saving row buffers. In some examples, when 90-degree, 135-degree, and 45-degree classification modes are selected, the pixels in the top and bottom rows of each LCU are not SAO processed; when 0-degree, 135-degree, and 45-degree modes are selected, the pixels in the leftmost and rightmost columns of each LCU are not SAO processed.

[0245] Figure 24Example (2400) of a syntax that might require signaling for a CTU if parameters are not merged from adjacent CTUs is shown. For example, the signaling syntax element `sao_type_idx[cldx][rx][ry]` can indicate the SAO type of a subregion. The SAO type can be BO (with offset) or EO (edge ​​offset). When the value of `sao_type_idx[cldx][rx][ry]` is 0, it indicates that the SAO is OFF; values ​​from 1 to 4 indicate the use of one of the four EO categories corresponding to 0°, 90°, 135°, and 45°; and a value of 5 indicates the use of BO. Figure 24 In the example, each of the BO and EO types has four SAO offset values ​​(sao_offset[cIdx][rx][ry][0] to sao_offset[cIdx][rx][ry][3]) signaled.

[0246] like Figure 24 As shown, SAO is a filtering process that requires additional signaling overhead to instruct on all relevant information needed to perform SAO filtering, such as... Figure 24 As shown, this may limit the potential of SAO.

[0247] Typically, the filtering process can use a reconstructed sample with a first color component as input (e.g., Y or Cb or Cr, or R or G or B) to generate an output, and apply the output of the filtering process to a second color component, which can be the same color component as the first color component or a different color component than the first color component.

[0248] In examples of cross-component filtering (CCF), the filter coefficients are derived based on mathematical formulas. These coefficients are signaled from the encoder to the decoder and used to generate an offset using a linear combination. The generated offset is then added to the reconstructed samples as part of the filtering process. For example, an offset is generated based on a linear combination of the filter coefficients and the luma sample, and this offset is added to the reconstructed chroma sample. Examples of CCF rely on the assumption of a linear mapping between the reconstructed luma sample values ​​and the incremental values ​​between the original and reconstructed chroma samples. However, the mapping between the reconstructed luma sample values ​​and the incremental values ​​between the original and reconstructed chroma samples does not necessarily follow a linear mapping, and therefore, the coding performance of CCF may be limited under the linear mapping assumption.

[0249] Various aspects of this disclosure provide nonlinear mapping techniques that can be used for cross-component filtering and / or same-color component filtering without significant signaling overhead. In one example, the nonlinear mapping technique can be used for cross-component filtering to generate cross-component sample offsets. In another example, the nonlinear mapping technique can be used for same-color component filtering to generate local sample offsets.

[0250] For convenience, the filtering process using nonlinear mapping techniques can be referred to as sample offset by nonlinear mapping (SO-NLM). SO-NLM in cross-component filtering can be called cross-component sample offset (CCSO). SO-NLM in same-color component filtering can be called local sample offset (LSO).

[0251] In the examples, CCSO and LSO can be used as loop filters to reduce distortion of the reconstructed samples. CCSO and LSO do not rely on the linear mapping assumptions used in the relevant example CCF. For example, CCSO does not rely on the assumption of a linear mapping between the luminance reconstructed sample values ​​and the incremental values ​​between the original chrominance sample and the chrominance reconstructed sample. Similarly, LSO does not rely on the assumption of a linear mapping between the reconstructed sample values ​​of a color component and the incremental values ​​between the original sample with that color component and the reconstructed sample with that color component.

[0252] The following description describes the SO-NLM filtering process: using reconstructed samples with a first color component as input (e.g., Y or Cb or Cr, or R or G or B) to generate an output, and applying the output of the filtering process to a second color component. This description applies to LSO when the second color component is the same as the first color component; and to CCSO when the second color component is different from the first color component.

[0253] In SO-NLM, a nonlinear mapping is derived at the encoder side. This nonlinear mapping lies between the reconstructed sample with the first color component in the filter support region and the offset to be added to the second color component in the filter support region. In LSO, the nonlinear mapping is used when the second color component is the same as the first color component; in CCSO, it is used when the second color component is different from the first color component. The domain of the nonlinear mapping is determined by different combinations of processed input reconstructed samples (also known as combinations of possible reconstructed sample values).

[0254] The SO-NLM technique can be illustrated using a specific example. In this example, reconstructed samples from the first color component are determined within a filter support region (also called the "filter support area"). The filter support region is the area within which a filter can be applied, and the filter support region can have any suitable shape.

[0255] Figure 25 Examples of filter support regions (2500) according to some embodiments of this disclosure are shown. The filter support region (2500) includes four reconstructed samples: P0, P1, P2, and P3, each with a first color component. Figure 25 In the example, the four reconstructed samples can form a cross shape in the vertical and horizontal directions, and the center of the cross is the location of the sample to be filtered. The sample at the center location that has the same color component as P0 to P3 is denoted by C. The sample at the center location that has a second color component is denoted by F. The second color component can be the same as or different from the first color component of P0 to P3.

[0256] Figure 26 An example of another filter support region (2600) according to some embodiments of this disclosure is shown. The filter support region (2600) includes four reconstructed samples P0, P1, P2, and P3 forming a square shape and having a first color component. Figure 26 In the example, the center of the square shape is the location of the sample to be filtered. A sample at the center position that has the same color component as P0 to P3 is denoted by C. A sample at the center position that has a second color component is denoted by F. The second color component can be the same as or different from the first color component of P0 to P3.

[0257] The reconstructed samples are then processed appropriately to form filter taps. In a specific example, the reconstructed samples are processed in the following two steps.

[0258] In the first step, the incremental values ​​between P0 and P3 and C are calculated respectively. For example, m0 represents the incremental value between P0 and C; m1 represents the incremental value between P1 and C; m2 represents the incremental value between P2 and C; and m3 represents the incremental value between P3 and C.

[0259] In the second step, the increment values ​​m0 to m3 are further quantized, and the quantized values ​​are represented as d0, d1, d2, and d3. In the example, based on the quantization process, the quantized value can be one of -1, 0, or 1. For example, when m is less than -N (where N is a positive value and is called the quantization step size), the value m can be quantized to -1; when m is in the range [-N, N], the value m can be quantized to 0; and when m is greater than N, the value m can be quantized to 1. In some examples, the quantization step size N can be one of 4, 8, 12, 16, etc.

[0260] In some implementations, the quantization values ​​d0 to d3 are filter taps and can be used to identify a combination in the filter domain. For example, filter taps d0 to d3 can form combinations in the filter domain. Each filter tap can have three quantization values, so when four filter taps are used, the filter domain includes 81 (3×3×3×3) combinations.

[0261] Figures 27A to 27C A table (2700) showing 81 combinations according to an embodiment of the present disclosure is illustrated. Table (2700) comprises 81 rows corresponding to the 81 combinations. In each row corresponding to a combination, a first column includes an index of the combination; a second column includes the value of filter tap d0 for that combination; a third column includes the value of filter tap d1 for that combination; a fourth column includes the value of filter tap d2 for that combination; a fifth column includes the value of filter tap d3 for that combination; and a sixth column includes an offset value associated with the combination of nonlinear mappings. In the example, when filter taps d0 to d3 are determined, an offset value (denoted by s) associated with the combination of d0 to d3 can be determined according to table (2700). In the example, offset values ​​s0 to s80 are integers, such as 0, 1, -1, 3, -3, 5, -5, -7, etc.

[0262] In some implementations, the final filtering process of SO-NLM can be applied as shown in equation (16):

[0263] f′=clip(F+s) Equation (16)

[0264] Where F is the reconstructed sample with the second color component to be filtered, and s is an offset value determined based on the filter taps as a result of processing the reconstructed sample with the first color component, such as using Table (2700). The sum of the reconstructed sample F and the offset value s is further clipped to a range associated with the bit depth to determine the final filtered sample f' with the second color component.

[0265] Note that in the case of LSO, the second color component described above is the same as the first color component; and in the case of CCSO, the second color component described above may be different from the first color component.

[0266] Note that the above description may be adjusted for other implementations of this disclosure.

[0267] According to aspects of this disclosure, at the encoder side, the encoding device can derive a mapping between a reconstructed sample with a first color component in the filter support region and an offset to be added to a reconstructed sample with a second color component. This mapping can be any suitable linear or non-linear mapping. A filtering process can then be applied at the encoder side and / or the decoder side based on this mapping. For example, the mapping is appropriately communicated to the decoder (e.g., the mapping is included in the encoded video bitstream transmitted from the encoder side to the decoder side), and the decoder can then perform a filtering process based on this mapping.

[0268] According to some aspects of this disclosure, the mapping is a nonlinear mapping between a reconstructed sample having a first color component and an offset to be applied to a reconstructed sample having a second color component. In some examples, to perform a filtering process in a filter support region, a reconstructed sample having the first color component in the filter support region can be input to the filtering process, and the offset is the output of the filtering process. The offset can be applied to a reconstructed sample having the second color component in the filter support region.

[0269] In some implementations, lookup tables are used to describe the mapping. A lookup table associates the input of the filtering process (reconstructed samples with a first color component) with the output of the filtering process (the offset to be applied to reconstructed samples with a second color component). Therefore, given a set of reconstructed sample values ​​as input, the corresponding output offset value can be retrieved from the lookup table.

[0270] In this implementation, the input is a reconstructed luminance sample located in a given filter support region (zone), and the output is the offset to be applied to the chroma reconstructed sample.

[0271] In another implementation, the reconstructed samples are first quantized before being used as input to the filtering process. For example, the reconstructed samples can be quantized based on a quantization step.

[0272] In another implementation, linear operations are used to process the reconstructed samples, and the processed reconstructed samples are then used as input to a filtering process. For example, a value (e.g., the center of the reconstructed sample, the average of the reconstructed samples) can be subtracted from the reconstructed samples. The result of the subtraction can be used as input to the filtering process.

[0273] In another implementation, the reconstructed samples are first processed using linear operations, and the output of the linear operations is further quantized and then used as input to the filtering process. In the specific example above, the first step uses linear operations to calculate the increment values ​​m0 to m3 between P0 to P3 and C, respectively. Then, in the second step, the increment values ​​m0 to m3 are further quantized to d0 to d3.

[0274] According to aspects of this disclosure, the domain of the mapping is determined by different combinations of processed (e.g., linearly processed) input reconstructed samples, such as combinations of P0 to P3, combinations of m0 to m3, and combinations of d0 to d3. In some examples, all possible combinations of processed input reconstructed samples are included in the domain (referred to as the complete domain). In one example, the combination of processed reconstructed samples is identified by combinations of quantization differences, such as d0 to d3, between the surrounding samples and the center sample. In another example, the combination of processed reconstructed samples is identified by the quantization differences between the surrounding samples. For example, n0 is the difference between P0 and P1, n1 is the difference between P1 and P2, n2 is the difference between P2 and P3, and n3 is the difference between P3 and P0. Then, n0 to n3 can be quantized similarly to m0 to m3 to determine e0 to e3, which can be used as combinations of processed reconstructed samples.

[0275] In the specific example above, four surrounding samples are used (e.g., Figure 25 or Figure 26 (As shown in the diagram) to identify each combination of the processed reconstructed samples. For example, the differences m0 to m3 between the surrounding samples P0 to P3 and the center sample C can be quantized to [-1, 0, 1] based on the quantization step size N (N is a positive integer, such as 1, 3, 5, 7). For example, to quantize a value, it is quantized to -1 when the value is less than -N; quantized to 0 when the value is in the range [-N, N]; and quantized to 1 when the value is greater than N. When using four surrounding samples, the total number of combinations is 81 (3 4 ), such as Figures 27A to 27C The table shown is (2700).

[0276] According to aspects of this disclosure, the combination of processed reconstructed samples depends on the internal bit depth at which the codec is operating. For example, the calculation of m0 to m3 depends on the internal bit depth, and m0 to m3 are quantized to determine d0 to d3. Therefore, the combination of d0 to d3 can depend on the internal bit depth.

[0277] According to another aspect of this disclosure, the combination of processed reconstructed samples depends on quantization parameters, such as the quantization step size. For example, the determination of d0 to d3 in the second step of a particular example is based on the quantization step size N.

[0278] Note that any suitable reconstructed sample from the filter support region can be used to determine the combination. In this implementation, the reconstructed sample located at the boundary of the filter support region is referred to as the surrounding sample and can be used to determine the combination.

[0279] In another implementation, reconstructed samples that do not include the center location in the filter support region are referred to as surrounding samples and can be used to determine combinations.

[0280] In another embodiment, the reconstructed samples that do not include samples near the center in the filter support region are called surrounding samples and can be used to determine the combination.

[0281] In some implementations, the domain of the mapping is determined by different combinations of processed (e.g., linearly processed) reconstructed samples (e.g., d0 to d3). In an example, a predefined subset of the combinations of processed reconstructed samples (e.g., d0 to d3) is included in the domain (referred to as the subset domain).

[0282] Figure 28 A lookup table (2800) according to an embodiment of this disclosure is shown. The mapping fields in the lookup table (2800) include all combinations of d0 to d3, such as Figure 28 A subset of the five combinations.

[0283] In some implementations, the offset output from the filtering process is such as Figures 27A to 27C In this context, s0 to s80 are integers within the predefined range [P, Q]. P and Q can be integers. An example of this range is [-K, K-1], where K is a positive integer. Example values ​​for K include, but are not limited to, 16, 8, 4, 2, and 1.

[0284] In one implementation, the values ​​of P and Q depend on the internal bit depth the codec is operating at. In another implementation, the values ​​of P and Q depend on quantization parameters, such as the quantization step size N. In yet another implementation, the offset output from the filtering process is a predefined subset of integers. For example, the predefined subset of integers includes values ​​of -2. T and / or 2 T (where T is an integer) such as 16, -16, 8, -8, 4, -4, 2, -2, 1, etc.

[0285] According to aspects of this disclosure, the offset associated with the combination of processed reconstructed samples in the mapping can be determined by the encoder based on a predefined cost metric. In an example, for a combination of processed reconstructed samples representing the first color component, the cost metric is defined as the average of the differences between the original values ​​of the samples to be filtered representing the second color component and the reconstructed sample values.

[0286] In the CCSO example, the reconstructed samples used as input to the filtering process come from a first color component, while the samples to be filtered come from a second color component that is different from the first color component. In the LSO example, the reconstructed samples used as input to the filtering process come from a first color component, while the samples to be filtered come from a second color component that is the same color component as the first color component.

[0287] In some implementations, the mapping information, such as for each Figures 27A to 27C The 81 offsets for the 81 combinations can be signaled from the encoder side to the decoder side. In the example, the mapping information is carried in the encoded video bitstream, such as in the video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), adaptive parameter set (APS), slice header, and tile header.

[0288] According to aspects of this disclosure, the filtering process determines an offset value based on a combination of a mapping and a reconstructed sample having a first color component. For example, the offset value is associated with a combination in the mapping (e.g., a lookup table). The offset value is then applied to the sample to be filtered, which has a second color component. In some embodiments, clipping is performed after the offset value is added to the sample to be filtered to ensure that the filtered sample value does not exceed a maximum or minimum value. In an example, clipping is performed according to equation (17):

[0289] clip(x)=max(min(x,max_val),min_val) Formula (17)

[0290] Where max represents the maximum operator, min represents the minimum operator, max_val represents the maximum sample value, min_val represents the minimum sample value, and x is the sample value to be limited.

[0291] Figure 29A flowchart outlining a method (2900) according to an embodiment of this disclosure is shown. The method (2900) can be used to reconstruct blocks in an image of an encoded video sequence. The term "block" can be interpreted as a prediction block, coding unit, luma block, chroma block, etc. In various embodiments, the method (2900) is executed by a processing circuit system such as those in terminal devices (310), (320), (330), and (340); a processing circuit system performing the function of a video encoder (403); a processing circuit system performing the function of a video decoder (410); a processing circuit system performing the function of a video decoder (510); a processing circuit system performing the function of a video encoder (603), etc. In some embodiments, the method (2900) is implemented as software instructions, and therefore, when the processing circuit system executes the software instructions, the processing circuit system executes the method (2900). The method begins at (S2901) and proceeds to (S2910).

[0292] At (S2910), a first combination is determined based on reconstructed samples having a first color component within the filter support region. In some embodiments, at least one of a linear operation and a quantization operation may be used to determine the first combination based on reconstructed samples having a first color component within the filter support region.

[0293] In one implementation, the difference between reconstructed samples having a first color component can be calculated. The difference is then quantized to determine a first combination. In some examples, the difference between surrounding reconstructed samples and juxtaposed samples having the first color component within the reconstructed sample is calculated. The juxtaposed samples are located at the same position as the sample to be filtered, which has a second color component. The difference is then quantized based on a quantization step size to determine the first combination. Note that the first combination can depend on the internal bit depth and quantization parameters such as the quantization step size.

[0294] Note that in this implementation, the surrounding reconstructed samples are located at the boundaries of the filter support region. In another example, the surrounding reconstructed samples are samples within the filter support region that do not include samples located at the center of the filter support region. In yet another example, the surrounding reconstructed samples are samples within the filter support region that do not include samples located near the center of the filter support region.

[0295] At (S2920), a first offset value associated with the first combination is determined based on a mapping that associates offset values ​​with combinations. In some embodiments, the mapping is a non-linear mapping defined in the form of a lookup table.

[0296] In some implementations, the mapping associates the offset value with a combination of the full domains of the reconstructed samples with the first color component within the filter support region. The full mapping domain includes all possible combinations. For example, when all four reconstructed samples are processed and quantized to one of three values, the number of all possible combinations is 81(3). 4 In some implementations, the mapping associates the offset value with a combination of subset domains for reconstructed samples having a first color component within the filter support region. The subset domains comprise a subset of the complete domain combination.

[0297] In some implementations, the mapping is carried in at least one of the video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), adaptive parameter set (APS), slice header, and tile header of the encoded video bitstream.

[0298] In some implementations, the offset value of the mapping is derived based on a cost metric. The mapping can then be included in at least one of the following: Video Parameter Set (VPS), Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptive Parameter Set (APS), Slice Header, and Tile Header of the encoded video bitstream.

[0299] At (S2930), a first offset value is applied to the sample to be filtered in the filter support region that has a second color component to determine the filtered sample that has a second color component. In this example, the second color component is the first color component. In another example, the second color component is a color component different from the first color component. For example, the first color component is a luminance component, while the second color component is a chrominance component.

[0300] In some examples, a first offset value is added to the sample to be filtered that has a second color component to determine an intermediate filtered sample. Then, the intermediate filtered sample is clipped based on the maximum and minimum values ​​to determine the filtered sample with the second color component.

[0301] Method (2900) proceeds to (S2999) and terminates.

[0302] Method (2900) can be adjusted as appropriate. Steps in method (2900) can be modified and / or omitted. Additional steps can be added. Any suitable implementation order can be used.

[0303] The embodiments described in this disclosure can be used individually or in any combination in any order. Furthermore, each of the methods (or embodiments), encoders, and decoders can be implemented using a processing circuitry system (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored on a non-transitory computer-readable medium.

[0304] The above-mentioned techniques can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. For example, Figure 30 A computer system (3000) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0305] Computer software can be coded using any suitable machine code or computer language. Machine code or computer language can be subjected to mechanisms such as assembly, compilation, and linking to create code that includes instructions. These instructions can be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or through interpretation, microcode execution, etc.

[0306] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0307] Figure 30 The components shown for the computer system (3000) are exemplary in nature and are not intended to impose any limitation on the scope of use or functionality of computer software implementing embodiments of this disclosure. The configuration of the components should also not be construed as having any dependency or requirement relating to any one or a combination of components shown in the exemplary embodiments of the computer system (3000).

[0308] The computer system (3000) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from one or more human users via, for example, tactile input (such as keystrokes, swipes, data glove movements), audio input (such as speech, tapping), visual input (such as gestures), and olfactory input (not depicted). The human-machine interface device may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (such as speech, music, ambient sounds), images (such as scanned images, photographic images obtained from still image capturing devices), and video (such as two-dimensional video, three-dimensional video including stereoscopic video).

[0309] The input human-machine interface device may include one or more of the following (only one of each is shown): keyboard (3001), mouse (3002), touchpad (3003), touch screen (3010), data glove (not shown), joystick (3005), microphone (3006), scanner (3007), and camera device (3008).

[0310] The computer system (3000) may also include certain human-machine interface output devices. Such human-machine interface output devices can stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include: haptic output devices (e.g., haptic feedback via a touchscreen (3010), data gloves (not shown), or joystick (3005), but haptic feedback devices that are not used as input devices may also exist); audio output devices (such as speakers (3009), headphones (not depicted)); visual output devices (such as screens (3010), including cathode ray tube (CRT) screens, liquid crystal displays (LCD) screens, plasma screens, organic light-emitting diode (OLED) screens, each with or without touchscreen input capability, each with or without haptic feedback capability—some of which may be able to output two-dimensional or more than three-dimensional visual outputs in a manner such as stereoscopic image output; virtual reality glasses (not depicted); holographic displays and ashtrays (not depicted)); and printers (not depicted).

[0311] The computer system (3000) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD read-only memory (ROM) / readwrite (RW) (3020) having media such as CD / DVD (3021), thumb drives (3022), removable hard disk drives or solid-state drives (3023), legacy magnetic media such as magnetic tapes and floppy disks (not depicted), and devices based on application-specific integrated circuits (ASIC) / programmable logic devices (PLD) such as security dongles (not depicted), etc.

[0312] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0313] The computer system (3000) may also include interfaces (3054) to one or more communication networks (3055). The networks may be, for example, wireless, wired, or optical. The networks may also be local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), vehicular and industrial networks, real-time networks, latency-tolerant networks, etc. Examples of networks include: local area networks such as Ethernet; wireless LANs; cellular networks including GSM (Global System for Mobile Communication), 3G (Third Generation), 4G (Fourth Generation), 5G (Fifth Generation), LTE (Long Term Evolution), etc.; cable or wireless wide area digital television networks including cable television, satellite television, and terrestrial broadcast television; vehicular networks including CAN bus and industrial networks, etc. Some networks typically require an external network interface adapter that is attached to a general-purpose data port or peripheral bus (3049) (such as, for example, the USB (Universal Serial Bus, USB) port of a computer system (3000); other networks are typically integrated into the core of the computer system (3000) by attaching to a system bus as described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). The computer system (3000) can communicate with other entities through any of these networks. Such communication can be one-way receive-only (e.g., broadcast television), one-way transmit-only (e.g., a CAN bus to a certain CAN bus device), or bidirectional, for example, using a local area digital network or a wide area digital network to other computer systems. Certain protocols and protocol stacks can be used on each of these networks and network interfaces as described above.

[0314] The human-machine interface device, human-accessible storage device and network interface mentioned above can be attached to the core (3040) of the computer system (3000).

[0315] The core (3040) may include one or more central processing units (CPUs) (3041), graphics processing units (GPUs) (3042), dedicated programmable processing units in the form of field-programmable gate arrays (FPGAs) (3043), hardware accelerators (3044) for certain tasks, graphics adapters (3050), etc. These devices, along with read-only memory (ROM) (3045), random access memory (3046), and internal mass storage devices such as internal non-user-accessible hard disk drives (SDs), solid state drives (SSDs), etc. (3047), may be connected via a system bus (3048). In some computer systems, the system bus (3048) may be accessed in the form of one or more physical connectors to allow for expansion with additional CPUs, GPUs, etc. Peripheral devices may be attached directly or via a peripheral bus (3049) to the core's system bus (3048). In this example, a screen (3010) may be connected to a graphics adapter (3050). Peripheral bus architectures include PCI (Peripheral Component Interconnect), USB, etc.

[0316] The CPU (3041), GPU (3042), FPGA (3043), and accelerator (3044) can execute certain instructions, which can be combined to form the aforementioned computer code. This computer code can be stored in ROM (3045) or RAM (3046). Transient data can also be stored in RAM (3046), while permanent data can be stored, for example, in an internal mass storage device (3047). Fast storage and retrieval of any storage device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (3041), GPUs (3042), mass storage devices (3047), ROMs (3045), RAMs (3046), etc.

[0317] Computer-readable media may contain computer code for performing operations of various computer implementations. The media and computer code may be media and computer code specifically designed and constructed for the purposes of this disclosure, or they may be media and computer code of types known and available to those skilled in the art of computer software.

[0318] By way of example and not limitation, a computer system (3000) with an architecture—and in particular, a core (3040)—can be functionalized by a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such a computer-readable medium can be a medium associated with a user-accessible mass storage device as described above, and with certain storage devices of the core (3040) having non-transitory characteristics, such as an internal mass storage device (3047) or ROM (3045) of the core. Software implementing various embodiments of this disclosure can be stored in such a device and executed by the core (3040). Depending on specific needs, the computer-readable medium may include one or more memory devices or chips. The software can cause the core (3040)—and in particular the processors therein (including CPUs, GPUs, FPGAs, etc.)—to execute specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM (3046) and modifying such data structures according to processes defined by the software. Alternatively or as an alternative, a computer system may be provided with functionality by hard-wired or otherwise embodied logic in circuitry (e.g., accelerator (3044)), which may replace or operate with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, reference to software may include logic, and conversely, reference to logic may include software. Where appropriate, reference to a computer-readable medium may include circuitry (such as an integrated circuit (IC)) storing software for execution, circuitry implementing logic for execution, or both. This disclosure includes any suitable combination of hardware and software.

[0319] Appendix A: Acronyms

[0320] JEM: Joint Development Model

[0321] VVC: Universal Video Coding

[0322] BMS: Benchmark Set

[0323] MV: Motion Vector

[0324] HEVC: High-Efficiency Video Coding

[0325] MPM: Most Likely Pattern

[0326] WAIP: Wide-angle Intra-frame Prediction

[0327] SEI: Supplemental Enhancement Information

[0328] VUI: Video Availability Information

[0329] GOP: Image Group

[0330] TU: Transformation Unit

[0331] PU: Prediction Unit

[0332] CTU: Coding Tree Unit

[0333] CTB: Coded Tree Block

[0334] PB: Prediction Block

[0335] HRD: Hypothetical Reference Decoder

[0336] SDR: Standard Dynamic Range

[0337] SNR: Signal-to-noise ratio

[0338] CPU: Central Processing Unit

[0339] GPU: Graphics Processing Unit

[0340] CRT: Cathode Ray Tube

[0341] LCD: Liquid Crystal Display

[0342] OLED: Organic Light Emitting Diode

[0343] CD: Compact Disc

[0344] DVD: Digital Video Disc

[0345] ROM: Read-Only Memory

[0346] RAM: Random Access Memory

[0347] ASIC: Application-Specific Integrated Circuit

[0348] PLD: Programmable Logic Device

[0349] LAN: Local Area Network

[0350] GSM: Global System for Mobile Communications

[0351] LTE: Long Term Evolution

[0352] CANBus: Controller Area Network Bus

[0353] USB: Universal Serial Bus

[0354] PCI: Peripheral Component Interconnect

[0355] FPGA: Field Programmable Gate Array

[0356] SSD: Solid State Drive

[0357] IC: Integrated Circuit

[0358] CU: Encoding Unit

[0359] PDPC: Location-Related Prediction Combination

[0360] ISP: Intra-Frame Sub-Partition

[0361] SPS: Sequence Parameter Settings

[0362] While this disclosure has described several exemplary embodiments, there are variations, substitutions, and various alternative equivalents that fall within the scope of this disclosure. Therefore, it will be appreciated that those skilled in the art will be able to conceive of many systems and methods that, while not expressly shown or described herein, embody the principles of this disclosure and thus its spirit and scope.

Claims

1. A filtering method for video decoding, characterized in that, The method includes: The first combination is determined based on the reconstructed samples with the first color component within the filter support region; A first offset value associated with the first combination is determined based on a mapping that associates the offset value determined from the processing result of the reconstructed sample having the first color component with possible combinations of reconstructed sample values; and The first offset value is applied to the filter sample with the second color component in the filter support region to determine the filter sample with the second color component; Determining the first combination includes: Calculate the difference between the surrounding reconstructed samples of the reconstructed sample and the juxtaposed sample within the reconstructed sample that has the first color component, wherein the juxtaposed sample and the sample to be filtered that has the second color component are located at the same position; and The difference is quantized based on the quantization step size to determine the first combination.

2. The method according to claim 1, characterized in that, The mapping is a non-linear mapping that is limited by a lookup table.

3. The method according to claim 1, characterized in that, The second color component is different from the first color component.

4. The method according to claim 1, characterized in that, The method further includes: The first combination is determined using at least one of linear operations and quantization operations based on the reconstructed samples with the first color component within the filter support region.

5. The method according to claim 1, characterized in that, The method further includes: Calculate the pairwise differences between the reconstructed samples having the first color component; and The difference is quantified to determine the first combination.

6. The method according to claim 1, characterized in that, The mapping associates the offset value with a combination of the full domains of the reconstructed samples with the first color component within the filter support region.

7. The method according to claim 1, characterized in that, The mapping associates the offset value with a combination of subsets of reconstructed samples having the first color component within the filter support region.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The mapping carried in at least one of the video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), adaptive parameter set (APS), slice header, and tile header of the received encoded video bitstream.

9. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The offset value of the mapping is derived based on the cost metric; and The mapping is included in at least one of the video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), adaptive parameter set (APS), slice header, and tile header of the encoded video bitstream.

10. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The first offset value is added to the sample to be filtered that has the second color component to determine the intermediate filtered sample; and The intermediate filtered samples are limited based on the maximum and minimum values ​​to determine the filtered samples with the second color component.

11. A filtering method for video coding, characterized in that, The method includes: The first combination is determined based on the reconstructed samples with the first color component within the filter support region; A first offset value associated with the first combination is determined based on a mapping that associates the offset value determined from the processing result of the reconstructed sample having the first color component with possible combinations of reconstructed sample values; and The first offset value is applied to the filter sample with the second color component in the filter support region to determine the filter sample with the second color component; Determining the first combination includes: Calculate the difference between the surrounding reconstructed samples of the reconstructed sample and the juxtaposed sample within the reconstructed sample that has the first color component, wherein the juxtaposed sample and the sample to be filtered that has the second color component are located at the same position; and The difference is quantized based on the quantization step size to determine the first combination.

12. A filtering device for video decoding, comprising: The processing circuit is configured to perform the method according to any one of claims 1 to 10.

13. A filtering device for video encoding, comprising: The processing circuit is configured to perform the method according to claim 11.

14. A filtering device for video decoding, characterized in that, The device includes: A first determining module is configured to determine a first combination based on reconstructed samples with a first color component within the filter support region; The second determining module is configured to determine a first offset value associated with the first combination based on a mapping that associates an offset value determined according to the processing result of the reconstructed sample having the first color component with possible combinations of reconstructed sample values; and A third determining module is configured to apply the first offset value to a filter sample having a second color component in the filter support region to determine a filter sample with the second color component. Determining the first combination includes: Calculate the difference between the surrounding reconstructed samples of the reconstructed sample and the juxtaposed sample within the reconstructed sample that has the first color component, wherein the juxtaposed sample and the sample to be filtered that has the second color component are located at the same position; and The difference is quantized based on the quantization step size to determine the first combination.

15. A non-transitory computer-readable medium storing computer-readable instructions, characterized in that, The computer-readable instructions are executed by a processor to perform the method according to any one of claims 1 to 11.

16. A method for storing a bit stream, characterized in that, Generate a bit stream by performing the method of claim 11; and store the bit stream.

17. A method for transmitting a bit stream, characterized in that, The method of claim 11 is used to generate a bit stream; and the bit stream is transmitted.

18. A computer-readable storage medium storing a computer program / instructions and a bit stream thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 11 to generate the bit stream.

Images