In-loop filters for video coding

Abstract

This invention provides a method and system for improving coding through bit-depth-based clipping operations in adaptive loop filtering. [Solution] The method includes the step of filtering each image sample using an adaptive loop filter having a filter length L and a set of filter coefficients, the step of identifying a set of related image samples for each image sample; identifying a clip value index and a corresponding filter coefficient for each of the related set of image samples; clipping the difference between each of the related set of image samples and each image sample based on the clip value index; and correcting each image sample using the clipped difference based on the corresponding filter coefficient.

Description

[Technical Field]

[0001] (Cross-reference of related applications) This application claims priority to U.S. Provisional Application No. 62 / 954,485, filed December 28, 2019, entitled “In-Loop Filters for Video Coding,” which is incorporated in its entirety by reference.

[0002] This application relates to the encoding and compression of video data in general, and more specifically to a method and system for improving the coding of chrominance and luminance components of image frames in a bitstream of video data. [Background technology]

[0003] Digital video is supported by a variety of electronic devices, including digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video teleconferencing devices, and video streaming devices. Such electronic devices transmit, receive, encode, decode, and / or store digital video data by implementing video compression and decompression standards defined by standards such as MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10 AVC (Advanced Video Coding), HEVC (High Efficiency Video Coding), and VVC (Versatile Video Coding). Generally, video compression involves reducing or removing redundancy inherent in video data by performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction. In block-based video coding, a video frame is divided into one or more slices, each slice having multiple video blocks, which may also be called coding tree units (CTUs). Each CTU may contain one coding unit (CU), or it may be recursively divided into smaller CUs until a predetermined minimum CU size is reached. Each CU (also named leaf CU) contains one or more transform units (TU) and one or more prediction units (PU). Each CU may be coded in intra-mode, inter-mode, or IBC mode. In an intra-coded (I) slice of a video frame, video blocks are coded using spatial prediction with respect to a reference sample in a neighboring block within the same video frame.The video blocks within an intercoded (P or B) slice in a video frame may use spatial predictions based on reference samples in neighboring blocks within the same video frame, or they may use temporal predictions based on reference samples in previous and / or future reference video frames.

[0004] For example, spatial or temporal prediction based on previously encoded reference blocks, such as neighboring blocks, yields a predicted block for the current video block being encoded. The process of finding the reference block can be achieved by a block-matching algorithm. Residual data representing the pixel difference between the current block being encoded and the predicted block is called a residual block or prediction error. Inter-encoded blocks are encoded according to a motion vector and residual block that points to the reference block in the reference frame forming the predicted block. The process of determining the motion vector is generally called motion prediction. Intra-encoded blocks are encoded according to an intra-prediction mode and residual block. For further compression, the residual block can be transformed from the pixel domain to a transformation domain, such as the frequency domain, yielding residual transformation coefficients, which are then quantized. The quantization transformation coefficients, initially placed in a two-dimensional array, may be scanned to generate a one-dimensional vector of transformation coefficients, which are then entropically encoded into a video bitstream to achieve even greater compression.

[0005] The encoded video bitstream is then stored on a computer-readable recording medium (e.g., flash memory) accessed by another electronic device with digital video capabilities, or transmitted directly to the electronic device via wired or wireless connection. The electronic device then performs video decompression (the reverse of the aforementioned video compression) by, for example, parsing the encoded video bitstream to obtain syntax elements from the bitstream, and, at least partially based on the syntax elements obtained from the bitstream, reconstructing the digital video data from the encoded video bitstream back into its original format, and then rendering the reconstructed digital video data on the electronic device's display.

[0006] The reconstructed video block is subjected to intra-loop filtering and then stored in a reference picture, which is used to encode other video blocks. Adaptive loop filters (ALFs) are applied to the chrominance and luminance components of the reconstructed video block, respectively. It would be beneficial to obtain a more efficient coding mechanism for encoding and decoding these color components while maintaining the image quality of the decoded video data. [Overview of the Initiative]

[0007] This application describes an implementation for encoding and decoding video data, and more specifically, an improvement to a method and system for coding chrominance and luminance samples of a video frame, relating to the application of bit-depth-based clipping operations in adaptive loop filtering. Each luminance or chrominance sample of a video frame is filtered based on a plurality of surrounding luminance or chrominance samples according to a respective adaptive loop filter (ALF) scheme. For each luminance or chrominance sample, the difference between the associated image sample and the sample is clipped to a respective dynamic range defined by a clipping boundary value equal to a power of 2 exponented by the respective clipping number.

[0008] In one embodiment, a method for coding video data includes the step of obtaining multiple image samples of a video frame from a bitstream. Each image sample corresponds to one of a luminance sample and a chrominance sample. The method further includes the step of filtering each of the multiple image samples using an adaptive in-loop filter having a filter length and a set of filter coefficients. The step of filtering each image sample further includes the steps of identifying a set of associated image samples at the filter length of each image sample, identifying a clip value index and a corresponding filter coefficient for each of the set of associated image samples, clipping the difference between each of the set of associated image samples and each image sample based on the clip value index, and modifying each image sample using the clipped difference of each of the set of associated image samples based on the filter coefficients. For each image sample, each clip value index corresponds to a clipping boundary value equal to a power of 2 with the clipping number as the exponent, where the clipping number is an integer. The method further includes the step of reconstructing a video frame using the multiple modified image samples.

[0009] In some embodiments, the difference between each associated image sample and each associated image sample is clipped by the following steps: determining the internal bit depth index (IBDI) of each image sample; determining a clipping boundary value for each set of associated image samples based on the IBDI and the respective clipping value index according to a predetermined clipping boundary value formula or table; and clipping the difference between each set of associated image samples and each associated image sample based on the respective clipping boundary value. Furthermore, in some embodiments, the predetermined clipping boundary value formula or table is stored locally in both the video encoder and the video decoder. For each image sample, the clipping value index of the associated image sample is obtained in a bitstream.

[0010] In another embodiment, the electronic device includes one or more processors and memory for storing instructions, and the one or more processors perform the method of coding video data as described above by executing instructions.

[0011] In another embodiment, instructions are stored in a non-temporary computer-readable storage medium, and one or more processors in an electronic device perform the method of coding video data as described above by executing the instructions.

[0012] The accompanying drawings included to provide a further understanding of the embodiments are incorporated herein and constitute part of this specification, and are intended to illustrate the embodiments described and to illustrate the basic principles together with the description. Similar reference figures refer to corresponding parts. [Brief explanation of the drawing]

[0013] [Figure 1] This block diagram shows exemplary video coding and decoding systems according to several embodiments. [Figure 2] A block diagram showing an exemplary video encoder according to some embodiments. [Figure 3] A block diagram showing an exemplary video decoder according to some embodiments. [Figure 4A] A schematic diagram of recursively dividing an image frame into video blocks of various sizes and shapes according to some embodiments. [Figure 4B] A schematic diagram of recursively dividing an image frame into video blocks of various sizes and shapes according to some embodiments. [Figure 4C] A schematic diagram of recursively dividing an image frame into video blocks of various sizes and shapes according to some embodiments. [Figure 4D] A schematic diagram of recursively dividing an image frame into video blocks of various sizes and shapes according to some embodiments. [Figure 4E] A schematic diagram of recursively dividing an image frame into video blocks of various sizes and shapes according to some embodiments. [Figure 5] A diagram showing a part of a video frame in a bitstream according to some embodiments. [Figure 6] A block diagram of an in-loop filter applied in a video encoder or a video decoder according to some embodiments. [Figure 7A] A diagram of an exemplary ALF filtering method in which a luminance sample is processed based on a set of related luminance samples by a luminance ALF according to some embodiments. [Figure 7B] A diagram of an exemplary ALF filtering method in which a chrominance sample is processed based on a set of related chrominance samples by a chrominance ALF according to some embodiments. [Figure 8] A diagram of an exemplary ALF filtering method having a clip value index for a set of related image samples of each image sample in a block of an image frame according to some embodiments. [Figure 9A] This is a table illustrating examples of predetermined clipping boundary values ​​according to several embodiments. [Figure 9B] This is a table illustrating examples of predetermined clipping boundary values ​​according to several embodiments. [Figure 9C] This is a table illustrating examples of predetermined clipping boundary values ​​according to several embodiments. [Figure 10] This diagram shows the data structure of values ​​clipped for an image sample, according to several embodiments. [Figure 11] This is a flowchart of a video coding method according to several embodiments. [Modes for carrying out the invention]

[0014] Specific embodiments are then referenced in detail, and these embodiments are shown in the accompanying drawings. In the following detailed description, many non-limiting and specific details are revealed to aid in understanding the subject matter presented herein. However, it will be apparent to those skilled in the art that various alternative forms may be used without departing from the claims, and that the subject matter may be implemented without these specific details. For example, it will be apparent to those skilled in the art that the subject matter presented herein may be implemented in many types of electronic devices with digital video capabilities.

[0015] Figure 1 is a block diagram illustrating an exemplary system 10 for performing parallel encoding and decoding of video blocks according to several embodiments. As shown in Figure 1, system 10 includes a source device 12 that generates and encodes video data to be later decoded by a destination device 14. The source device 12 and destination device 14 may comprise any of various electronic devices, including desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, the source device 12 and destination device 14 are equipped with wireless communication capabilities.

[0016] In some embodiments, the destination device 14 may receive the encoded video data to be decoded via link 16. Link 16 may comprise any type of communication medium or communication device capable of transferring the encoded video data from the source device 12 to the destination device 14. In one example, link 16 may comprise a communication medium that enables the source device 12 to directly transmit the encoded video data to the destination device 14 in real time. The encoded video data may be transmitted to the destination device 14 after being modulated according to a communication standard such as a wireless communication protocol. The communication medium may comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or other equipment that helps facilitate communication from the source device 12 to the destination device 14.

[0017] In some other embodiments, encoded video data may be transmitted from the output interface 22 to the recording device 32. Subsequently, the encoded video data in the recording device 32 may be accessed by the destination device 14 via the input interface 28. The recording device 32 may include any of various distributed or locally accessed data recording media, such as a hard disk drive, Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or other digital recording media suitable for storing encoded video data. In further examples, the recording device 32 may correspond to a file server or another intermediate recording device capable of holding encoded video data generated by the source device 12. The destination device 14 may access the stored video data by streaming or downloading it from the recording device 32. The file server may be any type of computer capable of storing encoded video data or transmitting encoded video data to the destination device 14. Exemplary file servers include web servers (for example, for websites), FTP servers, network-attached storage (NAS) devices, or local disk drives. The destination device 14 may access the encoded video data stored on the file server through any standard data connection, including a wireless channel suitable for accessing the encoded video data (e.g., Wi-Fi connection), a wired connection (e.g., DSL or cable modem), or a combination of both. Transmission of the encoded video data from the recording device 32 may be via streaming transmission, download transmission, or a combination of both.

[0018] As shown in Figure 1, the information source device 12 includes a video source 18, a video encoder 20, and an output interface 22. The video source 18 may include sources such as a video capture device, such as a video camera, a video archive containing previously captured video, a video supply interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video, or a combination of such sources. For example, if the video source 18 is a video camera of a security surveillance system, the information source device 12 and the destination device 14 may form a cameraphone or videophone. However, the embodiments described herein may be generally applicable to video coding and may be applicable to wireless and / or wired applications.

[0019] Captured, pre-captured, or computer-generated video can be encoded by the video encoder 20. The encoded video data can be transmitted directly to the destination device 14 through the output interface 22 of the source device 12. The encoded video data may also be stored in the recording device 32 (or alternatively) for later access by the destination device 14 or other devices for decoding and / or playback. The output interface 22 may further include a modem and / or transmitter.

[0020] The destination device 14 includes an input interface 28, a video decoder 30, and a display device 34. The input interface 28 may include a receiver and / or modem, which receives encoded video data via link 16. The encoded video data communicated via link 16 or supplied by the recording device 32 may include various syntax elements generated by the video encoder 20, which are used when the video decoder 30 decodes the video data. The encoded video data, which may contain such syntax elements, is transmitted over a communication medium and stored on a recording medium or file server.

[0021] In some embodiments, the display device 34 that the destination device 14 may include may be an integrated display device and an external display device configured to communicate with the destination device 14. The display device 34 displays the decoded video data to the user and may include any of various display devices such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0022] The video encoder 20 and video decoder 30 may operate based on intellectual property or industry standards such as VVC, HEVC, MPEG-4 Part 10 AVC (Advanced Video Coding), or extensions of these standards. It should be understood that this application is not limited to any specific video encoding / decoding standard and may be applicable to other video encoding / decoding standards. Generally, the video encoder 20 of the source device 12 is intended to be configured to encode video data according to either of these current or future standards. Similarly, the video decoder 30 of the destination device 14 is generally intended to be configured to decode video data according to either of these current or future standards.

[0023] The video encoder 20 and video decoder 30 can each be implemented as any of a variety of suitable encoding circuit configurations, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When implemented in part by software, the electronic device may store software instructions in a suitable non-temporary computer-readable medium and execute the instructions in hardware using one or more processors to perform the video encoding / decoding process disclosed herein. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, and any of them may be integrated as part of a combined encoder / decoder (CODEC) in each device.

[0024] Figure 2 is a block diagram illustrating an exemplary video encoder 20 according to several embodiments described in this application. The video encoder 20 may perform intra-predictive coding and inter-predictive coding of video blocks within a video frame. Intra-predictive coding relies on spatial prediction to reduce or eliminate spatial redundancy in video data within a given video frame or picture. Inter-predictive coding relies on temporal prediction to reduce or eliminate temporal redundancy in video data within adjacent video frames or pictures in a video sequence.

[0025] As shown in Figure 2, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transformation processing unit 52, a quantization unit 54, and an entropy encoding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a splitting unit 45, an intra-prediction processing unit 46, and an intra-block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transformation processing unit 60, and an adder 62 for reconstructing video blocks. An in-loop filter 66 may be located between the adder 62 and the DPB 64 and includes a deblocking filter for filtering block boundaries to remove grayscale artifacts from the reconstructed video. The in-loop filter 66 further includes a sample-adaptive offset (SAO) and an adaptive in-loop filter (ALF) for filtering the output of the adder 62 before it is input to the DPB 64 and used to encode other video blocks. The video encoder 20 may take the form of an immutable or programmable hardware unit, or it may be divided into one or more immutable or programmable hardware units.

[0026] The video data memory 40 can store video data encoded by the components of the video encoder 20. The video data in the video data memory 40 may be obtained, for example, from the video source 18. The DPB 64 is a buffer that records reference video data used by the video encoder 20 to encode the video data (for example, in intra-predictive coding mode or inter-predictive coding mode). The video data memory 40 and the DPB 64 may also be formed by any of the various recording devices. In various examples, the video data memory 40 may be on-chip together with the other components of the video encoder 20, or it may be off-chip relative to those components.

[0027] As shown in Figure 2, the internal splitting unit 45 of the prediction processing unit 41 splits the received video data into video blocks. This splitting may include dividing the video frame into slices, tiles, or other larger coding units (CUs) according to a predetermined splitting structure, such as a quadtree structure, associated with the video data. A video frame may be split into multiple video blocks (or sets of video blocks referred to as tiles). The prediction processing unit 41 may select one of several possible predictive coding modes for the current video block, such as one of several intra predictive coding modes or one of several inter predictive coding modes, based on the error result (e.g., coding rate or level of distortion). The prediction processing unit 41 may feed the resulting intra predictive coding block or inter predictive coding block to the adder 50 to generate a residual block, and may also feed this coding block to the adder 62 to reconfigure it for later use as part of a reference frame. The prediction processing unit 41 also supplies syntactic elements such as motion vectors, intra-mode indicators, segmentation information, and other such syntactic information to the entropy coding unit 56.

[0028] To select an appropriate intra-predictive coding mode for the current video block, an intra-predictive coding unit 46 within the prediction processing unit 41 may perform intra-predictive coding of the current video block with respect to one or more neighboring blocks in the same frame as the current block being coded to obtain a spatial prediction. A motion estimation unit 42 and a motion compensation unit 44 within the prediction processing unit 41 perform inter-predictive coding of the current video block with respect to one or more predicted blocks in one or more reference frames to obtain a temporal prediction. The video encoder 20 may perform multiple coding passes to select an appropriate coding mode for each block of video data, for example.

[0029] In some embodiments, the motion estimation unit 42 determines the inter-prediction mode for the current video frame by generating motion vectors that indicate the displacement of the prediction units (PUs) of the video blocks within the current video frame relative to the prediction blocks within a reference video frame, according to a predetermined pattern within a series of video frames. Motion prediction performed by the motion estimation unit 42 is the process of generating motion vectors that estimate the motion of the video blocks. The motion vectors may indicate, for example, the displacement of the PUs of the video blocks within the current video frame or picture relative to the prediction blocks within a reference frame (or other encoding unit), in relation to the current block being encoded within the current frame (or other encoding unit). The predetermined pattern may designate the video frames as P-frames or B-frames in a sequence. The intra-BC unit 48 may determine vectors for intra-BC encoding, such as block vectors, in a similar manner to how the motion estimation unit 42 determines the motion vectors for inter-prediction, or it may determine block vectors using the motion estimation unit 42.

[0030] The prediction block is a block of reference frames that is considered to correspond closely to the PU of the video block to be encoded in terms of pixel difference, and may be determined by the sum of absolute difference (SAD), the sum of square difference (SSD), or other difference criteria. In some embodiments, the video encoder 20 may calculate the values ​​of sub-integer pixel positions of the reference frame stored in the DPB64. For example, the video encoder 20 may interpolate the values ​​of 1 / 4 pixel positions, 1 / 8 pixel positions, or other fractional pixel positions of the reference frame. Thus, the motion estimation unit 42 can perform a motion search on the total pixel positions and fractional pixel positions to output a motion vector with fractional pixel precision.

[0031] The motion estimation unit 42 calculates a motion vector for the PU of a video block of an interpredictive coded frame by comparing the position of the predicted block of a reference frame selected from a first reference frame list (list 0) or a second reference frame list (list 1) with the position of the PU. Here, the first reference frame list or the second reference frame list each identify one or more reference frames stored in the DPB64. The motion estimation unit 42 sends the calculated motion vector to the motion compensation unit 44, and then to the entropy coding unit 56.

[0032] Motion compensation performed by the motion compensation unit 44 may include taking in or generating a predicted block based on a motion vector determined by the motion estimation unit 42. When the motion compensation unit 44 receives a motion vector for the PU of the current video block, it searches for the predicted block pointed to by the motion vector in one of the reference frame lists, takes the predicted block from the DPB 64, and transfers the predicted block to the adder 50. The adder 50 then forms a residual video block of pixel difference values ​​by subtracting the pixel values ​​of the predicted block provided by the motion compensation unit 44 from the pixel values ​​of the current video block to be encoded. The pixel difference values ​​forming the residual video block may include a luminance (luma) difference component, a chroma difference component, or both. The motion compensation unit 44 may also generate syntax elements related to the video block of the video frame that are used by the video decoder 30 when decoding the video block of the video frame. Syntax elements may include, for example, syntax elements that define motion vectors used to identify prediction blocks, optional flags indicating prediction modes, or other syntax information described herein. Although the motion estimation unit 42 and the motion compensation unit 44 can be almost integrated, they are shown separately for conceptual purposes.

[0033] In some embodiments, the intraBC unit 48 may capture a predicted block by generating vectors in a manner similar to that described above with respect to the motion estimation unit 42 and the motion compensation unit 44, wherein the predicted block is in the same frame as the current block being encoded, and the vectors are referred to as block vectors in contrast to motion vectors. In detail, the intraBC unit 48 may decide to use an intraprediction mode to encode the current block. In some examples, the intraBC unit 48 may encode the current block using various intraprediction modes, for example, during separate encoding passes, and analyze the performance of those intraprediction modes by rate-distortion analysis. The intraBC unit 48 may then select an appropriate intraprediction mode from among the various intraprediction modes tested to use to generate an intramode indicator. For example, the intraBC unit 48 may use rate-distortion analysis to calculate rate-distortion values ​​for the various intraprediction modes tested, and select the intraprediction mode with the best rate-distortion characteristics among the tested modes as the appropriate intraprediction mode to use. Rate-distortion analysis generally determines the bit rate (i.e., the number of bits) used to generate the encoded blocks, along with the amount of distortion (or error) between the encoded blocks and the original blocks before encoding that are encoded to generate those encoded blocks. The intraBC unit 48 may calculate the ratio of distortion to rate for various encoded blocks to determine an intra-prediction mode that shows the best rate-distortion value for that block.

[0034] In other examples, the intra-BC unit 48 may use the motion estimation unit 42 and the motion compensation unit 44, in whole or in part, to perform such functions for intra-BC prediction in accordance with the embodiments described herein. In either case, for intra-block copies, the prediction block may be a block that is considered to be closely corresponding to the block to be encoded in terms of pixel differences, and may be determined by the sum of absolute differences (SAD), the sum of squared differences (SSD), or other difference criteria. Identification of the prediction block may involve calculating the values ​​of sub-integer type pixel positions.

[0035] Whether the predicted block is from the same frame due to intra-prediction or from different frames due to inter-prediction, the video encoder 20 may form a residual video block by subtracting the pixel values ​​of the predicted block from the pixel values ​​of the current video block being encoded, thereby forming a pixel difference value. The pixel difference value forming the residual video block may include both luminance difference components and chrominance difference components.

[0036] The intra-prediction processing unit 46 may intra-predict the current video block as an alternative to the inter-prediction performed by the motion estimation unit 42 and the motion compensation unit 44, or the intra-block copy prediction performed by the intra-BC unit 48, as described above. More specifically, the intra-prediction processing unit 46 may decide to use an intra-prediction mode to encode the current block. To do so, the intra-prediction processing unit 46 may encode the current block using various intra-prediction modes during separate encoding passes, for example, and the intra-prediction processing unit 46 (or, in some examples, the mode selection unit) may select an appropriate intra-prediction mode to use from the tested intra-prediction modes. The intra-prediction processing unit 46 may supply the entropy encoding unit 56 with information representing the selected intra-prediction mode for that block. The entropy encoding unit 56 may encode information indicating the selected intra-prediction mode in the bitstream.

[0037] After the prediction processing unit 41 determines the prediction block for the current video block by either interpretation or intraprediction, the adder 50 generates a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be contained in one or more transformation units (TUs) and supplied to the transformation processing unit 52. The transformation processing unit 52 converts the residual video data into residual transformation coefficients using a transformation such as a discrete cosine transform (DCT) or a conceptually similar transformation.

[0038] The conversion processing unit 52 may send the resulting conversion coefficients to the quantization unit 54. The quantization unit 54 quantizes the conversion coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be changed by adjusting the quantization parameters. In some examples, the quantization unit 54 may then perform a scan of a matrix containing the quantized conversion coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0039] Following quantization, the entropy coding unit 56 entropy codes the quantization conversion coefficients into a video bitstream using, for example, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy coding (PIPE), or another entropy coding technique or technique. The coded bitstream is then transmitted to the video decoder 30, or can be recorded in the recording device 32 for later transmission to the video decoder 30, or for later retrieval by the video decoder 30. The entropy coding unit 56 may also entropy code the motion vector and other syntax elements relating to the current video frame being coded.

[0040] In order to generate reference blocks for predicting other video blocks, the inverse quantization unit 58 applies inverse quantization to reconstruct the residual video blocks in the pixel region, and the inverse transformation processing unit 60 applies inverse transformation. As described above, the motion compensation unit 44 can generate motion-compensated prediction blocks from one or more reference blocks of frames stored in the DPB 64. The motion compensation unit 44 may also apply one or more interpolation filters to the prediction blocks to calculate sub-integer pixel values ​​used for motion prediction.

[0041] The adder 62 generates a reference block for storage in the DPB 64, in addition to the reconstructed residual block, motion compensation prediction block generated by the motion compensation unit 44. The reference block can then be used by the intraBC unit 48, motion estimation unit 42, and motion compensation unit 44 as a prediction block for interpreting another video block in a subsequent video frame.

[0042] Figure 3 is a block diagram illustrating an exemplary video decoder 30 according to several embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transformation processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-prediction processing unit 84, and an intra-BC unit 85. The video decoder 30 may perform a decoding process that is entirely inverse to the encoding process described with respect to the video encoder 20 in relation to Figure 2. For example, the motion compensation unit 82 may generate prediction data based on motion vectors received from the entropy decoding unit 80, while the intra-prediction processing unit 84 may generate prediction data based on an intra-prediction mode indicator received from the entropy decoding unit 80.

[0043] In some examples, units of the video decoder 30 may be tasked with performing embodiments of the present invention. Also, in some examples, embodiments of the present disclosure may be divided among one or more units of the video decoder 30. For example, the intraBC unit 85 may perform embodiments of the present invention alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra prediction processing unit 84, and the entropy decoder unit 80. In some examples, the video decoder 30 may not include the intraBC unit 85, and the functionality of the intraBC unit 85 may be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0044] The video data memory 79 may store video data, such as an encoded video bitstream, which is decoded by other components of the video decoder 30. Video data stored in the video data memory 79 may be retrieved from the recording device 32, from a local video source such as a camera, by wired or wireless network communication of video data, or by accessing a physical data recording medium such as a flash drive or hard disk. The video data memory 79 may include an encoded picture buffer (CPB) that stores encoded video data from the encoded video bitstream. The decoded picture buffer (DPB) 92 of the video decoder 30 stores reference video data used by the video decoder 30 to encode video data (for example, in intra-predictive coding mode or inter-predictive coding mode). The video data memory 79 and DPB 92 may be formed by any of the various memory devices, including synchronous dynamic random access memory (DRAM), such as synchronous DRAM (SDRAM), magneto-resistive RAM (MRAM), resistive random access memory (RRAM), or other types of memory devices. For illustrative purposes, the video data memory 79 and DPB 92 are represented as two separate components of the video decoder 30 in Figure 3. However, it will be apparent to those skilled in the art that the video data memory 79 and DPB 92 may be provided by the same memory device or separate memory devices. In some examples, the video data memory 79 may be on-chip with the other components of the video decoder 30, or off-chip relative to those components.

[0045] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of encoded video frames and associated syntax elements. The video decoder 30 may receive syntax elements at the video frame level and / or video block level. The entropy decoding unit 80 of the video decoder 30 entropy-decodes the bitstream to generate quantization coefficients, motion vectors or intra-prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then transfers the motion vectors and other syntax elements to the prediction processing unit 81.

[0046] When a video frame is encoded as an intra-predictive coding (I) frame or for an intra-coded prediction block in another type of frame, the intra-predictive processing unit 84 of the prediction processing unit 81 may generate prediction data for the video block of the current video frame based on the signaled intra-predictive mode and reference data from previously decoded blocks of the current frame.

[0047] When a video frame is encoded as an interpredictive coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the video block of the current video frame based on the motion vector and other syntax elements received from the entropy decoding unit 80. Each prediction block may be generated from a reference frame inside one of the reference frame lists. The video decoder 30 may configure the reference frame lists, list 0 and list 1, using default configuration techniques based on reference frames stored in the DPB 92.

[0048] In some examples, when a video block is encoded according to the intraBC mode described herein, the intraBC section 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from the entropy decoding unit 80. The prediction block may be located within a reconfigured region of the same picture as the current video block defined by the video encoder 20.

[0049] The motion compensation unit 82 and / or the intra-BC unit 85 determine prediction information about the video block of the current video frame by analyzing the motion vector and other syntax elements, and then use the prediction information to generate a prediction block for the current video block to be decoded. For example, the motion compensation unit 82 uses some of the received syntax elements to determine the prediction mode used to encode the video block of the video frame (e.g., intra-predict or inter-predict), the inter-predict frame type (e.g., B or P), one or more configuration pieces of reference frame list for the frame, the motion vector of each inter-predict coded video block in the frame, the inter-predict state of each inter-predict coded video block in the frame, and other information for decoding the video block in the current video frame.

[0050] Similarly, the intraBC unit 85 may use some of the received syntax elements, such as flags, to determine that the current video block was predicted using intraBC mode, the configuration information of the video block of the frame that is to be stored in the DPB 92 within the reconstructed region, the block vector of each intraBC predicted video block of the frame, the intraBC predicted state of each intraBC predicted video block of the frame, and other information for decoding the video block in the current video frame.

[0051] The motion compensation unit 82 may also perform interpolation using an interpolation filter, such as the one used by the video encoder 20 to calculate the interpolation values ​​of the sub-integer pixels of the reference block during the encoding of the video block. In this case, the motion compensation unit 82 may determine the interpolation filter used by the video encoder 20 from the received syntax elements and use the interpolation filter to generate the predicted block.

[0052] The inverse quantization unit 86 inverse quantizes the quantization transformation coefficients that are given in the bitstream and entropy-decoded by the entropy decoding unit 80, using the same quantization parameters that were calculated by the video encoder 20 to determine the degree of quantization for each video block in the video frame. The inverse transformation processing unit 88 applies an inverse transformation to the transformation coefficients, such as an inverse DCT, an inverse integer transformation, or a conceptually similar inverse transformation process, in order to reconstruct the residual blocks in the pixel region.

[0053] After the motion compensation unit 82 or the intra-BC unit 85 generates a prediction block for the current video block based on vectors and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by summing the residual block from the inverse transformation processing unit 88 with the corresponding prediction block generated by the motion compensation unit 82 and the intra-BC unit 85. An in-loop filter 94 may be placed between the adder 90 and the DPB 92 and includes a deblocking filter for filtering block boundaries to remove grayscale artifacts from the decoded video block. The in-loop filter 94 further includes an SAO filter and an ALF for filtering the decoded video block output by the adder 90. The decoded video block in a given frame is then stored in the DPB 92, which stores a reference frame to be used for subsequent motion compensation of the next video block. The DPB 92 or a separate memory device may also store the decoded video for later presentation on a display device such as the display device 34 in Figure 1.

[0054] In a typical video coding process, a video sequence generally contains an ordered set of frames or pictures. Each frame may contain three sample arrays, represented as SL, SCb, and SCr. SL is a two-dimensional array of luminance (luma) samples. SCb is a two-dimensional array of Cb chrominance samples. SCr is a two-dimensional array of Cr chrominance samples. In other cases, a frame may be black and white and therefore contain only one two-dimensional array of luminance samples.

[0055] As shown in Figure 4A, the video encoder 20 (more specifically the splitting unit 45) generates an encoded representation of a frame by first splitting the frame into a set of encoded tree units (CTUs). A video frame may contain an integer number of CTUs that are sequentially ordered in the raster scan order from left to right and top to bottom. Each CTU is the largest logical encoded unit, and the width and height of the CTUs are signaled by the video encoder 20 in the sequence parameter set such that all CTUs in the video sequence have the same size, one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that this application is not necessarily limited to a specific size. As shown in Figure 4B, each CTU may contain one encoded tree block (CTB) consisting of luminance (luma) samples, an encoded tree block consisting of two corresponding chrominance samples, and syntax elements used to encode the samples in the encoded tree block. The syntax elements describe the characteristics of various types of units in the pixel coding block, and a method by which the video decoder 30 can reconstruct the video sequence, including interprediction or intraprediction, intraprediction mode, motion vector, and other parameters. For a black and white picture or a picture with three separate color planes, the CTU may include a single coding tree block and syntax elements used to code samples in the coding tree block. The coding tree block may be an N×N block of samples.

[0056] To achieve better performance, the video encoder 20 may recursively perform tree partitioning on the coding tree block of the CTU, such as binary-tree partitioning, ternary-tree partitioning, quad-tree partitioning, or a combination thereof, to divide the CTU into smaller coding units (CUs). As shown in Figure 4C, the 64x64 CTU 400 is first divided into four smaller CUs, each with a block size of 32x32. Of the four smaller CUs, CU410 and CU420 are each divided into four CUs with a block size of 16x16. The two 16x16 CUs, 430 and 440, are further divided into four CUs, each with a block size of 8x8. Figure 4D represents a quadtree data structure showing the final result of the partitioning process of CTU400 as shown in Figure 4C, where each leaf node of the quadtree corresponds to one CU of each size ranging from 32x32 to 8x8. Each CU may contain, similar to the CTU shown in Figure 4B, an encoded block (CB) of luminance samples, two corresponding encoded blocks of color difference samples of the same-sized frame, and syntax elements used to encode the samples in the encoded block. For a black and white picture or a picture with three separate color planes, the CU may contain a single encoded block and a syntax structure used to encode the samples in the encoded block. Note that the quadtree partitions shown in Figures 4C and 4D are for illustrative purposes only, and a single CTU can be partitioned into CUs based on quadtree / ternary / binary partitions to suit various local characteristics. In a composite tree structure, one CTU is divided by a quadtree structure, and each leaf CU of the quadtree can be further divided by binary and ternary tree structures. As shown in Figure 4E, there are five types of divisions: quadrifting, horizontal 2-partitioning, vertical 2-partitioning, horizontal 3-partitioning, and vertical 3-partitioning.

[0057] In some embodiments, the video encoder 20 may further divide the encoded block of the CU into one or more M×N prediction blocks (PB). A prediction block is a rectangular (square or non-square) block of samples to which the same (inter or intra) prediction is applied. A prediction unit (PU) of the CU may include a prediction block for luminance samples, two corresponding prediction blocks for chrominance samples, and syntax elements used to predict the prediction blocks. For a black and white picture or a picture with three separate color planes, a PU may include a single prediction block and a syntax structure used to predict the prediction block. The video encoder 20 may generate predicted luminance, luminance-related Cb and Cr blocks, and Cb and Cr prediction blocks in each PU of the CU.

[0058] The video encoder 20 may use intra-prediction or inter-prediction to generate prediction blocks for the PU. If the video encoder 20 uses intra-prediction to generate prediction blocks for the PU, the video encoder 20 may generate prediction blocks for the PU based on decoded samples of frames associated with the PU. If the video encoder 20 uses inter-prediction to generate prediction blocks for the PU, the video encoder 20 may generate prediction blocks for the PU based on decoded samples of one or more frames other than frames associated with the PU.

[0059] The video encoder 20 may generate predicted luminance blocks, predicted Cb blocks, and predicted Cr blocks for one or more PUs in the CU, and then generate a luminance residual block for the CU by subtracting the predicted luminance block for the CU from the original luminance coded block for the CU, such that each sample in the luminance residual block for the CU represents the difference between a luminance sample in one of the predicted luminance blocks for the CU and a corresponding sample in the original luminance coded block for the CU. Similarly, the video encoder 20 may generate a Cb residual block and a Cr residual block for the CU, respectively, such that each sample in the Cb residual block for the CU represents the difference between a Cb sample in one of the predicted Cb blocks for the CU and a corresponding sample in the original Cb coded block for the CU, and each sample in the Cr residual block for the CU may represent the difference between a Cr sample in one of the predicted Cr blocks for the CU and a corresponding sample in the original Cr coded block for the CU.

[0060] Furthermore, as shown in Figure 4C, the video encoder 20 uses a quadtree decomposition to decompose the luminance, Cb, and Cr residual blocks of the CU into one or more luminance, Cb, and Cr transformation blocks. A transformation block is a rectangular (square or non-square) block of samples to which the same transformation is applied. A transformation unit (TU) of the CU may include a transformation block of luminance samples, two corresponding transformation blocks of chrominance samples, and a syntax element used to predict the transformation block samples. Thus, each TU of the CU may be associated with a luminance transformation block, a Cb transformation block, and a Cr transformation block. In some examples, a luminance transformation block associated with a TU may be a subblock of the luminance residual block of the CU. A Cb transformation block may be a subblock of the Cb residual block of the CU. A Cr transformation block may be a subblock of the Cr residual block of the CU. In a black and white picture or a picture with three separate color planes, a TU may include a single transformation block and a syntax structure used to transform the samples of the transformation block.

[0061] The video encoder 20 may generate a luminance coefficient block for TU by applying one or more transformations to the luminance transformation block of TU. The coefficient block may be a two-dimensional array of transformation coefficients. The transformation coefficients may be scalar quantities. The video encoder 20 may generate a Cb coefficient block for TU by applying one or more transformations to the Cb transformation block of TU. The video encoder 20 may generate a Cr coefficient block for TU by applying one or more transformations to the Cr transformation block of TU.

[0062] The video encoder 20 may quantize coefficient blocks (e.g., luminance coefficient blocks, Cb coefficient blocks, or Cr coefficient blocks) after generating them. Quantization generally refers to the process by which transformation coefficients are quantized in order to somehow reduce the amount of data used to represent the transformation coefficients, resulting in further compression. After quantizing the coefficient blocks, the video encoder 20 may entropy encode the syntax elements representing the quantized transformation coefficients. For example, the video encoder 20 may perform context-adaptive binary arithmetic coding (CABAC) on the syntax elements representing the quantized transformation coefficients. Finally, the video encoder 20 may output a bitstream containing a set of bits that form a representation of the encoded frame and associated data, which is stored in the recording device 32 or transmitted to the destination device 14.

[0063] The video decoder 30, after receiving the bitstream generated by the video encoder 20, may parse the bitstream to obtain syntax elements from it. The video decoder 30 may reconstruct frames of video data based at least partially on the syntax elements obtained from the bitstream. The process of reconstructing video data is generally the reverse of the encoding process performed by the video encoder 20. For example, the video decoder 30 may reconstruct residual blocks associated with the TU of the current CU by performing an inverse transform on the coefficient blocks associated with the TU of the current CU. The video decoder 30 also reconstructs the encoded blocks of the current CU by adding samples of the predicted blocks for the PU of the current CU to samples of the transformed blocks for the corresponding TU of the current CU. The video decoder 30 may reconstruct the frame after reconstructing the encoded blocks for each CU of the frame.

[0064] As mentioned above, video coding achieves video compression primarily using two modes: intra-frame prediction (i.e., intra-prediction) and inter-frame prediction (i.e., inter-prediction). Palette-based coding is another coding method adopted by many video coding standards. Palette-based coding is particularly well-suited for coding content generated on a screen, in which the video coder (e.g., video encoder 20 or video decoder 30) forms a color palette table representing the video data of a given block. The palette table contains the most dominant (e.g., frequently used) pixel values ​​in a given block. Pixel values ​​that are not frequently represented in the video data of a given block are either not included in the palette table or are included in the palette table as avoided colors.

[0065] Each entry in the palette table contains an index relating to the corresponding pixel value in the palette table. The palette index for a sample within a block can be encoded to indicate the entry in the palette table used to predict or reconstruct the sample. This palette mode begins with the process of generating a palette predictor for the first block of such a grouping of pictures, slices, tiles, or video blocks. As described below, palette predictors for subsequent video blocks are generally generated by updating the previously used palette predictor. For illustrative purposes, it is assumed that the palette predictor is defined at the picture level. In other words, a picture may contain multiple encoded blocks, each with its own palette table, but there is one palette predictor for the entire picture.

[0066] To reduce the number of bits required to transmit palette entries in a video bitstream, a video decoder may utilize a palette predictor to determine new palette entries in the palette table used to reconstruct a video block. For example, a palette predictor may contain palette entries from a previously used palette table, or it may be initialized with the most recently used palette table by containing all entries from the most recently used palette table. In some embodiments, the palette predictor may contain fewer entries than all entries from the most recently used palette table, and then incorporate some entries from other previously used palette tables. The size of the palette predictor may be the same as, larger than, or smaller than the size of the palette table used to code different blocks. In one example, the palette predictor is implemented as a first-in, first-out (FIFO) table containing 64 palette entries.

[0067] To generate a palette table for blocks of video data from a palette predictor, the video decoder may receive a 1-bit flag from the encoded video bitstream for each input of the palette predictor. The 1-bit flag may have a first value (e.g., 1 in binary) indicating that the corresponding input of the palette predictor is included in the palette table, or a second value (e.g., 0 in binary) indicating that the corresponding input of the palette predictor is not included in the palette table. If the size of the palette predictor is larger than the palette table used for blocks of video data, the video decoder may stop receiving further flags once the maximum size of the palette table is reached.

[0068] In some embodiments, some entries in the palette table may be signaled directly in the encoded video bitstream rather than being determined using palette predictors. For such entries, the video decoder may receive three distinct m-bit values ​​from the encoded video bitstream, indicating the pixel values ​​for the luminance component and two chrominance components associated with the entry, where m represents the bit depth of the video data. Multiple m-bit values ​​are required for palette entries that are signaled directly, whereas palette entries derived from palette predictors require only a single-bit flag. Therefore, signaling some or all of the palette inputs using palette predictors can significantly reduce the number of bits required to signal the inputs to a new palette table, thereby improving the overall encoding efficiency of palette-mode coding.

[0069] In many cases, the palette predictor for a single block is determined based on the palette table used to encode one or more previously encoded blocks. However, when encoding the first encoded tree unit in a picture, slice, or tile, the palette table of previously encoded blocks may not be available. Therefore, it is not possible to generate a palette predictor using entries from a previously used palette table. In such cases, a set of palette predictor initialization specifiers, which are the values ​​used to generate the palette predictor when a previously used palette table is unavailable, may be signaled in the sequence parameter set (SPS) and / or picture parameter set (PPS). The SPS generally refers to a syntax structure of syntax elements that fit a series of consecutive encoded video pictures called an encoded video sequence (CVS), as determined by the content of the syntax elements found in the PPS, which are referenced by the syntax elements found in each slice segment header. The PPS generally refers to a syntax structure of syntax elements that fit one or more individual pictures within the CVS, as determined by the syntax elements found in each slice segment header. Therefore, SPS is generally considered a higher level of syntax structure than PPS, meaning that the syntax elements contained in SPS generally do not change as frequently and are more suitable for larger portions of video data compared to the syntax elements contained in PPS.

[0070] Figure 5 shows a portion of a video frame 500 in a bitstream according to several embodiments. The video frame 500 contains multiple pixels, each pixel being composed of multiple color elements (e.g., blue, green, and red). In video encoding and decoding, the color information of multiple pixels is represented by multiple luminance samples 502 and multiple chrominance samples 504. Each of the multiple pixels corresponds to its own luminance sample 502, and each luminance sample 502 corresponds to each pixel in the video frame 500. Each chrominance sample 504 corresponds to each set of luminance samples 502 according to the subsampling scheme. Each luminance sample 502 has a luminance component Y', and each chrominance sample 504 has a blue difference chrominance component Cb and a red difference chrominance component Cr. The subsampling scheme for the luminance and chrominance components (Y':Cb:Cr) has a ratio of three parts (e.g., 4:1:1, 4:2:0, 4:2:2, 4:4:4, 4:4:0). Specifically, the luminance samples 502 and chrominance samples 504 of video frame 500 conform to a subsampling scheme with a 3-part ratio equal to 4:1:1, and on average, every four luminance samples 502 corresponds to one chrominance sample 504 having a blue chrominance component Cb and a red chrominance component Cr.

[0071] Figure 6 is a block diagram of an in-loop filter 600 applied in a video encoder 20 or video decoder 30 according to several embodiments. In video encoding or coding, each of the luminance samples 502 and chrominance samples 504 is reconstructed from the residual block of the video frame 500 and filtered by a deblocking filter, one or more sample-adaptive offset (SAO) filters 602, and one or more adaptive loop filters (ALF) 604 of the in-loop filter 600 (e.g., in-loop filters 66 and 94 in Figures 2 and 3) to remove artifacts. The filtered luminance samples 606 and chrominance samples 608 are stored in a decoded image buffer 64 or 92 and used for encoding or decoding other video blocks in the video frame 500. In some embodiments, each of the deblocking filter, SAO filter 602, and ALF 604 is configured to filter luminance samples 502 or chrominance samples 504 based on samples of the same type, for example, filtering each luminance sample 502 based on each set of adjacent luminance samples 502 and filtering each luminance sample 502 based on each set of adjacent chrominance samples 504. In some embodiments, the in-loop filter 600 further includes a cross-component filter 610 configured to filter each chrominance sample 504 based on one or more luminance samples 502 adjacent to each chrominance sample 504. Conversely, in some embodiments, the in-loop filter 600 includes an alternative cross-component filter configured to filter each luminance sample 502 based on one or more chrominance samples 504 adjacent to each luminance sample 502.

[0072] Specifically, the video encoder 20 or decoder 30 acquires multiple luminance samples 502 and multiple chrominance samples 504 of the video frame 500. Each luminance sample 502 has its own luminance value, and each chrominance sample 504 has its own luminance value. The SAO filter 602 compensates for each of the multiple luminance samples 502 and multiple chrominance samples 504. Specifically, the SAO filters 602A, 602B, and 602C compensate for the chrominance component Cb of the blue difference in the luminance sample 502 and the chrominance component Cr of the red difference in the chrominance sample 504, respectively. The ALF 604 is coupled to the SAO filter 602. Each of the compensated luminance samples 612 updates luminance sample 606 using luminance ALF 604A based on the set of adjacent compensated luminance samples 612, and each of the compensated chrominance samples 614A and 614B updates chrominance sample 608A or 608B using chrominance ALF 604B based on the set of adjacent compensated chrominance samples 614.

[0073] In some embodiments, the cross-component filter 610 is configured to generate a chrominance improvement value 616 for each chrominance sample 504 based on a set of luminance samples 502. Each chrominance sample 504 is updated using the chrominance improvement value 616; that is, the chrominance value of each chrominance sample 504 is improved using the chrominance improvement value 616. Each updated chrominance sample 608 is stored in relation to the video frame 500. In some embodiments, the cross-component filter 610 includes a first cross-component filter 610A configured to generate a first improvement value 616A and a second cross-component filter 610 configured to generate a second improvement value 616B. The blue difference chrominance component 618A and the red difference chrominance component 618B are updated separately using the first improvement value 616A and the second improvement value 616B, respectively, to output a first improved chrominance value 608A and a second improved chrominance value 608B.

[0074] Each of the deblocking filter, SAO filter, and ALF of the in-loop filter 600 includes one or more in-loop filter coefficients, and the cross-component ALF 610 also includes multiple cross-component filter coefficients. The in-loop filter coefficients and cross-component filter coefficients are signaled in a set of fitting parameters (APS). In one example, the APS carries and signals multiple sets of luminance filter coefficients and clip value indices (e.g., up to 25 sets), and multiple sets of chrominance filter coefficients and clip value indices (e.g., up to 8 sets). The APS is a bitstream and is transmitted along with the video frame 500 from the video encoder 20 to the video encoder 30; i.e., the APS is the overhead of bitstream transmission. In some embodiments, filter coefficients for separate classifications with respect to the luminance component of luminance sample 502 are combined to reduce the overhead of bitstream transmission. In one example, the indices of the APS used for image slicing are signaled in the corresponding slice header.

[0075] Figure 7A is an exemplary ALF filtering scheme 700 in which, according to several embodiments, a luminance sample 502A is processed by a luminance ALF 604A based on a set of associated luminance samples 502B, and Figure 7B is an exemplary ALF filtering scheme 750 in which, according to several embodiments, a chrominance sample 504A is processed by a chrominance ALF 604B based on a set of associated chrominance samples 504B. The luminance ALF 604A has a rhombus filter shape (for example, a 7x7 rhombus shape), and for each 4x4 block, a plurality of predetermined filters (for example, 25 filters with predetermined filter coefficients) are selected based on the direction and function of the local gradient. Each square in Figure 7A represents a luminance sample 502 having a rhombus shape and labeled with the corresponding filter coefficients (C0~C12) of the luminance ALF 604A. To combine 25 luminance samples 502 using luminance ALF604A, a total of 13 filter coefficients (C0 to C12) are applied symmetrically to the luminance samples 502A. Similarly, chrominance ALF604B has a diamond-shaped filter (e.g., a 5x5 diamond shape) and is selected from a plurality of predetermined filters (e.g., eight filters with predetermined filter coefficients). Each square in Figure 7B represents a chrominance sample 504 with a diamond shape, labeled with the corresponding filter coefficients (C0 to C6) of chrominance ALF604B. To combine 13 chrominance samples 504 in chrominance ALF604B, a total of 7 filter coefficients (C0 to C6) are applied symmetrically to the chrominance samples 504A.

[0076] For example, when adaptive loop filtering is enabled for CTB, each image sample R(i,j) inside the CU (e.g., luminance sample 502A, chrominance sample 504A) is filtered to obtain the sample value R'(i,j) given by the following equation.

[0077]

number

[0078] For clipping in adaptive loop filtering, each image sample encompasses a set of associated image samples with a filter length L of the ALF604. Referring to Figure 7A, the associated luminance sample 502B of each luminance sample 502A includes the three rows of luminance samples above each luminance sample 502A (i.e., nine luminance samples), the three rows of luminance samples below each luminance sample 502A (i.e., nine luminance samples), the three luminance samples to the left of each luminance sample 502A, and the three luminance samples to the right of each luminance sample 502A. Referring to Figure 7B, the associated color difference sample 504B of each color difference sample 504A includes the two rows of color difference samples above each color difference sample 504A (i.e., four color difference samples), the two rows of color difference samples below each color difference sample 50rA (i.e., four color difference samples), the two color difference samples to the left of each color difference sample 504A, and the two color difference samples to the right of each color difference sample 504A.

[0079] In some embodiments, the ALF filter parameters are transmitted via an APS containing clip value indices representing the aforementioned clipping parameters c(k,l). A single APS may transmit up to 25 sets of luminance filter coefficients (e.g., the C0-C12 set in Figure 7A) and clipping value indices, and up to 8 sets of chromatic difference filter coefficients (e.g., the C0-C6 set in Figure 7B) and clipping value indices. To reduce the overhead of transmitted bits, separate classifications of filter coefficients for luminance samples 502A may be combined. The slice header transmits the index of the APS used for the current slice. The clipping value indices decoded from the APS allow for the determination of clipping boundary values ​​using a predetermined clipping boundary value formula or table. These boundary clipping values ​​depend on the internal bit depth and define the dynamic range in which the difference between the relevant image sample and the set of image samples 502A or 504A is clipped. In some embodiments, separate predetermined clipping boundary value formulas or tables are applied to the luminance sample 502A and the color difference sample 504A.

[0080] In some embodiments, multiple APS indices (e.g., seven indices) may be signaled in the slice header to select a subset of multiple corresponding luminance and chrominance filter sets for adaptive loop filtering in the current slice. This adaptive loop filtering process may be controlled at the CTB level. A flag is signaled to indicate whether adaptive loop filtering is applied to the luminance CTB. Depending on the determination that the flag enables adaptive loop filtering, the luminance CTB selects a luminance filter set from multiple (e.g., 16) luminance filter sets. To indicate the luminance filter set to be applied, the index of the luminance filter set is signaled to the luminance CTB. Each of the multiple filter sets is predefined and hardcoded in both the encoder 20 and the decoder 30, and only the index of the luminance filter set needs to be transmitted in the bitstream carrying the video frame 500. Similarly, for chrominance sample 504A, when the flag enables adaptive loop filtering, the index of the chrominance filter set is signaled in the slice header to select one of multiple chrominance filter sets for adaptive loop filtering in the current slice. If there are multiple color difference filter sets in the APS, the index of each color difference filter set is transmitted at the CTB level for each color difference CTB. Each filter set is locally stored in the video encoder or video decoder, along with the filter coefficients f(k,l) and clipping parameter c(k,l), or the index of the filter coefficients f(k,l) and clipping parameter c(k,l).

[0081] In some embodiments, the filter coefficients are quantized using a norm equal to 128. The values ​​of the filter coefficients with a non-neutral point are [-2 7 ~2 7 Bitstream fitting is applied so that the result falls within the range of -1. The filter coefficients with a neutral point are estimated to be 128, without being transmitted in the bitstream.

[0082] Figure 8 shows an exemplary ALF filtering scheme 800 having clip value indices for a set of associated image samples 804 for each image sample 802 in a block of image frames, according to several embodiments. The ALF 604 has a filter length L for each image sample 802 (e.g., luminance sample 502A, chrominance sample 504A) and corresponds to a set of filter coefficients (e.g., C0 to C13 for luminance sample 502A). The set of associated image samples 804 is identified by the filter length of each image sample 802. For each of the sets of associated image samples 804 (e.g., luminance sample 502B corresponding to filter coefficients C1 to C12 in Figure 7), the respective clip value index and corresponding filter coefficient are identified. For example, each associated image sample 804 in Figure 8 has a clip value index of 0, 1, 2, or 3.

[0083] Each clip value index corresponds to a respective clipping boundary value M equal to a power of 2 with the respective clipping number as the exponent, where each clipping number i is an integer. ALF604 clips the difference between each of the associated set of image samples 804 and each image sample 802 to a dynamic range DR defined by the respective clipping boundary value M associated with each clip value index. Each image sample 802 (e.g., luminance sample 502A, chrominance sample 504A) is modified based on its respective filter coefficient using the clipped difference of each of the associated set of image samples 804, so that an image frame can be reconstructed using multiple modified image samples 802. When the boundary values ​​are restricted to powers of 2 with integer exponents, the step of clipping the difference between the associated image sample 804 and each image sample 802 involves only logical AND and / or logical OR operations. By these means, this clipping operation does not involve any comparison operations on each image sample 802, thereby reducing the amount of computation required for adaptive loop filtering and saving computational resources from coding the corresponding video frames.

[0084] In some embodiments, with respect to a first block of image samples, a single set of clip value indices is determined for all image samples 802 in the first block. The set of clip value indices is transferred from the video encoder 20 to the video decoder 30 along with the block of image samples, so that the ALF 604 can process each of the image samples 802 in the first block based on the same single set of clip value indices. An exemplary block is an entire video frame, slice, brick, tile, tile group, coded tree, or any other coded unit. Furthermore, in some embodiments, the video frame includes a second block distinct from the first block. The second set of clip value indices is used for adaptive loop filtering of the image samples 802 in the second block. The second set of clip value indices is determined independently of the single set of clip value indices used for the first block of images.

[0085] Figures 9A to 9C show three examples of predetermined clipping boundary value tables 900A to 900C according to several embodiments. Each of the predetermined clipping boundary value tables 900 associates a set of clip value indices 902 with a set of clipping boundary values ​​(M) 904, depending on a set of internal bit depth indices (IBDI) 906. Every clipping boundary value M in these tables 900A to 900C is equal to a power of 2 with the respective clipping number i as the exponent. Each clipping boundary value table 900 is applied to the ALF604 as a reference for determining the clipping boundary value M for each clip value index. For example, for the first image sample 802 of a video frame, when the difference between the first image sample 802 and one of its associated image samples 804A is clipped, the associated image sample 804A has a clip value index 902A (i.e., 0), and the IBDI 906A (e.g., IBDI=10) can be determined from the first image sample 802 being filtered. Each clipping boundary value 904A (i.e., 1024) is identified from a given clipping boundary value table 900A at the intersection of the column corresponding to the clip value index 902A and the row corresponding to the IBDI 906A.

[0086] Each clipping boundary value 904 is equal to a power of 2 with the respective clipping number i as the exponent. For the same clipping value index (e.g., 902A), each clipping number i is an integer that increases linearly with IBDI906. Each boundary value 904(M) is expressed as a function of the respective clipping number i by the following equation: M=2 i (1)

[0087] In some embodiments, with respect to each associated image sample 804 used for adaptive loop filtering of the first image sample 802, the clipped difference between the associated image sample 804 and the first image sample 802 is in the range [-M, M-1], i.e., [-2i , 2 i -1]. Alternatively, in some embodiments, the clipped difference between the associated image sample 804 and the first image sample 802 is in the range [-M + 1, M], i.e., [-2 i + 1, 2 i . Alternatively, in some embodiments, the clipped difference between the associated image sample 804 and the first image sample 802 is in the range [-M + 1, M - 1], i.e., [-2 i + 1, 2 i -1]. Alternatively, in some embodiments, the clipped difference between the associated image sample 804 and the first image sample 802 is in the range [-M, M], i.e., [-2 i , 2 i .

[0088] In some embodiments, a predetermined clipping boundary value table 900 used for filtering blocks of image sample 802 is acquired in the bitstream carrying the blocks, allowing the table 900 to be updated for each block of image sample 802. An exemplary block may be an entire video frame, slice, brick, tile, tile group, coded tree, or any other coded unit. Alternatively, two copies of the same predetermined clipping boundary value table 900 used in adaptive loop filtering are stored separately in the video encoder 20 and the video decoder 30. During the filtering of blocks of image sample 802, the predetermined clipping boundary value table 900 is retrieved from local memory pre-stored with one or more separate predetermined clipping boundary value tables (e.g., a subset or all of tables 900A-900C), rather than being received in the video frame. Each image sample 802 in a single block shares a set of clip value indices and boundary value table indicators. A set of clip value indices and boundary value table indicators are acquired in a bitstream in relation to the block of image sample 802, and one or more clipping boundary value tables are stored locally. Based on the boundary value table indicators, one of the clipping boundary value tables 900 is selected and applied to a clipping operation to filter each image sample 802 in the block.

[0089] The predetermined clipping boundary value table 900A is expressed by the following formula.

[0090]

number

[0091] The clipping boundary values ​​for a clipping value index of 2 differ between the predetermined clipping boundary value tables 900A and 900B. The predetermined clipping boundary value table 900B is expressed by the following formula.

[0092]

number

[0093] In some embodiments, the clipping boundary value table 900B is not stored in local memory, but equation (3) is stored. For each image sample 802, and for each associated image sample 804, the clipping boundary value is determined based on the IBDI and the respective clip value index according to the predetermined clipping boundary value equation (3) described above.

[0094] The clipping boundary values ​​for a clipping value index of 1 differ between the predetermined clipping boundary value tables 900A and 900C. The predetermined clipping boundary value table 900C is expressed by the following formula.

[0095]

number

[0096] In some embodiments, the clipping boundary value table 900C is not stored in local memory, but formula (4) is stored. For each image sample 802, and for each associated image sample 804, the clipping boundary value is determined based on the IBDI and the respective clip value index according to the predetermined clipping boundary value formula (4) described above.

[0097] In some embodiments, the bitstream includes a plurality of image samples 802 of the video frame. The plurality of image samples 802 includes a subset of luminance samples 502A and a subset of chrominance samples 504A. The subset of luminance samples 502A corresponds to a first set of clip value indices that define the respective clip value indices for each of the associated sets of luminance samples 502B at a first filter length L1 for each luminance sample 502A. The subset of chrominance samples 504A corresponds to a second set of clip value indices that define the respective clip value indices for each of the associated sets of chrominance samples 504B at a second filter length L2 for each chrominance sample 504A. The second set of clip value indices is different from the first set of clip value indices. Furthermore, in some embodiments, each set of the first and second sets of clip value indices corresponds to a table or expression of clipping boundary values, associating each clip value index in each set of clip value indices with its respective clipping boundary value based on IBDI. In addition, in some embodiments, the table or expression of clipping boundary values ​​is selected from a predetermined number of the table or expression of clipping boundary values. Alternatively, in some embodiments, the first and second sets of clip value indices correspond to a first table / expression of clipping boundary values ​​and a second table / expression of values ​​different from the first table / expression of clipping boundary values.

[0098] Figure 10 shows the data structure of the value 1000 to be clipped for image sample 802 according to several embodiments. Image sample 802 corresponds to one of the luminance sample 502 and chrominance sample 504 of the video frame. In some embodiments, the value 1000 to be clipped corresponds to the difference between the associated image sample 804 and image sample 802. The first bit of the binary representation of the value 1000 is defined as the least significant bit (LSB), which is usually the rightmost bit. The bit index increases by one from the LSB toward the most significant bit (MSB), which is the leftmost bit. In some embodiments, the dynamic range for clipping the difference between the associated image sample 804 and each image sample 802 is defined by an upper limit -M and a lower limit M-1 of the clipping and is expressed in the form [-M, M-1], where M is 2 i Therefore, the corresponding clipping operation can be performed as a logical AND and / or logical OR operation performed on the bit range from the (i+1)th bit to the MSB. Using a logical AND and / or logical OR operation is simpler than performing a comparison-based clipping operation.

[0099] For example, the value to be clipped, 1000, is -2 11 ~2 11 It is assumed to be represented by an 11-bit value with a range of -1 (including both ends). The upper and lower limits of the clipping are [-2 4 ,2 4 A dynamic range of -1 is defined, and to determine whether this value 1000 exceeds the dynamic range defined by the upper and lower limits, the bits in the range from the 5th bit to the MSB must be examined. If the value 1000 to be clipped is positive, a logical OR operation is performed on all those bits in the range from the 5th bit to the MSB. If the result of the logical OR operation is 1, then the value 1000 is 2 4 Greater than -1, 2 4It is clipped to -1. If the clipped value 1000 is negative, a logical AND operation is performed on all those bits in the range from the 5th bit to the MSB. If the result of the logical AND operation is 0, the value 1000 becomes -2 4 Smaller than -2 4 It will be clipped.

[0100] In another example, the clipped value 1000 is in the range of -16 to 15. The value 1000 has 5 bits. The most significant bit is used to represent the sign of the value 1000, and the other 4 bits are used to represent the magnitude of the value 1000. The clipping boundary value is 4, and the dynamic range of clipping is [-2] 2 ,2 2 It is set to -1. If the input value 1000 is 14 (binarized as 01110), a logical "union" operation is applied to the third and fourth bins, resulting in 1, indicating that the input value 1000 exceeds the upper limit, and is therefore clipped to the upper limit value of 3. If the input value 1000 is 2 (binarized as 00010), a logical "union" operation is applied to the third and fourth bins, resulting in 0, indicating that the input value 1000 does not exceed the upper limit, and is therefore left unchanged. If the input value 1000 is -14 (binarized as 10010), a logical "interface" operation is applied to the third and fourth bins, resulting in 0, indicating that the input value 1000 falls below the lower limit, and is set to the lower limit value of -4. If the input value 1000 is -2 (which is binarized as 11110), a logical AND operation is applied to the third and fourth bins, resulting in 1, which indicates that the input value 1000 does not fall below the lower limit, and therefore remains unchanged.

[0101] Figure 11 is a flowchart of a video coding method 1100 according to several embodiments. The video coding method 1100 is implemented in an electronic device having a video encoder 20 or a video decoder 30. The electronic device obtains a plurality of image samples 802 of a video frame from a bitstream (1102). Each image sample 802 corresponds to one of a luminance sample 502 and a chrominance sample 504. For each of the plurality of image samples 802, each image sample 802 is filtered using an adaptive loop filter 604 having a filter length L and a set of filter coefficients (1104). Specifically, the electronic device identifies a set of associated image samples 804 at the filter length L of each image sample 802 (1106). For each of the associated set of image samples 804, the respective clip value index and corresponding filter coefficient are identified (1108). The difference between each of the associated set of image samples 804 and each image sample 802 is clipped based on the respective clip value index (1110). Each clip value index corresponds to each clipping boundary value equal to a power of 2 with each clipping number i as the exponent (1112), where each clipping number i is an integer. The electronic device modifies each image sample 802 using the clipped difference of each associated set of image samples 804 based on each filter coefficient (1114). The video frame is reconstructed using the multiple modified image samples 802 (1116). In some embodiments, the multiple image samples 802 form a block of video frames. The block is optionally an entire video frame, slice, brick, tile, tile group, coded tree, or any other coded unit.

[0102] In some embodiments, for each image sample 802, the IBDI of each image sample 802 is determined (1118). The electronic device determines each clipping boundary value for each of the associated set of image samples 804 based on the IBDI and the respective clipping value index in a predetermined clipping boundary value table 900 (1120). The difference between each of the associated set of image samples 804 and each image sample 802 is clipped based on the respective clipping boundary value (1122). Furthermore, in some embodiments, the predetermined clipping boundary value table 900 is acquired as a bitstream. Alternatively, in some embodiments, the predetermined clipping boundary value table 900 is extracted from local memory. Local memory may store multiple clipping boundary value tables (e.g., a subset or all of tables 900A to 99C), and the predetermined clipping boundary value table 900 is selected from multiple clipping boundary value tables. Examples of the predetermined clipping boundary value table 900 are shown in Figures 9A to 9C.

[0103] In some embodiments, the IBDI of each image sample 802 is determined. The electronic device determines the respective clipping boundary value for each of the associated set of image samples 804 based on the IBDI and the respective clipping value index according to a predetermined clipping boundary value formula. The difference between each of the associated set of image samples 804 and each image sample 802 is clipped based on the respective clipping boundary value. The predetermined clipping boundary value formula is expressed as one of formulas (2) to (4).

[0104] In some embodiments, for each of the associated set of image samples, the clip value index is selected from a plurality of consecutive numbers (e.g., 0, 1, 2, and 3). Each clipping number i is at least a function of the clipping value index, i.e., it changes with the clipping value index. Also, for each clipping value index, each clipping number i has a linear relationship with IBDI. For example, when the clipping value index is 0, each clipping number i is equal to IBDI, and when the clipping value index is 3, each clipping number i is equal to IBDI minus 7.

[0105] In some embodiments, for each of the set of related image samples, the clipped difference of each related image sample is within the range [-2 i ,2 i Enter -1]

[0106] In some embodiments, for each of the set of related image samples, the clipped difference of each related image sample is [-2 i +1,2 i ], [-2 i ,2 i ], and [-2 i +1,2 i It falls into one of several ranges, including -1.

[0107] In some embodiments, the step of clipping the difference between each of the associated set of image samples and each image sample includes only logical AND and / or logical OR operations.

[0108] In some embodiments, the image samples include a subset of luminance samples 502A and a subset of chrominance samples 504A. The subset of luminance samples 502A corresponds to a first set of clip value indices that define the respective clip value indices for each of the associated sets of luminance samples 502B at a first filter length L1 for each luminance sample 502A. The subset of chrominance samples 504A corresponds to a second set of clip value indices that define the respective clip value indices for each of the associated sets of chrominance samples 504B at a second filter length L2 for each chrominance sample 504A. The second set of clip value indices is different from the first set of clip value indices. Furthermore, in some embodiments, each set of the first and second sets of clip value indices corresponds to a table or expression of clipping boundary values, associating each clip value index in each set of clip value indices with its respective clipping boundary value based on IBDI. In addition, in some embodiments, the table or expression of clipping boundary values ​​corresponding to the first or second set of clip value indices is selected from a predetermined number of the table or expression of clipping boundary values. Furthermore, in some embodiments, the first and second sets of clip value indices correspond to a table / expression of first clipping boundary values ​​and a table / expression of second values ​​that are different from the table / expression of first clipping boundary values.

[0109] In one or more examples, the described function may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the function may be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable recording medium corresponding to a tangible medium such as a data recording medium, or a communication medium including any medium that facilitates the transfer of a computer program from one location to another, for example, according to a communication protocol. Thus, the computer-readable medium may generally correspond to (1) a non-temporary, tangible computer-readable recording medium, or (2) a communication medium such as a signal or carrier wave. The data recording medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes and / or data structures for implementing the embodiments described in this application. A computer program product may include a computer-readable medium.

[0110] The terminology used in the description of embodiments herein is intended to describe only specific embodiments and is not intended to limit the scope of the claims. As used in the descriptions of embodiments and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural form unless the context clearly indicates otherwise. The term “and / or” as used herein will also be understood to refer to and encompass any possible combination of one or more of the related enumerated items. The terms “equipped with” and / or “equipped with” when used herein will specify the presence of an expressed feature, element, and / or component, but will not be understood to exclude the presence or addition of one or more other features, elements, components, and / or groups thereof.

[0111] In this specification, terms such as "first," "second," etc., may be used to describe various elements, but it should be understood that these elements should not be limited by these terms. These terms are simply used to distinguish one element from another. For example, without departing from the scope of the embodiment, the first electrode may be referred to as the second electrode, and similarly, the second electrode may be referred to as the first electrode. The first electrode and the second electrode are both electrodes, but they are not the same electrode.

[0112] The description in this application is provided for explanatory and illustrative purposes only and is not intended to be exhaustive or to limit the invention to the disclosed forms. Many modifications, variations, and alternative embodiments should be apparent to those skilled in the art who benefit from the teachings presented in the preceding description and associated drawings. The embodiments have been selected and described to best illustrate the principles and practical applications of the invention, to enable others skilled in the art to understand the invention in relation to various embodiments, and to best utilize the basic principles and various embodiments with various modifications suitable for a particular intended use. Accordingly, it should be understood that the claims are not limited to specific examples and modifications of the disclosed embodiments, and other embodiments are intended to be included within the scope of the appended claims.

Claims

1. A method for coding video data, A step of obtaining multiple image samples of a video frame from a bitstream, wherein each image sample corresponds to one of a luminance sample and a chrominance sample. The step of filtering each of the plurality of image samples using an adaptive loop filter having a filter length L and a set of filter coefficients, further, The steps include identifying a set of related image samples at the filter length L of each of the aforementioned image samples, For each of the aforementioned set of related image samples, the steps include identifying the respective clip value index and the corresponding filter coefficient, A step of clipping the difference between each of the associated set of image samples and each of the image samples based on the respective clip value index, wherein each clip value index corresponds to a clipping boundary value equal to a power of 2 with the respective clipping number as the exponent, and the respective clipping number is an integer. A step comprising modifying each of the associated image samples using the clipped difference of each of the associated set of image samples based on the respective filter coefficients, The steps include: reconstructing the video frame using the plurality of modified image samples; A method that includes [a certain feature].

2. The method according to claim 1, For each image sample, the step of clipping the difference based on the respective clip value index is as follows: The steps include determining the IBDI of each of the aforementioned image samples, For each of the aforementioned set of related image samples, the steps include determining the respective clipping boundary values ​​based on the IBDI and the respective clipping value index, according to a predetermined clipping boundary value table, A step of clipping the difference between each of the associated set of image samples and each of the respective image samples based on the respective clipping boundary values. Methods that further include this.

3. A method according to claim 2, further comprising the step of obtaining the predetermined clipping boundary value table from the bitstream.

4. A method according to claim 2, further comprising the step of extracting the predetermined clipping boundary value table from local memory.

5. The method according to claim 4, The local memory stores multiple clipping boundary value tables, A method further comprising the step of selecting a predetermined clipping boundary value table from the plurality of clipping boundary value tables.

6. The method according to claim 2, A method wherein the predetermined clipping boundary value table is represented as one of the following three tables. Table 1

7. The method according to claim 1, For each image sample, the step of clipping the difference based on the respective clip value index is as follows: The steps include determining the IBDI of each of the aforementioned image samples, For each of the aforementioned set of related image samples, the steps include determining the respective clipping boundary values ​​based on the IBDI and the respective clipping value index, according to a predetermined clipping boundary value formula, A step of clipping the difference between each of the associated set of image samples and each of the respective image samples based on the respective clipping boundary values. It further includes, A method in which the formula for the predetermined clipping boundary value is expressed as one of the following formulas, where CVI is the respective clipping value index and CBV is the respective clipping boundary value for each of the associated set of image samples. [Math 1]

8. A method according to any one of claims 1 to 7, wherein for each of the associated set of image samples, the clip value index is selected from 0, 1, 2, and 3, and the respective clipping number i is at least a function of the clip value index.

9. A method according to any one of claims 1 to 8, wherein for each of the set of associated image samples, the clipped difference of each associated image sample is within the range [-2 ｉ ,2 ｉ A method to enter [-1].

10. A method according to any one of claims 1 to 9, wherein for each of the set of associated image samples, the clipped difference of each associated image sample is [-2 ｉ +1, 2 ｉ ] [-2 ｉ ,2 ｉ ], and [-2 ｉ +1, 2 ｉ A method that falls into one of several ranges, including -1.

11. A method according to any one of claims 1 to 10, wherein the step of clipping the difference between each of the associated set of image samples and each of the image samples comprises only logical AND and / or logical OR operations.

12. A method according to any one of claims 1 to 11, The plurality of image samples include a subset of luminance samples and a subset of color difference samples. A subset of the luminance samples corresponds to a first set of clip value indices that define, for each of the sets of associated luminance samples at the first filter length L of each luminance sample, １ the respective clip value indices. A subset of the aforementioned color difference samples has a second filter length L of each color difference sample. ２ Corresponding to a second set of clip value indices that define each clip value index for each of the associated sets of color difference samples in, wherein the second set of clip value indices is different from the first set of clip value indices. method.

13. A method according to claim 12, wherein each set of the first and second sets of clip value indices corresponds to a table or expression of clipping boundary values, and each clip value index in each set of clip value indices is associated with the respective clipping boundary value based on an internal bit depth increase (IBDI).

14. A method according to claim 13, further comprising the step of selecting a table or expression of clipping boundary values ​​from a predetermined number of the table or expression of clipping boundary values.

15. A method according to claim 12, wherein the first and second sets of clip value indices correspond to a table / expression of first clipping boundary values ​​and a table / expression of second values ​​different from the table / expression of first clipping boundary values.

16. One or more processors, It comprises a memory in which instructions are stored, An electronic device in which one or more processors perform one of the methods described in any one of claims 1 to 15 by executing the instructions.

17. A non-temporary computer-readable medium on which instructions are stored, wherein one or more processors perform one of the methods according to any one of claims 1 to 15 by executing the instructions.

Images