Side window bidirectional filtering for video coding

The side window-based bidirectional filtering process addresses the challenge of efficient compression and edge preservation in video coding, enhancing image quality by adaptively filtering video frames.

KR103045235B1Active Publication Date: 2026-11-06BEIJING DAJIA INTERNET INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
KR1020247002283
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Priority Date
2021-06-22
Filing Date
2022-06-21
Publication Date
2026-11-06
Estimated Expiration
2042-06-21

AI Technical Summary

Technical Problem

Existing video coding technologies face challenges in achieving efficient compression while preserving image sharpness and reducing smoothing on edges during bidirectional filtering.

Method used

Implementing a side window-based bidirectional filtering (SWBIF) process that adaptively adjusts the coding strategy for improved compression performance by using a group of candidate filtering windows, including a pool filtering window and side filtering windows, to filter video frames.

Benefits of technology

Enhances coding efficiency by effectively filtering noise while preserving sharpness, thereby improving the quality of compressed video data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure R1020247002283_ABST
    Figure R1020247002283_ABST
Patent Text Reader

Abstract

Embodiments of the present disclosure provide a video processing system and a method. The video processing method may include the step of receiving a video frame of video for in-loop filtering by one or more processors. The video processing method may further include the step of performing in-loop filtering on a target pixel of the video frame by selecting a bidirectional filtering window from a group of candidate filtering windows by one or more processors for the target pixel. The group of candidate filtering windows includes a plurality of side filtering windows and a pool filtering window. The video processing method may further include the step of filtering a target pixel of the video frame using the selected bidirectional filtering window by one or more processors.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The present invention relates to video coding and compression. More specifically, the present invention relates to a video processing system and method for bidirectional filtering using a side window in video coding. Background Technology

[0002] Digital video can be supported by various electronic devices such as digital TVs, laptops or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing devices, and video streaming devices. These electronic devices transmit and receive digital video data over communication networks or communicate via other methods, and / or store digital video data in storage devices. Due to the limited bandwidth capacity of communication networks and the limited memory resources of storage devices, video coding may be used to compress video data according to one or more video coding standards before the video data is transmitted or stored. For example, video coding standards may include Versatile Video Coding (VVC), Joint Exploration Test Model (JEM), High-Efficiency Video Coding (HEVC / H.265), Advanced Video Coding (AVC / H.264), and Moving Picture Expert Group (MPEG) coding. Video coding generally uses prediction methods (e.g., inter-prediction, intra-prediction, etc.) that utilize the redundancy inherent in video data. Video coding aims to compress video data into a form that uses a lower bit rate, thereby avoiding or minimizing degradation of video quality. The problem to be solved

[0003] The implementations of the present disclosure provide a video processing method for bidirectional filtering using a side window. means of solving the problem

[0004] A video processing method according to an embodiment of the present disclosure may include the step of receiving a video frame of video for in-loop filtering by one or more processors. The video processing method may include the step of performing in-loop filtering on a target pixel of the video frame by one or more processors using a group of candidate filtering windows including a plurality of side filtering windows and a pool filtering window. The video processing method may further include the step of filtering a target pixel of the video frame using a selected bidirectional filtering window by one or more processors.

[0005] Embodiments of the present disclosure also provide a video processing device for bidirectional filtering having a side window. The video processing device may include memory and one or more processors. The memory may be configured to store video. One or more processors may be configured to receive video frames of video for in-loop filtering. For a target pixel of a video frame, one or more processors may be further configured to perform in-loop filtering on the target pixel by selecting a bidirectional filtering window from a group of candidate filtering windows including a plurality of side filtering windows and a pool filtering window. One or more processors may be further configured to filter the target pixel of the video frame using the selected bidirectional filtering window.

[0006] An embodiment of the present disclosure also provides a non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause one or more processors to perform a video processing method for bidirectional filtering having a side window. The video processing method may include the step of receiving a video frame of video for in-loop filtering. The video processing method may further include the step of performing in-loop filtering on a target pixel of a video frame by selecting a bidirectional filtering window from a group of candidate filtering windows, including a plurality of side filtering windows and a pool filtering window, for the target pixel of the video frame. The video processing method may further include the step of filtering the target pixel of the video frame using the selected bidirectional filtering window. The filtered video frame is stored in a non-transitory computer-readable storage medium.

[0007] It should be understood that the foregoing general description and the following detailed description are merely illustrative and are not limited to the present disclosure. Brief explanation of the drawing

[0008] The attached drawings included in and constituting part of this specification illustrate examples consistent with this specification and, together with descriptions, serve to explain the principles of this specification. FIG. 1 is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to some implementations of the present disclosure. FIG. 2 is a block diagram showing an exemplary video encoder according to some implementation methods of the present disclosure. FIG. 3 is a block diagram showing an exemplary video decoder according to some implementation methods of the present disclosure. FIGS. 4A through 4E are graphic representations illustrating a method in which a frame is recursively divided into a plurality of video blocks of different sizes and shapes according to some implementations of the present disclosure. Figure 5 is a diagram showing typical edge types in video frames. FIG. 6 is a graphic representation showing a typical side filtering window according to some implementations of the present disclosure. FIG. 7 is a block diagram illustrating an exemplary process for bidirectional filtering using a side filtering window according to some implementations of the present disclosure. FIG. 8 is a flowchart of an exemplary method for bidirectional filtering using a side filtering window according to some implementations of the present disclosure. FIG. 9 is a flowchart of a first exemplary implementation of a bidirectional filtering method using the side filtering window of FIG. 8 according to some implementations of the present disclosure. FIG. 10 is a flowchart of a second exemplary implementation of a method for bidirectional filtering using the side filtering window of FIG. 8 according to some implementations of the present disclosure. FIG. 11 is a flowchart of a third exemplary implementation of a bidirectional filtering method using the side filtering window of FIG. 8 according to some implementations of the present disclosure. FIG. 12 is a flowchart of an exemplary method for selecting a filtered block for a video block according to some implementations of the present disclosure. FIG. 13 is a block diagram showing a computing environment combined with a user interface according to some implementation methods of the present disclosure. Specific details for implementing the invention

[0009] Specific implementations illustrated in the accompanying drawings are described in detail below. In the following detailed description, a number of non-limiting specific details are described to aid in understanding the contents described herein. However, it is obvious to those skilled in the art that various modifications may be made without departing from the scope of the claims and that the subject matter may be practiced without these specific details. For example, it is obvious to those skilled in the art that the technical solutions described herein may be implemented in various types of electronic devices equipped with digital video functions.

[0010] It should be illustrated that terms such as “first,” “second,” etc., used in the description of the present disclosure, claims, and accompanying drawings are used to distinguish objects and are not used to describe any specific order or sequence. It should be understood that data used in this manner may be interchangeable under appropriate conditions, and accordingly, embodiments of the present disclosure described herein may be implemented in an order other than that shown in the accompanying drawings or described in the present disclosure.

[0011] In current VVC and AVS3 standards, one or more in-loop filtering modules may exist, including de-blocking filters (DBF), sample adaptive offset (SAO) filters, and adaptive in-loop filters (ALF). During the development of the VVC standard, bidirectional filters were initially proposed to refine blocks reconstructed after inverse transformation. Later, the application of bidirectional filters was extended as part of in-loop filtering that could be used in conjunction with SAOs. For example, the outputs of a joint bidirectional filter and an SAO filter can be expressed using the following equation:

[0012] (1)

[0013] In the above equation (1) represents the output of the joint bidirectional filter and the SAO filter. represents the intensity of the center sample. represents the difference value generated by a bidirectional filter using samples from a de-blocking filter as input. represents the offset value generated by the SAO filter using the sample from the de-blocking filter as input. represents a clipping function to check if the output is within the range [minValue, maxValue] defined as follows:

[0014] (2)

[0015] In Exploration Experiment (EE) 2 for enhanced compression beyond VVC capabilities, bidirectional filters are considered one of the promising tools for further compression performance enhancement. In some exemplary applications, pixels to be filtered by the bidirectional filter are positioned at the center of the filtering window, and this type of filtering behavior can result in smoothing on edges, which is expected to preserve sharpness during filtering. Side window filtering techniques can be used to filter out noise while preserving sharpness or other types of desirable signals on edges to prevent undesirable smoothing on edges.

[0016] In accordance with the present disclosure, side window filtering techniques are applied in video coding to improve coding efficiency. To achieve the goal of bidirectional filtering in video coding to make the filtered image as close as possible to the original image, the selection of side windows is carefully considered. In accordance with the present disclosure, the coding strategy for side window filtering can be adaptively adjusted for better compression performance. For example, the quality and statistical properties of video blocks in a video frame may vary from block to block, and a block adaptation mechanism for side window filtering may need to be used to improve the compression performance of the video.

[0017] Consistent with the present disclosure, the present specification discloses a video processing system and method for improving the coding efficiency of bidirectional filtering. For example, the system and method disclosed in the present specification may integrate a side window filtering mechanism into bidirectional filtering, and thereby apply a side window-based bidirectional filtering (SWBIF) process to improve the coding efficiency of bidirectional filtering.

[0018] FIG. 1 is a block diagram illustrating an exemplary system (10) for encoding and decoding video blocks in parallel according to some implementations of the present disclosure. As illustrated in FIG. 1, the system (10) includes a source device (12) that generates and encodes video data to be later decoded by a destination device (14). The source device (12) and the destination device (14) may include any one of various electronic devices such as a desktop or laptop computer, a tablet computer, a smartphone, a set-top box, a digital television, a camera, a display device, a digital media player, a video game console, or a video streaming device. In some implementations, the source device (12) and the destination device (14) may have wireless communication capabilities.

[0019] In some implementations, the destination device (14) may receive encoded video data to be decoded via a link (16). The link (16) may include any type of communication medium or device capable of forwarding the encoded video data from the source device (12) to the destination device (14). In one example, the link (16) may be a communication medium that enables the source device (12) to directly transmit the encoded video data to the destination device (14) in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device (14). The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network or a wide-area network, or a global network, such as the Internet. The communication medium may include a router, switch, base station, or any other equipment that can be used to facilitate communication from a source device (12) to a destination device (14).

[0020] In some other implementations, encoded video data may be transferred from the output interface (22) to the storage device (32). Next, the encoded video data within the storage device (32) can be accessed by the destination device (14) through the input interface (28). The storage device (32) may include any one of various distributed or local access data storage media (e.g., hard drive, Blu-ray disc, DVD (Digital Versatile Disks), CD-ROM (Compact Disc Read-Only Memories), flash memory, volatile or non-volatile memory), or any other digital storage media suitable for storing encoded video data. In additional examples, the storage device (32) may correspond to a file server or other intermediate storage device capable of storing encoded video data generated by the source device (12). The destination device (14) may access the stored video data from the storage device (32) via streaming or download. The file server may be any type of computer capable of storing the encoded video data and transferring the encoded video data to the destination device (14). An exemplary file server may include a web server (e.g., for a website), a File Transfer Protocol (FTP) server, a Network Attached Storage (NAS) device, or a local disk drive. The destination device (14) may access the encoded video data stored in the file server via any standard data connection including a wireless channel (e.g., a Wi-Fi (Wireless Fidelity) connection), a wired connection (e.g., a Digital Subscriber Line (DSL), a cable modem, etc.), or any combination thereof.The transmission of encoded video data from the storage device (32) may be a streaming transmission, a download transmission, or a combination thereof.

[0021] As illustrated in FIG. 1, the source device (12) includes a video source (18), a video encoder (20), and an output interface (22). The video source (18) may include a video capture device (e.g., a video camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data into a source video, or a combination thereof. For example, if the video source (18) is a video camera of a security surveillance system, the source device (12) and the destination device (14) may form a camera phone or a video phone. However, the implementations described in this application are generally applicable to video coding and may be applicable to wireless and / or wired applications.

[0022] A captured, previously captured, or computer-generated video may be encoded by a video encoder (20). The encoded video data may be transmitted directly to a destination device (14) through an output interface (22) of a source device (12). Additionally, the encoded video data may be stored (or alternatively) in a storage device (32) so that it can be accessed later by the destination device (14) or another device to be decoded and / or played back. The output interface (22) may further include a modem and / or transmitter.

[0023] The destination device (14) includes an input interface (28), a video decoder (30), and a display device (34). The input interface (28) may include a receiver and / or a modem and may receive encoded video data via the link (16). The encoded video data transmitted via the link (16) or provided on the storage device (32) may include various syntax elements generated by the video encoder (20) and used by the video decoder (30) to decode the video data. These syntax elements may be included in the encoded video data transmitted via a communication medium, stored on a storage medium, or stored on a file server.

[0024] In some implementations, the destination device (14) may include a display device (34) which may be an integrated display device and an external display device configured to communicate with the destination device (14). The display device (34) displays decoded video data to a user and may include any one of various display devices such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0025] The video encoder (20) and video decoder (30) may operate according to proprietary or industry standards such as VVC, HEVC, MPEG-4, Part 10, AVC (Advanced Video Coding), or extensions of these standards. It should be understood that this application is not limited to the specified video encoding / decoding standards and may apply to other video encoding / decoding standards. Generally, it should be considered that the video encoder (20) of the source device (12) may be configured to encode video data according to any one of these current or future standards. Similarly, it should be generally considered that the video decoder (30) of the destination device (14) may be configured to decode video data according to any one of these current or future standards.

[0026] Each of the video encoder (20) and video decoder (30) may be implemented as any one of various suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. In the case of partial software implementation, the electronic device may perform the video encoding / decoding operations disclosed in this disclosure by storing instructions for the software on a suitable non-transient computer-readable medium and executing the instructions in hardware using one or more processors. Each of the video encoder (20) and video decoder (30) may be included in one or more encoders or decoders, and these may be directly connected to each device as part of a combined encoder / decoder (CODEC).

[0027] FIG. 2 is a block diagram illustrating an exemplary video encoder (20) according to some of the implementation methods described in the present application. The video encoder (20) can perform intra- and inter-predictive coding of video blocks within a video frame. Intra-predictive coding can reduce or eliminate spatial redundancy of video data within a given video frame or picture by relying on spatial prediction. Inter-predictive coding can reduce or eliminate temporal redundancy of video data within adjacent video frames or pictures of a video sequence by relying on temporal prediction. It should be noted that the term "frame" may be used as a synonym for the terms "image" or "picture" in the field of video coding.

[0028] As illustrated in FIG. 2, the video encoder (20) includes a video data memory (40), a prediction processing unit (41), a decoded picture buffer (DPB, 64), an adder (50), a transformation processing unit (52), a quantization unit (54), and an entropy encoding unit (56). The prediction processing unit (41) includes a motion estimation unit (42), a motion compensation unit (44), a segmentation unit (45), an intra prediction processing unit (46), and an intra block copying (BC) unit (48). In some implementations, the video encoder (20) also includes an inverse quantization unit (58), an inverse transformation processing unit (60), and an adder (62) for video block reconstruction. An in-loop filter (63), such as a de-blocking filter, may be positioned between the adder (62) and the DPB (64) to filter block boundaries and remove block artifacts from the reconstructed video. In addition to the de-blocking filter, other in-loop filters, such as an SAO filter and / or an adaptive in-loop filter (ALF), may also be used to filter the output of the adder (62). In some examples, the in-loop filters may be omitted, and the decoded video block may be provided directly to the DPB (64) by the adder (62). The video encoder (20) may be configured in the form of a fixed or programmable hardware unit or divided into one or more of the illustrated fixed or programmable hardware devices.

[0029] The video data memory (40) can store video data to be encoded by the components of the video encoder (20). The video data in the video data memory (40) can be obtained from a video source (18), for example, as shown in FIG. 1. The DPB (64) is a buffer that stores reference video data (e.g., a reference frame or picture) used by the video encoder (20) to encode video data (e.g., in intra or inter predictive coding mode). The video data memory (40) and the DPB (64) can be formed by any one of various memory devices. In various examples, the video data memory (40) may be on-chip with other components of the video encoder (20) or off-chip with respect to these components.

[0030] As illustrated in FIG. 2, after receiving video data, the partitioning unit (45) within the prediction processing unit (41) partitions the video data into video blocks. Meanwhile, this partitioning may include dividing the video frame into slices, tiles (e.g., sets of video blocks), or other larger coding units (CU) according to a pre-set partitioning structure, such as a quad-tree (QT) structure associated with the video data. A video frame may be considered as a two-dimensional array or matrix of samples having sample values. Samples within the array may also be referred to as pixels or pels. Multiple samples in the horizontal and vertical directions (or axes) of the array or picture define the size and / or resolution of the video frame. A video frame may be partitioned into multiple video blocks, for example, by using QT partitioning. A video block may be considered as a two-dimensional array or matrix of samples having sample values, although having a smaller dimension than the video frame. Multiple samples of the horizontal and vertical directions (or axes) of the video block define the size of the video block. The video block may be further partitioned into one or more block partitionings or sub-blocks (which may again form blocks) by repeatedly using, for example, QT partitioning, Binary-Tree (BT) partitioning, Triple-Tree (TT) partitioning, or any combination thereof. It should be noted that the terms “block” or “video block” as used herein may refer to a part of a frame or picture, particularly a rectangular (square or non-square) portion.For example, referring to HEVC and VVC, a block or video block may be a CTU, CU, Prediction Unit (PU), or Transform Unit (TU), or may be a corresponding block, and / or may be a Coding Tree Block (CTB), Coding Block (CB), Prediction Block (PB), or Transform Block (TB). Alternatively or additionally, a block or video block may be a sub-block such as a CTB, CB, PB, TB, or may be a corresponding block.

[0031] The prediction processing unit (41) may select one of a plurality of possible prediction coding modes, such as one of a plurality of intra prediction coding modes or one of a plurality of inter prediction coding modes for the current video block, based on the error results (e.g., coding rate and distortion level). The prediction processing unit (41) may provide the intra or inter prediction coded block (e.g., prediction block), which is the processing result, to the adder (50) to generate a residual block, and provide it to the adder (62) to reconstruct the encoded block so that it can be used later as part of a reference frame. Meanwhile, the prediction processing unit (41) may provide syntax elements such as motion vectors, intra-mode indicators, partition information, and other such syntax information to the entropy encoding unit (56).

[0032] To select an appropriate intra-predictive coding mode for the current video block, the intra-predictive processing unit (46) within the prediction processing unit (41) may provide spatial prediction by performing intra-predictive coding of the current video block for one or more adjacent blocks within the same frame as the current block to be coded. The motion estimation unit (42) and motion compensation unit (44) within the prediction processing unit (41) may provide temporal prediction by performing inter-predictive coding of the current video block for one or more prediction blocks among one or more reference frames. The video encoder (20) may perform multiple coding passes to select, for example, an appropriate coding mode for each block of video data.

[0033] In some implementations, the motion estimation unit (42) determines the inter-prediction mode for the current video frame by generating motion vectors according to a predetermined pattern within the video frame sequence, said motion vectors may indicate the displacement of a video block in the current video frame relative to a prediction block in a reference frame. The motion estimation performed by the motion estimation unit (42) is the process of generating motion vectors capable of estimating the motion of video blocks. For example, the motion vectors may indicate the displacement of a video block in the current video frame or picture relative to a prediction block in a reference frame. The predetermined pattern may designate the video frame of the sequence as a P frame or a B frame. The intra-BC unit (48) may determine vectors for intra-BC coding (e.g., block vectors) in a manner similar to the determination of motion vectors by the motion estimation unit (42) for inter-prediction, or may determine block vectors using the motion estimation unit (42).

[0034] In terms of pixel difference, the prediction block for the video block may be a block of reference frame or a reference block considered to closely match the video block to be coded, or may be equivalent, and the pixel difference may be determined by the sum of absolute difference (SAD), the sum of square difference (SSD), or other difference metrics. In some implementations, the video encoder (20) may calculate values ​​for sub-integer pixel locations of reference frames stored in the DPB (64). For example, the video encoder (20) may interpolate values ​​for 1 / 4 pixel locations, 1 / 8 pixel locations, or other fractional pixel locations of the reference frame. Thus, the motion estimator (42) may perform motion search for whole pixel locations and fractional pixel locations and output a motion vector with fractional pixel precision.

[0035] The motion estimation unit (42) calculates the motion vector of a video block within an inter-predicted coded frame by comparing the position of the prediction block of a reference frame, wherein the reference frame may be selected from a first reference frame list (List 0) or a second reference frame list (List 1) that each identify one or more reference frames stored in the DPB (64). The motion estimation unit (42) transmits the calculated motion vector to the motion compensation unit (44) and then to the next entropy encoding unit (56).

[0036] Motion compensation performed by the motion compensation unit (44) may include fetching or generating a prediction block based on a motion vector determined by the motion estimation unit (42). Upon receiving the motion vector of the current video block, the motion compensation unit (44) may find the prediction block indicated by the motion vector in one of the reference frame lists, retrieve the prediction block from the DPB (64), and pass the prediction block to the adder (50). Next, the adder (50) may form a pixel difference value of the remaining block by subtracting the pixel value of the prediction block provided by the motion compensation unit (44) from the pixel value of the current video block being coded. The pixel difference value forming the remaining video block may include a luminance or chroma difference component, or both. Additionally, the motion compensation unit (44) may generate syntax elements associated with the video block of the video frame that the video decoder (30) uses when decoding the video block of the video frame. Syntax elements may include, for example, syntax elements defining motion vectors used to identify a prediction block, any flag indicating a prediction mode, or any other syntax information described herein. Note that the motion estimation unit (42) and the motion compensation unit (44) may be integrated together, which is illustrated separately in FIG. 2 for conceptual purposes.

[0037] In some implementations, the intra-BC unit (48) may generate vectors and fetch prediction blocks in a manner similar to that described above in relation to the motion estimation unit (42) and motion compensation unit (44), but these prediction blocks are within the same frame as the current block being coded, and the vectors are referred to as block vectors rather than motion vectors. Specifically, the intra-BC unit (48) may determine the intra-prediction mode used to encode the current block. In some examples, the intra-BC unit (48) may encode the current block using various intra-prediction modes, for example, during each encoding pass, and may test their performance through rate-distortion analysis. Next, the intra-BC unit (48) may select and use an appropriate intra-prediction mode from among the various tested intra-prediction modes and generate an intra-mode identifier accordingly. For example, the intra-BC section (48) can calculate rate-distortion values ​​using rate-distortion analyses for various tested intra-prediction modes, and can select and use an intra-prediction mode having optimal rate-distortion characteristics among the tested modes as an appropriate intra-prediction mode. Generally, rate-distortion analysis determines the amount of distortion (or error) between the encoded block and the original unencoded block being encoded to generate the encoded block, and the bit rate (i.e., number of bits) for generating the encoded block. The intra-BC section (48) can determine which intra-prediction mode indicates the optimal rate-distortion value for the block by calculating a ratio from the distortion and rate for various encoded blocks.

[0038] In other examples, the intra-BC unit (48) may perform functions for intra-BC prediction according to the implementation methods described herein by using the motion estimation unit (42) and the motion compensation unit (44) wholly or partially. In both cases, for an intra-block copy, the prediction block may be a block considered to be closely matched to the block to be coded in terms of pixel difference determined by the sum of absolute differences (SAD), the sum of squared differences (SSD), or other difference metrics, and the identification of the prediction block may include the calculation of values ​​for sub-integer pixel locations.

[0039] Regardless of whether the prediction block is generated from the same frame according to intra prediction or from a different frame according to inter prediction, the video encoder (20) can form a residual block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being coded to form pixel difference values. The pixel difference values ​​forming the residual block may include both lumina and chroma component differences.

[0040] As an alternative to the inter-prediction performed by the motion estimation unit (42) and motion compensation unit (44) described above, or the intra-block copy prediction performed by the intra-BC unit (48), the intra-prediction processing unit (46) can intra-predict the current video block. Specifically, the intra-prediction processing unit (46) can determine the intra-prediction mode to use to encode the current block. For example, the intra-prediction processing unit (46) can encode the current block using various intra-prediction modes, for example, during each encoding pass, and the intra-prediction processing unit (46) (or the mode selection unit in some examples) can select and use an appropriate intra-prediction mode from the tested intra-prediction modes. The intra-prediction processing unit (46) can provide information indicating the selected intra-prediction mode for the block to the entropy encoding unit (56). The entropy encoding unit (56) can encode the information indicating the selected intra-prediction mode into the bitstream.

[0041] After the prediction processing unit (41) determines a prediction block for the current video block through inter-prediction or intra-prediction, the adder (50) subtracts the prediction block from the current video block to form a residual block. The residual video data within the residual block may be included in one or more transformation units (TU) and provided to the transformation processing unit (52). The transformation processing unit (52) converts the residual video data into transformation factors using a transformation such as the Discrete Cosine Transform (DCT) or a conceptually similar transformation.

[0042] The conversion processing unit (52) can transmit the obtained conversion coefficients to the quantization unit (54). The quantization unit (54) can quantize the conversion coefficients to further reduce the bit rate. Additionally, the quantization process can reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit (54) can perform a scan of a matrix containing the quantized conversion coefficients. Alternatively, the entropy encoding unit (56) can perform the scan.

[0043] After quantization, the entropy encoding unit (56) may encode the quantized transform coefficients into a video bitstream using an entropy encoding technique, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy encoding methods or techniques. Then, the encoded bitstream may be transmitted to a video decoder (30) as shown in FIG. 1, or stored in a storage device (32) as shown in FIG. 1 so that it can be transmitted to a video decoder (30) or retrieved by a video decoder (30) later. The entropy encoding unit (56) can also encode motion vectors and other syntax elements used in the current video frame being coded using entropy encoding technology.

[0044] The inverse quantization unit (58) and the inverse transform processing unit (60) can reconstruct residual blocks in the pixel domain by applying inverse quantization and inverse transform, respectively, to generate reference blocks for predicting other video blocks. Reconstructed residual blocks can be generated. As described above, the motion compensation unit (44) can generate motion-compensated prediction blocks from one or more reference blocks of frames stored in the DPB (64). The motion compensation unit (44) can also apply one or more interpolation filters to the prediction blocks to produce sub-integer pixel values ​​for use in motion estimation.

[0045] The adder (62) can generate a reference block to be stored in the DPB (64) by adding the reconstructed residual block and the motion-compensated prediction block generated by the motion compensation unit (44). Next, the reference block is used as a prediction block by the intra-BC unit (48), the motion estimation unit (42), and the motion compensation unit (44) to inter-predict other video blocks within a subsequent video frame.

[0046] FIG. 3 is a block diagram illustrating an exemplary video decoder (30) according to some implementation methods of the present application. The video decoder (30) includes a video data memory (79), an entropy decoding unit (80), a prediction processing unit (81), an inverse quantization unit (86), an inverse transform processing unit (88), an adder (90), and a DPB (92). The prediction processing unit (81) further includes a motion compensation unit (82), an intra prediction unit (84), and an intra BC unit (85). The video decoder (30) can generally perform a decoding process opposite to the encoding process described above for the video encoder (20) in relation to FIG. 2. For example, the motion compensation unit (82) can generate prediction data based on motion vectors received from the entropy decoding unit (80), and the intra prediction unit (84) can generate prediction data based on intra prediction mode indicators received from the entropy decoding unit (80).

[0047] In some examples, the components of the video decoder (30) may perform functions to implement the implementations of the present application. Additionally, in some examples, the components of the video decoder (30) may be divided into one or more parts to implement the implementations of the present disclosure. For example, the intra-BC unit (85) may perform the implementations of the present application alone or in combination with other parts of the video decoder (30), such as the motion compensation unit (82), the intra-prediction unit (84), and the entropy decoding unit (80). In some examples, the video decoder (30) may not include the intra-BC unit (85), and the function of the intra-BC unit (85) may be performed by other parts of the prediction processing unit (81), such as the motion compensation unit (82).

[0048] The video data memory (79) may store video data to be decoded by other parts of the video decoder (30), such as an encoded video bitstream. The video data stored in the video data memory (79) may be obtained, for example, from a storage device (32), obtained via wired or wireless network communication of video data, or obtained from a local video source, such as a camera, by accessing a physical data storage medium (e.g., a flash drive or a hard disk). The video data memory (79) may include a coded picture buffer (CPB) that stores encoded video data of the encoded video bitstream. The decoded picture buffer (DPB) (92) of the video decoder (30) may store reference video data for the video decoder (30) to use to decode video data (e.g., in intra or inter predictive coding mode). The video data memory (79) and DPB (92) may be formed from any of the various memory devices, such as SDRAM, MRAM, RRAM, DRAM, or other types of memory devices. For illustrative purposes, the video data memory (79) and DPB (92) are shown in FIG. 3 as two separate parts of the video decoder (30). However, it is obvious to those skilled in the art that the video data memory (79) and DPB (92) may be provided by the same memory device or by separate memory devices. In some examples, the video data memory (79) may be on-chip with other parts of the video decoder (30) or off-chip with these parts.

[0049] During the decoding process, the video decoder (30) receives an encoded video bitstream pointing to video blocks of the encoded video frames and associated syntax elements. The video decoder (30) may receive syntax elements at the video frame level and / or video block level. The entropy decoding unit (80) of the video decoder (30) may obtain quantized coefficients, motion vectors or intra-prediction mode indicators and other syntax elements by decoding the bitstream using an entropy decoding technique. The entropy decoding unit (80) transmits the motion vectors or intra-prediction mode indicators and other syntax elements to the prediction processing unit (81).

[0050] When a video frame is coded into an intra-predictive coding (e.g., I) frame or coded into an intra-coded prediction block in another type of frame, the intra-predictive unit (84) of the prediction processing unit (81) can generate prediction data for a video block of the current video frame based on the signaled intra-predictive mode and reference data from previously decoded blocks of the current frame.

[0051] When a video frame is coded as an inter-predicted coded (i.e., B or P) frame, the motion compensation unit (82) of the prediction processing unit (81) generates one or more prediction blocks for a video block of the current video frame based on motion vectors and other syntax elements received from the entropy decoding unit (80). Each prediction block may be generated from a reference frame within one of the reference frame lists. The video decoder (30) may generate reference frame lists (e.g., List0 and List1) using default configuration techniques based on the reference frames stored in the DPB (92).

[0052] In some examples, when a video block is coded according to the intra-BC mode described herein, the intra-BC section (85) of the prediction processing unit (81) generates prediction blocks for the current video block based on block vectors and other syntax elements received from the entropy decoding unit (80). The prediction blocks may be within a reconstructed area of ​​the same picture as the current video block processed by the video encoder (20).

[0053] The motion compensation unit (82) and / or the intra-BC unit (85) parses motion vectors and other syntax elements to determine prediction information for video blocks of the current video frame, and then uses the prediction information to generate prediction blocks for the current video block to be decoded. For example, the motion compensation unit (82) may determine, using some of the received syntax elements, the prediction mode used for coding the video blocks of the video frame (e.g., intra or inter prediction), the inter prediction frame type (e.g., B or P), configuration information for one or more of the reference frame lists for the frame, motion vectors for each inter prediction coded video block of the frame, the inter prediction state for each inter prediction coded video block of the frame, and other information for decoding the video blocks of the current video frame.

[0054] Similarly, the intra-BC section (85) can use some of the received syntax elements (e.g., flags) to determine the construction information that the current video block was predicted using intra-BC mode, which video blocks of the frame are within the reconstructed area, which should be stored in the DPB (92), block vectors for each intra-BC predicted video block of the frame, the intra-BC predicted state for each intra-BC predicted video block of the frame, and other information for decoding the video blocks within the current video frame.

[0055] The motion compensation unit (82) may also perform interpolation using interpolation filters used by the video encoder (20) during the encoding of video blocks to calculate interpolation values ​​for sub-integer pixels of reference blocks. In this case, the motion compensation unit (82) may determine the interpolation filters used by the video encoder (20) from the received syntax elements and generate prediction blocks using the determined interpolation filters.

[0056] The inverse quantization unit (86) determines the degree of quantization by inversely quantizing the transform coefficients provided in the bitstream and decoded by the entropy decoding unit (80), using the same quantization parameters calculated by the video encoder (20) for each video block in the video frame. The inverse transform processing unit (88) applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the remaining blocks in the pixel domain.

[0057] After generating a prediction block for the current video block based on vectors and other syntax elements in the motion compensation unit (82) or the intra-BC unit (85), the adder (90) reconstructs the decoded video block for the current video block by adding the residual block provided by the inverse transformation processing unit (88) and the corresponding prediction block generated by the motion compensation unit (82) and the intra-BC unit (85). The decoded video block may also be referred to as the reconstructed block for the current video block. An in-loop filter (91), such as a de-blocking filter, an SAO filter, and / or an ALF, may be placed between the adder (90) and the DPB (92) to further process the decoded video block. In some examples, the in-loop filter (91) may be omitted, and the decoded video block may be provided directly to the DPB (92) by the adder (90). Decoded video blocks within a given frame are stored in a DPB (92) that stores reference frames used for subsequent motion compensation of the next video blocks. The DPB (92) or a memory device separate from the DPB (92) can also store the decoded video so that it can be displayed later on a display device such as the display device (34) of FIG. 1.

[0058] In a typical video coding process (including, for example, a video encoding process and a video decoding process), a video sequence generally contains a set of frames or pictures in a predetermined order. Each frame may contain three sample matrices denoted as SL, SCb, and SCr. SL is a two-dimensional matrix of luminance samples. SCb is a two-dimensional matrix of Cb chroma samples. SCr is a two-dimensional matrix of Cr chroma samples. In other cases, since the frame may be monochrome, it may contain only one two-dimensional matrix of luminance samples.

[0059] As illustrated in FIG. 4A, a video encoder (20) (or more specifically, a splitting unit (45)) first generates an encoded representation of a frame by splitting the frame into a set of CTUs. A video frame may contain an integer number of CTUs arranged sequentially in raster scan order from left to right and from top to bottom. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder (20) as a sequence parameter set, so that all CTUs in the video sequence have the same size, which is one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that the CTUs in the present invention are not necessarily limited to a specific size. As illustrated in FIG. 4B, each CTU may include one CTB of luminance samples, two corresponding coding tree blocks of chroma samples, and syntax elements used to code the samples of the coding tree blocks. The syntax elements describe the characteristics of different types of units of the coded pixel blocks, including inter or intra prediction, intra prediction mode, motion vector, and other parameters, and how the video sequence can be reconstructed in the video decoder (30). In monochrome pictures or pictures having three distinct color planes, the CTU may include a single coding tree block and syntax elements used to code the samples of the coding tree block. The coding tree block may be an N×N block of samples.

[0060] To achieve better performance, the video encoder (20) may recursively perform tree partitioning, such as binary-tree partitioning, ternary-tree partitioning, quad-tree partitioning, or a combination thereof, on the coding tree blocks of the CTU and partition the CTU into smaller coding units (CUs). As illustrated in FIG. 4C, a 64×64 CTU (400) is first partitioned into four smaller CUs, each having a block size of 32×32. Of the four smaller CUs, CU (410) and CU (420) are each partitioned into four 16×16 CUs according to their block size. The two 16×16 CUs (430 and 440) are further partitioned into four 8×8 CUs according to their block size. FIG. 4D illustrates a quad-tree data structure representing the final result of the partitioning process of a CTU (400) as illustrated in FIG. 4C, where each leaf node of the quad tree corresponds to a single CU of a corresponding size ranging from 32×32 to 8×8. Similar to the CTU illustrated in FIG. 4B, each CU may include a coding block (CB) of luminance samples, two corresponding coding blocks of chroma samples of the same size frame, and syntax elements used to code the samples of the coding block. In monochrome pictures or pictures having three distinct color planes, the CU may include a single coding block and syntax structures used to code the samples of the coding block. It should be noted that the quad-tree partitioning illustrated in FIG. 4C and 4D is for illustrative purposes only, and that a single CTU may be partitioned into CUs to adapt to various local characteristics based on quad / ternary / binary-tree partitionings. In a multi-type tree structure, one CTU is divided into a quad-tree structure, and each quad-tree leaf CU can be further divided into binary and ternary-tree structures.As shown in FIG. 4E, a coding block with width W and height H has various possible partitioning types, namely quaternary partitioning, vertical binary partitioning, horizontal binary partitioning, vertical ternary partitioning, vertically extended ternary partitioning, horizontal ternary partitioning, and horizontally extended ternary partitioning.

[0061] In some implementations, the video encoder (20) may further subdivide the coding block of the CU into one or more M×N prediction blocks (PB). A PB may include a rectangular (square or non-square) block of samples to which the same prediction (inter or intra prediction) is applied. A PU of the CU may include a PB of luminance samples, two corresponding PBs of chroma samples, and syntax elements used to predict the PB. In monochrome pictures or pictures having three distinct color planes, a PU may include a single PB and syntax structures used to predict the PB. The video encoder (20) may generate a predicted luminance block, a predicted Cb block, and a predicted Cr block for the luminance PB, Cb PB, and Cr PB of each PU of the CU.

[0062] The video encoder (20) may use intra prediction or inter prediction to generate prediction blocks for the PU. If the video encoder (20) uses intra prediction to generate prediction blocks for the PU, the video encoder (20) may generate prediction blocks for the PU based on decoded samples of frames associated with the PU. If the video encoder (20) uses inter prediction to generate prediction blocks for the PU, the video encoder (20) may generate prediction blocks for the PU based on decoded samples of one or more frames that are not associated with the PU.

[0063] After the video encoder (20) generates predicted luma blocks, Cb blocks, and Cr blocks of one or more PUs of the CU, the video encoder (20) may generate a luma residual block for the CU by subtracting the predicted luma blocks of the CU from the original luma coding block of the CU, so that each sample of the luma residual block of the CU represents the difference between one luma sample of the predicted luma blocks of the CU and the corresponding sample of the original luma coding block of the CU. Similarly, the video encoder (20) may generate a Cb residual block and a Cr residual block for the CU, respectively, so that each sample of the Cb residual block of the CU represents the difference between one Cb sample of the predicted Cb blocks of the CU and the corresponding sample of the original Cb coding block of the CU, and each sample of the Cr residual block of the CU represents the difference between one Cr sample of the predicted Cr blocks of the CU and the corresponding sample of the original Cr coding block of the CU.

[0064] Additionally, as illustrated in FIG. 4C, the video encoder (20) may use quad-tree partitioning to decompose the luminance, Cb, and Cr residual blocks of the CU into one or more luminance, Cb, and Cr transform blocks, respectively. A transform block may include a rectangular (square or non-square) block of samples to which the same transform is applied. A transform unit (TU) of the CU may include a transform block of luminance samples, two corresponding transform blocks of chroma samples, and syntax elements used to transform the transform block samples. Thus, each TU of the CU may be associated with a luminance transform block, a Cb transform block, and a Cr transform block. In some examples, the luminance transform block associated with the TU may be a sub-block of the luminance residual block of the CU. The Cb transform block may be a sub-block of the Cb residual block of the CU. The Cr transform block may be a sub-block of the Cr residual block of the CU. In monochrome pictures or pictures having three distinct color planes, TU may include a single transform block and syntax structures used to predict samples of the transform block.

[0065] The video encoder (20) may apply one or more transformations to the luminance transformation block of the TU to generate a luminance coefficient block for the TU. The coefficient block may be a two-dimensional array of transformation coefficients. The transformation coefficients may be scalar quantities. The video encoder (20) may apply one or more transformations to the Cb transformation block of the TU to generate a Cb coefficient block for the TU. The video encoder (20) may apply one or more transformations to the Cr transformation block of the TU to generate a Cr coefficient block for the TU.

[0066] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), the video encoder (20) can quantize the coefficient block. Quantization generally refers to a process that provides additional compression by quantizing the transform coefficients to reduce as much as possible the amount of data used to represent the transform coefficients. After the video encoder (20) quantizes the coefficient block, the video encoder (20) can apply entropy encoding techniques to encode syntax elements representing the quantized transform coefficients. For example, the video encoder (20) can perform Context-Adaptive Binary Arithmetic Coding (CABAC) on the syntax elements representing the quantized transform coefficients. Finally, the video encoder (20) can output a bitstream containing a sequence of bits forming a representation of the coded frames and related data, which is stored in the storage device (32) or transmitted to the destination device (14).

[0067] After receiving a bitstream generated by a video encoder (20), a video decoder (30) can parse the bitstream to obtain syntax elements from the bitstream. The video decoder (30) can reconstruct frames of video data based at least partially on the syntax elements obtained from the bitstream. The process of reconstructing video data is generally opposite to the encoding process performed by the video encoder (20). For example, the video decoder (30) can reconstruct residual blocks associated with the TUs of the current CU by performing inverse transformation on the coefficient blocks associated with the TUs of the current CU. The video decoder (30) also reconstructs coding blocks of the current CU by adding samples of prediction blocks for the PUs of the current CU to the corresponding samples of transformation blocks for the TUs of the current CU. After reconstructing the coding blocks for each CU of the frame, the video decoder (30) can reconstruct the frame.

[0068] As previously mentioned, video coding achieves video compression primarily using two modes: intra-frame prediction (or intra-prediction) and inter-frame prediction (or inter-prediction). It should be noted that intra-block copy (IBC) can be considered as intra-frame prediction or a third mode. Between the two modes, inter-frame prediction contributes more to coding efficiency than intra-frame prediction by using motion vectors to predict the current video block from a reference video block.

[0069] However, due to continuous improvements in video data capturing technology and finer video block sizes to preserve video data details, the amount of data required to represent motion vectors for the current frame has also increased significantly. One way to overcome this problem is to take advantage of the fact that adjacent CU groups in both spatial and temporal domains not only have similar video data for prediction purposes but also have similar motion vectors between these adjacent CUs. Therefore, motion information from spatially adjacent CUs and / or temporally co-located CUs can be used as an approximation of the motion information (e.g., motion vector) of the current CU by exploring spatial and temporal correlations, also known as the "Motion Vector Predictor (MVP)" of the current CU.

[0070] Instead of encoding the actual motion vector of the current CU into the video bitstream (e.g., the actual motion vector determined by the motion estimator (42) as described above in relation to FIG. 2), the motion vector predictor of the current CU is subtracted from the actual motion vector of the current CU to generate the Motion Vector Difference (MVD) for the current CU. By doing so, there is no need to encode the motion vector determined by the motion estimator (42) into the video bitstream for each CU of the frame, and the amount of data used to represent motion information in the video bitstream can be significantly reduced.

[0071] Similar to the process of selecting a prediction block from a reference frame during inter-frame prediction of a code block, both the video encoder (20) and the video decoder (30) construct a motion vector candidate list (also called a “merge list”) for the current CU using potential candidate motion vectors associated with spatially adjacent CUs and / or temporally identical CUs of the current CU, and then a series of rules must be adopted by both the video encoder (20) and the video decoder (30) to select one member from the motion vector candidate list as a motion vector predictor for the current CU. By doing so, there is no need to transmit the motion vector candidate list itself from the video encoder (20) to the video decoder (30), and the index of the motion vector predictor selected within the motion vector candidate list is sufficient for the video encoder (20) and the video decoder (30) to use the same motion vector predictor within the motion vector candidate list to encode and decode the current CU. Therefore, only the index of the selected motion vector predictor needs to be transmitted from the video encoder (20) to the video decoder (30).

[0072] A brief discussion of bidirectional filtering is provided herein. For the filter kernel of bidirectional filtering, the contribution of each sample within the filtering window depends not only on the spatial distance between samples but also on the difference in intensity between samples. For example, location The sample in is the location You can filter using adjacent samples in. Sample Regarding the filtering of the sample The weight assigned to It can be expressed by the following formula:

[0073] (3)

[0074] In the above equation (3), I and I Each is a sample and Represents the intensity value. represents the spatial intensity parameter. represents the intensity parameter. The intensity of a bidirectional filter is and Controlled by. Position The output sample of a bidirectional filter for can be the weighted average of the samples within the filtering window. For example,

[0075] (4)

[0076] In the above equation (4), is location Represents the output sample in. is a position within a filtering window where 1≤k≤K and 1≤l≤L Represents samples in (e.g., 1≤k≤K and 1≤l≤L). represents pixels within the filtering window). silver Represents the weight for. Product represents the total number of samples within the filtering window.

[0077] In some implementation methods, the picture parameter set raw byte sequence payload (RBSP) syntax, slice header syntax, and coding tree unit syntax for bidirectional filters can be implemented using Tables 1, 2, and 3 below, respectively.

[0078] pic_parameter_set_rbsp( ) { 디스크립터 ... pps_bilateral_filter_enabled_flag u(1) if( pps_bilateral_filter_enabled_flag) { bilateral_filter_strength u(2) bilateral_filter_qp_offset se(v) }

[0079] Table 1 - Picture Parameter Set RBSP Syntax for Bidirectional Filters

[0080] slice_header( ) { 디스크립터 ... if( pps_bilateral_filter_enabled_flag ) { slice_bilateral_filter_all_ctb_enabled_flag u(1) if( !slice_bilateral_filter_all_ctb_enabled_flag ) slice_bilateral_filter_enabled_flag u(1) }

[0081] Table 2 - Slice Header Syntax for Bidirectional Filters

[0082] coding_tree_unit( ) { 디스크립터 ... if( !slice_bilateral_filter_all_ctb_enabled_flag && slice_bilateral_filter_enabled_flag ) bilateral_filter_ctb_flag [ xCtb >> CtbLog2SizeY ][ yCtb >> CtbLog2SizeY ] u(1)

[0083] Table 3 - Coding Tree Unit Syntax for Bidirectional Filters

[0084] A brief discussion of side window filtering (SWF) is also provided herein with reference to FIGS. 5 and 6, wherein FIG. 5 illustrates typical edge types in video frames and FIG. 6 illustrates a typical side filtering window according to some implementations of the present disclosure. In the case of image filtering, it is desirable to smooth out noise while preserving edges and other signal details within the image. In some application scenarios, image filtering methods may be based on a linear approximation assuming that the image is fragmentarily linear. Then, the pixel to be filtered is positioned at the center of a local filtering window, and the corresponding filtered pixel can be calculated as the weighted average of adjacent pixels associated with the pixel to be filtered across the local filtering window. For some edges, e.g., step edges, ramp edges, or roof edges (as illustrated in FIG. 5), the filtering window for each edge may be limited to one side of the edge, so that sharpness at the edge can be preserved after filtering.

[0085] For example, as illustrated in FIG. 5, for pixel "a" on the step edge (510), a side filtering window (512) can be restricted to the left of pixel "a" and can be used to filter pixel "a". For pixel "b" on the step edge (510), a side filtering window (514) can be restricted to the right of pixel "b" and can be used to filter pixel "b". Likewise, for pixel "c" on the ramp edge (520), pixel "c" can be filtered using a side filtering window (522) restricted to one side of pixel "c". For pixel "d" on the ramp edge (520), pixel "d" can be filtered using a side filtering window (524) restricted to one side of pixel "d". Additionally, for pixel "e" on the roof edge (530), pixel "e" can be filtered using a side filtering window (532) that is limited to one side of pixel "e". For pixel "f" on the roof edge (530), pixel "f" can be filtered using a side filtering window (534) that is limited to one side of pixel "f".

[0086] FIG. 6 illustrates various side filtering windows according to some implementations of the present disclosure. For example, section (a) of FIG. 6 illustrates a continuous side filtering window (610), which has three parameters It can be explained as . represents the radius of the side filtering window, represents the angle between the window and the horizon, represents the location of the target pixel (x, y). A fixed pattern side filtering window may be a special case of a continuous filtering window.

[0087] In fact, as illustrated in sections (b)-(d) of FIG. 6, at least eight fixed-pattern side filtering windows may be used to simplify the filtering calculation. For example, section (b) of FIG. 6 shows the left filtering window (L) and the right filtering window (R) of the target pixel (x, y). Section (c) of FIG. 6 shows the upper filtering window (U) and the lower filtering window (D) of the target pixel (x, y). Section (d) of FIG. 6 shows the northeast filtering window (NE), southeast filtering window (SE), southwest filtering window (SW), and northwest filtering window (NW), respectively, of the target pixel (x, y).

[0088] For each pixel on the edge, eight possible filtered outputs can be obtained when applying a filter kernel using eight side filtering windows, as respectively illustrated in sections (b)-(d) of FIG. 6. By selecting one of the eight possible filtered outputs (correspondingly one of the eight side filtering windows) as the output filtering pixel, the distance between the input raw pixel and the output filtering pixel at the edge can be minimized to preserve sharpness at the edge. That is, since the output filtering pixel can be as close as possible to the input raw pixel at the edge, the filtered output having the minimum distance for the input intensity can be selected as the output filtering pixel. Any filter kernel is considered to be applicable to the side window filtering method disclosed herein by adjusting the corresponding filtering window and renormalizing the corresponding filter weights, and the type of filter kernel is not limited in this specification.

[0089] In accordance with the present disclosure, the side window filtering method disclosed herein can improve the coding performance of in-loop filtering beyond VVC and AVS3 standards. In some implementations, bidirectional filtering may work in conjunction with SAO filters to improve coding efficiency. For example, bidirectional filtering techniques are used to illustrate the concept of the side window filtering method in the present disclosure. It is considered that the side window filtering method disclosed herein is not limited to bidirectional filtering and can be applied to any filtering module in modern video coding technology.

[0090] FIG. 7 is a block diagram illustrating an exemplary process for bidirectional filtering using a side filtering window according to some implementations of the present disclosure. In some implementations, the exemplary process of FIG. 7 may be performed by in-loop filters (63) of a video encoder (20) or by in-loop filters (91) of a video decoder (30). In some implementations, the exemplary process of FIG. 7 may be performed by a video processor on the encoder side or the decoder side (e.g., a processor (1320) as shown in FIG. 13). For exemplary purposes only, the following description of the exemplary process of FIG. 7 is provided in relation to a video processor.

[0091] First, a video processor can receive a video frame of a video and use it for in-loop filtering. The video frame may include multiple video blocks, and each video block includes multiple target pixels to be processed. For each target pixel of a video block within the video frame, the video processor can perform side window-based bidirectional filtering (700) on the target pixel.

[0092] For example, for each target pixel, the video processor may perform a bidirectional filtering window determination operation (702) to select a bidirectional filtering window for the target pixel from a group of candidate filtering windows. The group of candidate filtering windows may include a plurality of side filtering windows and full filtering windows. The plurality of side filtering windows may include, for example, an L window, R window, U window, D window, NW window, SE window, SW window, or SE window of the target pixel, as illustrated in FIG. 6. The full filtering window may be, for example, a symmetric filtering window in which the target pixel is located in the middle of the window.

[0093] Next, the video processor can perform a bidirectional filtering operation (704) to filter target pixels of a video block within a video frame using a selected bidirectional filtering window. For example, the video processor can derive filtered pixels for the target pixels based on the selected bidirectional filtering window by applying the above equation (3).

[0094] In the following, a plurality of exemplary implementations for side window-based bidirectional filtering (700) are provided in this specification to derive a corresponding filtered pixel corresponding to each target pixel within a video frame. In the bidirectional filtering disclosed in this specification, the filtering of each target pixel may be a process of assigning weights to all adjacent pixels within a bidirectional filtering window. The bidirectional filtering window may be a full filtering window or one of the side filtering windows disclosed in this specification. One or more criteria may be used to determine the filtering window most suitable for the target pixel, as described in more detail below.

[0095] In the first exemplary implementation of the side window-based bidirectional filtering (700), the video processor can filter a target pixel using a plurality of side filtering windows to obtain a plurality of filtered pixel values. For example, for each side filtering window, the video processor can apply the above equation (3) to derive a filtered pixel value corresponding to the target pixel based on the side filtering window. As a result, a plurality of filtered pixel values ​​can be generated for each of the plurality of side filtering windows.

[0096] Subsequently, the video processor can calculate multiple difference values ​​corresponding to multiple filtered pixel values, wherein each difference value is the difference between the corresponding filtered pixel value and the raw pixel value of the target pixel. The video processor can identify the filtered pixel value associated with the minimum difference value among the multiple difference values ​​from the multiple filtered pixel values ​​as the optimal filtered pixel value. The video processor can select a side filtering window corresponding to the optimal filtered pixel value as a bidirectional filtering window for the target pixel. The video processor can output a filtered pixel for the target pixel using the selected bidirectional filtering window.

[0097] For example, a first exemplary implementation of side window-based bidirectional filtering (700) can be described as follows:

[0098]

[0099] In accordance with the present disclosure, a first exemplary implementation of side window-based bidirectional filtering (700) follows the criterion that an optimal side filtering window can result in an output filtered pixel closest to the raw pixel. In the first exemplary implementation, only a side filtering window is selected, and a full filtering window is not considered.

[0100] In one or more embodiments, a second exemplary implementation of side window-based bidirectional filtering (700) may be formed by modifying a first exemplary implementation. In some implementations, a full filtering window may be incorporated into a group of candidate filtering windows by modifying the first exemplary implementation. That is, in the second exemplary implementation, the group of candidate filtering windows may include a full filtering window. It can be modified as, and here represents a full filtering window. Accordingly, a side filtering window or a full filtering window can be selected for bidirectional filtering.

[0101] In some implementations, additional judgment conditions to increase the confidence in the selection of the side filtering window may be provided in the second exemplary implementation. For example, additional judgment conditions may be added after step 2 of the first exemplary implementation. As described in more detail, (a) the candidate filtering window corresponding to the optimal filtered pixel value in step 2 of the second exemplary implementation is the side filtering window, and (b) if the first difference value between the raw pixel value before filtering and the full window filtered pixel value generated using the full filtering window is clearly greater than the second difference value between the raw pixel value and the optimal filtered pixel value generated using the side filtering window (e.g., first difference value - second difference value ≥ a predetermined threshold). ), a side filtering window corresponding to the optimal filtered pixel value can be selected as a bidirectional filtering window. Otherwise, a full filtering window can be selected as a bidirectional filtering window.

[0102] Specifically, similar to the first exemplary implementation, the video processor may filter the target pixel using a plurality of side filtering windows to obtain a plurality of filtered pixel values ​​in the second exemplary implementation. For example, for each side filtering window, the video processor may apply Equation (3) to derive a filtered pixel value corresponding to the target pixel based on the side filtering window. As a result, a plurality of filtered pixel values ​​may be generated for each of the plurality of side filtering windows.

[0103] Subsequently, the video processor can calculate multiple difference values ​​corresponding to multiple filtered pixel values, wherein each difference value is the difference between the corresponding filtered pixel value and the raw pixel value of the target pixel. The video processor can identify the filtered pixel value corresponding to the minimum difference value among the multiple difference values ​​from the multiple filtered pixel values ​​as the optimal filtered pixel value.

[0104] The video processor may also filter the target pixel with a full filtering window to obtain a full window filtered pixel value. The video processor may calculate a first difference value between the full window filtered pixel value and the original pixel value of the target pixel. The video processor may also calculate a second difference value between the optimal filtered pixel value and the raw pixel value of the target pixel. The video processor may determine whether the first difference value and the second difference value satisfy a predetermined condition. In some implementations, the predetermined condition is that a third difference value between the first difference value and the second difference value is a predetermined threshold value. It may indicate that it is less than a predetermined threshold. For example, the third difference value is less than a predetermined threshold. If less than, a predetermined condition (e.g., 3rd difference = 1st difference - 2nd difference, 3rd difference < a predetermined threshold) ) is satisfied.

[0105] If the first difference value and the second difference value satisfy a predetermined condition, the video processor may select the full filtering window as the bidirectional filtering window. Otherwise, the video processor may select the side filtering window corresponding to the optimal filtering pixel value as the bidirectional filtering window for the target pixel. The video processor may output a filtered pixel for the target pixel using the selected bidirectional filtering window.

[0106] For example, a second exemplary implementation method can be described as follows:

[0107]

[0108] A third exemplary implementation of side window-based bidirectional filtering (700) may include determining a bidirectional filtering window for each target pixel based on an optimal pixel similarity value. For example, as described in more detail below, the average difference value between the target pixel and the pixels within the bidirectional filtering window may be the smallest average difference value among the groups of average difference values ​​associated with the group of candidate filtering windows.

[0109] Specifically, the video processor can determine a group of pixel similarity values ​​for each group of candidate filtering windows. For example, the pixel similarity value for each candidate filtering window is the pixel similarity value between a target pixel and an adjacent pixel within the target pixel's candidate filtering window. The pixel similarity value for each candidate filtering window may be the average of the difference values ​​determined between the raw pixel value of the target pixel and the raw pixel value of an adjacent pixel within the candidate filtering window. Consequently, a group of average difference values ​​is determined for each group of candidate filtering windows and used as the group of pixel similarity values. The video processor can determine the optimal pixel similarity value among the groups of pixel similarity values. For example, the video processor can determine the optimal pixel similarity value as the average of the minimum difference values ​​among the groups of average difference values.

[0110] The video processor may select a candidate filtering window corresponding to the optimal pixel similarity value from a group of candidate filtering windows as a bidirectional filtering window. Specifically, if the candidate filtering window corresponding to the optimal pixel similarity value is a side filtering window, the video processor may select a bidirectional filtering window based on a comparison between the optimal pixel similarity value and the pixel similarity value determined for the full filtering window. For example, the difference between the pixel similarity value for the full filtering window and the optimal pixel similarity value is a predetermined threshold value If it is less than, the video processor may select the full filtering window as a bidirectional filtering window. However, if the difference between the pixel similarity value for the full filtering window and the optimal pixel similarity value is a predetermined threshold In the above case, the video processor can select a side filtering window corresponding to the optimal pixel similarity value as a bidirectional filtering window.

[0111] For example, a third exemplary implementation method can be described as follows:

[0112]

[0113] In some application scenarios, due to the diversity and non-static nature of the video content, side window filtering may not be directly applied to some video blocks. For example, some video blocks may be applied with full window filtering, while others may be applied with side window filtering. To provide flexibility for applying full window filtering or side window filtering at the block level, the video processor may also perform filtered block selection (706) to select an appropriate output filtered block for each video block. For example, a first potential filtering block for a video block may be a full window filtered block having all filtered pixels generated using only the full filtering window. A second potential filtering block for a video block may be a side window filtered block having all pixels generated using the side window-based bidirectional filtering (700) described above. The second potential filtering block may also be any other appropriate filtering block, which is not limited to a filtering block generated using the side window-based bidirectional filtering (700). The output filtered block for the video block can be determined as one of the first and second potential filtering blocks.

[0114] In some implementations, filtered block selection (706) can be performed at different granularity levels. Each video block may be a transform unit (TU), a coding unit (CU), or a coding tree unit (CTU). A video processor may flag each video block to indicate whether the video block is full window filtered or side window filtered. For example, a video processor may flag each video block to indicate whether the output filtered block for the video block is a full window filtered block or a side window filtered block. Filtered block selection (706) is also described in more detail below with reference to FIG. 12.

[0115] For example, since bidirectional filtering can be performed at the TU level, filtered block selection (706) can be performed at the TU level. The following Table 4 provides the syntax elements for filtered block selection at the TU level. For each TU, the element "side_window_flag" can be signaled to indicate whether a full window filtered block or a side window filtered block is used as an output filtered block. For example, a first value of side_window_flag can indicate that a full window filtered block is used as an output filtered block. A second value of side_window_flag can indicate that a side window filtered block is used as an output filtered block. On the encoder side, rate distortion optimization can be applied to determine whether a full window filtered block or a side window filtered block is used as an output filtered block.

[0116] transform_tree( x0, y0, tbWidth, tbHeight , treeType, chType ) { 디스크립터 InferTuCbfLuma = 1 if( IntraSubPartitionsSplitType = = ISP_NO_SPLIT && !cu_sbt_flag ) { if( tbWidth > MaxTbSizeY | | tbHeight > MaxTbSizeY ) { verSplitFirst = ( tbWidth > MaxTbSizeY && tbWidth > tbHeight ) ? 1 : 0 trafoWidth = verSplitFirst ? ( tbWidth / 2 ) : tbWidth trafoHeight = !verSplitFirst ? ( tbHeight / 2 ) : tbHeight transform_tree( x0, y0, trafoWidth, trafoHeight, treeType, chType ) if( verSplitFirst ) transform_tree( x0 + trafoWidth, y0, trafoWidth, trafoHeight, treeType, chType ) else transform_tree( x0, y0 + trafoHeight, trafoWidth, trafoHeight, treeType, chType ) } else { transform_unit( x0, y0, tbWidth, tbHeight, treeType, 0, chType ) if(chType == CHANNEL_TYPE_LUMA && max(tbWidth, tbHeight)<128 && qp>17 &&(isIntra || (isInter && tu_y_coded_flag[ x0 ][ y0 ]&& min(tbWidth, tbHeight)<32))){ side_window_flag u(1) } } } else if( cu_sbt_flag ) { if( !cu_sbt_horizontal_flag ) { trafoWidth = tbWidth * SbtNumFourthsTb0 / 4 transform_unit( x0, y0, trafoWidth, tbHeight, treeType , 0, 0 ) if(chType == CHANNEL_TYPE_LUMA && max(trafoWidth, tbHeight)<128 && qp>17 &&(isIntra || (isInter && tu_y_coded_flag[ x0 ][ y0 ]&& min(trafoWidth, tbHeight)<32))){ side_window_flag u(1) } transform_unit( x0 + trafoWidth, y0, tbWidth - trafoWidth, tbHeight, treeType, 1, 0 ) if(chType == CHANNEL_TYPE_LUMA && max(tbWidth - trafoWidth, tbHeight)<128 && qp>17 &&(isIntra || (isInter && tu_y_coded_flag[ x0+trafoWidth ][ y0 ]&& min(tbWidth - trafoWidth, tbHeight)<32))){ side_window_flag u(1) } } else { trafoHeight = tbHeight * SbtNumFourthsTb0 / 4 transform_unit( x0, y0, tbWidth, trafoHeight, treeType , 0, 0 ) if(chType == CHANNEL_TYPE_LUMA && max(tbWidth, trafoHeight)<128 && qp>17 &&(isIntra || (isInter && tu_y_coded_flag[ x0 ][ y0 ]&& min(tbWidth, trafoHeight)<32))){ side_window_flag u(1) } transform_unit( x0, y0 + trafoHeight, tbWidth, tbHeight - trafoHeight, treeType, 1, 0 ) if(chType == CHANNEL_TYPE_LUMA && max(tbWidth, tbHeight- trafoHeight)<128 && qp>17 &&(isIntra || (isInter && tu_y_coded_flag[ x0 ][ y0+trafoHeight]&& min(tbWidth, tbHeight- trafoHeight)<32))){ side_window_flag u(1) } } } else if( IntraSubPartitionsSplitType = = ISP_HOR_SPLIT ) { trafoHeight = tbHeight / NumIntraSubPartitions for( partIdx = 0; partIdx < NumIntraSubPartitions; partIdx++ ){ transform_unit( x0, y0 + trafoHeight * partIdx, tbWidth, trafoHeight, treeType, partIdx, 0 ) if(chType == CHANNEL_TYPE_LUMA && max(tbWidth, trafoHeight)<128 && qp>17 &&(isIntra || (isInter && tu_y_coded_flag[ x0 ][ y0+ trafoHeight * partIdx ]&& min(tbWidth, trafoHeight)<32))){ side_window_flag u(1) } } } else if( IntraSubPartitionsSplitType = = ISP_VER_SPLIT ) { trafoWidth = tbWidth / NumIntraSubPartitions for( partIdx = 0; partIdx < NumIntraSubPartitions; partIdx++ ){ transform_unit( x0 + trafoWidth * partIdx, y0, trafoWidth, tbHeight, treeType, partIdx, 0 ) if(chType == CHANNEL_TYPE_LUMA && max(trafoWidth, tbHeight)<128 && qp>17 &&(isIntra || (isInter && tu_y_coded_flag[ x0 + trafoWidth * partIdx ][ y0 ]&& min(trafoWidth, tbHeight)<32))){ side_window_flag u(1) } } } }

[0117] Table 4 - Syntax elements of block selection filtered at the TU level

[0118] In another example, filtered block selection (706) can be performed at the CU level. The side_window_flag can be signaled at the CU level. The following Table 5 provides the syntax elements for filtered block selection at the CU level.

[0119] coding_unit( x0, y0, cbWidth, cbHeight, cqtDepth, treeType, modeType ) { 디스크립터 if( sh_slice_type = = I && ( cbWidth > 64 | | cbHeight > 64 ) ) modeType = MODE_TYPE_INTRA ... ... if( cu_coded_flag ) { if( CuPredMode[ chType ][ x0 ][ y0 ] = = MODE_INTER && sps_sbt_enabled_flag && !ciip_flag[ x0 ][ y0 ] && cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY ) { allowSbtVerH = cbWidth >= 8 allowSbtVerQ = cbWidth >= 16 allowSbtHorH = cbHeight >= 8 allowSbtHorQ = cbHeight >= 16 if( allowSbtVerH | | allowSbtHorH ) cu_sbt_flag ae(v) if( cu_sbt_flag ) { if( ( allowSbtVerH | | allowSbtHorH ) && ( allowSbtVerQ | | allowSbtHorQ ) ) cu_sbt_quad_flag ae(v) if( ( cu_sbt_quad_flag && allowSbtVerQ && allowSbtHorQ ) | | ( !cu_sbt_quad_flag && allowSbtVerH && allowSbtHorH ) ) cu_sbt_horizontal_flag ae(v) cu_sbt_pos_flag ae(v) } } if( sps_act_enabled_flag && CuPredMode[ chType ][ x0 ][ y0 ] != MODE_INTRA && treeType = = SINGLE_TREE ) cu_act_enabled_flag ae(v) LfnstDcOnly = 1 LfnstZeroOutSigCoeffFlag = 1 MtsDcOnly = 1 MtsZeroOutSigCoeffFlag = 1 transform_tree( x0, y0, cbWidth, cbHeight, treeType, chType ) lfnstWidth = ( treeType = = DUAL_TREE_CHROMA ) ? cbWidth / SubWidthC : ( ( IntraSubPartitionsSplitType = = ISP_VER_SPLIT ) ? cbWidth / NumIntraSubPartitions : cbWidth ) lfnstHeight = ( treeType = = DUAL_TREE_CHROMA ) ? cbHeight / SubHeightC : ( ( IntraSubPartitionsSplitType = = ISP_HOR_SPLIT) ? cbHeight / NumIntraSubPartitions : cbHeight ) lfnstNotTsFlag = ( treeType = = DUAL_TREE_CHROMA | | !tu_y_coded_flag[ x0 ][ y0 ] | |transform_skip_flag[ x0 ][ y0 ][ 0 ] = = 0 ) && ( treeType = = DUAL_TREE_LUMA | | ( ( !tu_cb_coded_flag[ x0 ][ y0 ] | | transform_skip_flag[ x0 ][ y0 ][ 1 ] = = 0 ) && ( !tu_cr_coded_flag[ x0 ][ y0 ] | | transform_skip_flag[ x0 ][ y0 ][ 2 ] = = 0 ) ) ) if( Min( lfnstWidth, lfnstHeight ) >= 4 && sps_lfnst_enabled_flag = = 1 && CuPredMode[ chType ][ x0 ][ y0 ] = = MODE_INTRA && lfnstNotTsFlag = = 1 && ( treeType = = DUAL_TREE_CHROMA | | !IntraMipFlag[ x0 ][ y0 ] | | Min( lfnstWidth, lfnstHeight ) >= 16 ) && Max( cbWidth, cbHeight ) <= MaxTbSizeY) { if( ( IntraSubPartitionsSplitType != ISP_NO_SPLIT | | LfnstDcOnly = = 0 ) && LfnstZeroOutSigCoeffFlag = = 1 ) lfnst_idx ae(v) } if( treeType != DUAL_TREE_CHROMA && lfnst_idx = = 0 && transform_skip_flag[ x0 ][ y0 ][ 0 ] = = 0 && Max( cbWidth, cbHeight ) <= 32 && IntraSubPartitionsSplitType = = ISP_NO_SPLIT && cu_sbt_flag = = 0 && MtsZeroOutSigCoeffFlag = = 1 && MtsDcOnly = = 0 ) { if( ( ( CuPredMode[ chType ][ x0 ][ y0 ] = = MODE_INTER && sps_explicit_mts_inter_enabled_flag ) | | ( CuPredMode[ chType ][ x0 ][ y0 ] = = MODE_INTRA && sps_explicit_mts_intra_enabled_flag ) ) ) mts_idx ae(v) } } isBIFEnabled = false for(tuIdx=0; tuIdx<NumTUInCU;tuIdx++){ if(bilateral_filter_ctb_flag[ x0 >> CtbLog2SizeY ][ y0 >> CtbLog2SizeY] && max(tbWidth, tbHeight)<128 && qp>17 &&(isIntra || (isInter && tu_y_coded_flag&& min(tbWidth, tbHeight)<32)) ){ isBIFEnabled = true break } } if(isBIFEnabled && chType == CHANNEL_TYPE_LUMA && bilateral_filter_ctb_flag[ x0 >> CtbLog2SizeY ][ y0 >> CtbLog2SizeY ]){ side_window_flag u(1) } }

[0120] Table 5 - Syntax elements of block selection filtered at the CU level

[0121] In another example, since filtered block selection at the TU or CU level can cause overhead, filtered block selection (706) may be performed at the CTU level to avoid overhead. The following Table 6 provides the syntax elements for filtered block selection at the CTU level. If bilateral_filter_ctb_flag is true, side_window_flag is signed to indicate whether side window filtered blocks are used.

[0122] coding_tree_unit( ) { 디스크립터 ... if( !slice_bilateral_filter_all_ctb_enabled_flag && slice_bilateral_filter_enabled_flag ) bilateral_filter_ctb_flag[ xCtb >> CtbLog2SizeY ][ yCtb >> CtbLog2SizeY ] u(1) if(bilateral_filter_ctb_flag[ xCtb >> CtbLog2SizeY ][ yCtb >> CtbLog2SizeY ]){ side_window_flag u(1) }

[0123] Table 6 - Syntax elements of block selection filtered at the CTU level

[0124] In some implementations, the video processor may also perform block-level filtering window selection (708). For example, the video processor may adaptively select a bidirectional filtering window for each video block. For example, at the encoder side, the video processor may determine that each pixel of a video block can be filtered using a full filtering window. Alternatively, the video processor may determine that each pixel within a video block can be filtered using a specific side filtering window. A filtering window index may be signaled to indicate which filtering window is used for the video block. At the decoder side, the filtering window index may be parsed, and then bidirectional filtering may be performed with the filtering window signaled by the filtering window index.

[0125] In accordance with the present disclosure, block-level filtering window selection (708) may also be performed at different particle size levels. The following Table 7 provides the syntax elements for filtering window selection at the TU level. For each TU, filter_window_index may be signaled to indicate the filtering window used for the TU.

[0126] transform_tree( x0, y0, tbWidth, tbHeight , treeType, chType ) { 디스크립터 InferTuCbfLuma = 1 if( IntraSubPartitionsSplitType = = ISP_NO_SPLIT && !cu_sbt_flag ) { if( tbWidth > MaxTbSizeY | | tbHeight > MaxTbSizeY ) { verSplitFirst = ( tbWidth > MaxTbSizeY && tbWidth > tbHeight ) ? 1 : 0 trafoWidth = verSplitFirst ? ( tbWidth / 2 ) : tbWidth trafoHeight = !verSplitFirst ? ( tbHeight / 2 ) : tbHeight transform_tree( x0, y0, trafoWidth, trafoHeight, treeType, chType ) if( verSplitFirst ) transform_tree( x0 + trafoWidth, y0, trafoWidth, trafoHeight, treeType, chType ) else transform_tree( x0, y0 + trafoHeight, trafoWidth, trafoHeight, treeType, chType ) } else { transform_unit( x0, y0, tbWidth, tbHeight, treeType, 0, chType ) if(chType == CHANNEL_TYPE_LUMA && max(tbWidth, tbHeight)<128 && qp>17 &&(isIntra || (isInter && tu_y_coded_flag[ x0 ][ y0 ]&& min(tbWidth, tbHeight)<32))){ filter_window_index ae(v) } } } else if( cu_sbt_flag ) { if( !cu_sbt_horizontal_flag ) { trafoWidth = tbWidth * SbtNumFourthsTb0 / 4 transform_unit( x0, y0, trafoWidth, tbHeight, treeType , 0, 0 ) if(chType == CHANNEL_TYPE_LUMA && max(trafoWidth, tbHeight)<128 && qp>17 &&(isIntra || (isInter && tu_y_coded_flag[ x0 ][ y0 ]&& min(trafoWidth, tbHeight)<32))){ filter_window_index ae(v) } transform_unit( x0 + trafoWidth, y0, tbWidth -trafoWidth, tbHeight, treeType, 1, 0 ) if(chType == CHANNEL_TYPE_LUMA && max(tbWidth - trafoWidth, tbHeight)<128 && qp>17 &&(isIntra || (isInter && tu_y_coded_flag[ x0+trafoWidth ][ y0 ]&& min(tbWidth - trafoWidth, tbHeight)<32))){ filter_window_index ae(v) } } else { trafoHeight = tbHeight * SbtNumFourthsTb0 / 4 transform_unit( x0, y0, tbWidth, trafoHeight, treeType , 0, 0 ) if(chType == CHANNEL_TYPE_LUMA && max(tbWidth, trafoHeight)<128 && qp>17 &&(isIntra || (isInter && tu_y_coded_flag[ x0 ][ y0 ]&& min(tbWidth, trafoHeight)<32))){ filter_window_index ae(v) } transform_unit( x0, y0 + trafoHeight, tbWidth, tbHeight - trafoHeight, treeType, 1, 0 ) if(chType == CHANNEL_TYPE_LUMA && max(tbWidth, tbHeight- trafoHeight)<128 && qp>17 &&(isIntra || (isInter && tu_y_coded_flag[ x0 ][ y0+trafoHeight]&& min(tbWidth, tbHeight- trafoHeight)<32))){ filter_window_index ae(v) } } } else if( IntraSubPartitionsSplitType = = ISP_HOR_SPLIT ) { trafoHeight = tbHeight / NumIntraSubPartitions for( partIdx = 0; partIdx < NumIntraSubPartitions; partIdx++ ){ transform_unit( x0, y0 + trafoHeight * partIdx, tbWidth, trafoHeight, treeType, partIdx, 0 ) if(chType == CHANNEL_TYPE_LUMA && max(tbWidth, trafoHeight)<128 && qp>17 &&(isIntra || (isInter && tu_y_coded_flag[ x0 ][ y0+ trafoHeight * partIdx ]&& min(tbWidth, trafoHeight)<32))){ filter_window_index ae(v) } } } else if( IntraSubPartitionsSplitType = = ISP_VER_SPLIT ) { trafoWidth = tbWidth / NumIntraSubPartitions for( partIdx = 0; partIdx < NumIntraSubPartitions; partIdx++ ){ transform_unit( x0 + trafoWidth * partIdx, y0, trafoWidth, tbHeight, treeType, partIdx, 0 ) if(chType == CHANNEL_TYPE_LUMA && max(trafoWidth, tbHeight)<128 && qp>17 &&(isIntra || (isInter && tu_y_coded_flag[ x0 + trafoWidth * partIdx ][ y0 ]&& min(trafoWidth, tbHeight)<32))){ filter_window_index ae(v) } } } }

[0127] Table 7 - Syntax elements for filtering window selection at the TU level. The following Table 8 provides the syntax elements for filtering window selection at the CU level. It signals filter_window_index to indicate the filtering window used in the CU.

[0128] coding_unit( x0, y0, cbWidth, cbHeight, cqtDepth, treeType, modeType ) { 디스크립터 if( sh_slice_type = = I && ( cbWidth > 64 | | cbHeight > 64 ) ) modeType = MODE_TYPE_INTRA ... ... if( cu_coded_flag ) { if( CuPredMode[ chType ][ x0 ][ y0 ] = = MODE_INTER && sps_sbt_enabled_flag && !ciip_flag[ x0 ][ y0 ] && cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY ) { allowSbtVerH = cbWidth >= 8 allowSbtVerQ = cbWidth >= 16 allowSbtHorH = cbHeight >= 8 allowSbtHorQ = cbHeight >= 16 if( allowSbtVerH | | allowSbtHorH ) cu_sbt_flag ae(v) if( cu_sbt_flag ) { if( ( allowSbtVerH | | allowSbtHorH ) && ( allowSbtVerQ | | allowSbtHorQ ) ) cu_sbt_quad_flag ae(v) if( ( cu_sbt_quad_flag && allowSbtVerQ && allowSbtHorQ ) | | ( !cu_sbt_quad_flag && allowSbtVerH && allowSbtHorH ) ) cu_sbt_horizontal_flag ae(v) cu_sbt_pos_flag ae(v) } } if( sps_act_enabled_flag && CuPredMode[ chType ][ x0 ][ y0 ] != MODE_INTRA && treeType = = SINGLE_TREE ) cu_act_enabled_flag ae(v) LfnstDcOnly = 1 LfnstZeroOutSigCoeffFlag = 1 MtsDcOnly = 1 MtsZeroOutSigCoeffFlag = 1 transform_tree( x0, y0, cbWidth, cbHeight, treeType, chType ) lfnstWidth = ( treeType = = DUAL_TREE_CHROMA ) ? cbWidth / SubWidthC : ( ( IntraSubPartitionsSplitType = = ISP_VER_SPLIT ) ? cbWidth / NumIntraSubPartitions : cbWidth ) lfnstHeight = ( treeType = = DUAL_TREE_CHROMA ) ? cbHeight / SubHeightC : ( ( IntraSubPartitionsSplitType = = ISP_HOR_SPLIT) ? cbHeight / NumIntraSubPartitions : cbHeight ) lfnstNotTsFlag = ( treeType = = DUAL_TREE_CHROMA | | !tu_y_coded_flag[ x0 ][ y0 ] | | transform_skip_flag[ x0 ][ y0 ][ 0 ] = = 0 ) && ( treeType = = DUAL_TREE_LUMA | | ( ( !tu_cb_coded_flag[ x0 ][ y0 ] | | transform_skip_flag[ x0 ][ y0 ][ 1 ] = = 0 ) && ( !tu_cr_coded_flag[ x0 ][ y0 ] | | transform_skip_flag[ x0 ][ y0 ][ 2 ] = = 0 ) ) ) if( Min( lfnstWidth, lfnstHeight ) >= 4 && sps_lfnst_enabled_flag = = 1 && CuPredMode[ chType ][ x0 ][ y0 ] = = MODE_INTRA && lfnstNotTsFlag = = 1 && ( treeType = = DUAL_TREE_CHROMA | | !IntraMipFlag[ x0 ][ y0 ] | | Min( lfnstWidth, lfnstHeight ) >= 16 ) && Max( cbWidth, cbHeight ) <= MaxTbSizeY) { if( ( IntraSubPartitionsSplitType != ISP_NO_SPLIT | | LfnstDcOnly = = 0 ) && LfnstZeroOutSigCoeffFlag = = 1 ) lfnst_idx ae(v) } if( treeType != DUAL_TREE_CHROMA && lfnst_idx = = 0 && transform_skip_flag[ x0 ][ y0 ][ 0 ] = = 0 && Max( cbWidth, cbHeight ) <= 32 && IntraSubPartitionsSplitType = = ISP_NO_SPLIT && cu_sbt_flag = = 0 && MtsZeroOutSigCoeffFlag = = 1 && MtsDcOnly = = 0 ) { if( ( ( CuPredMode[ chType ][ x0 ][ y0 ] = = MODE_INTER && sps_explicit_mts_inter_enabled_flag ) | | ( CuPredMode[ chType ][ x0 ][ y0 ] = = MODE_INTRA && sps_explicit_mts_intra_enabled_flag ) ) ) mts_idx ae(v) } } isBIFEnabled = false for(tuIdx=0; tuIdx<NumTUInCU;tuIdx++){ if(bilateral_filter_ctb_flag[ x0 >> CtbLog2SizeY ][ y0 >> CtbLog2SizeY ] && max(tbWidth, tbHeight)<128 && qp>17 &&(isIntra || (isInter && tu_y_coded_flag&& min(tbWidth, tbHeight)<32)) ){ isBIFEnabled = true break } if(isBIFEnabled &&chType == CHANNEL_TYPE_LUMA&& bilateral_filter_ctb_flag[ x0 >> CtbLog2SizeY ][ y0 >> CtbLog2SizeY ]){ side_window_index ae(v) } }

[0129] Table 8 - Syntax elements of filtering window selection at the CU level

[0130] Table 9 below provides the syntax elements for selecting a filtering window at the CTU level. When bidirectional filtering is enabled for a CTU, filter_window_index can be signaled to indicate the filtering window used for the CTU.

[0131] coding_tree_unit( ) { 다스크립터 ... if( !slice_bilateral_filter_all_ctb_enabled_flag && slice_bilateral_filter_enabled_flag ) bilateral_filter_ctb_flag[ xCtb >> CtbLog2SizeY ][ yCtb >> CtbLog2SizeY ] u(1) if(lateral_filter_ctb_flag[ xCtb >> CtbLog2SizeY ][ yCtb >> CtbLog2SizeY ]){ filter_window_index ae(v) }

[0132] Table 9 - Syntax elements of filtering window selection at the CTU level

[0133] In accordance with the present disclosure, a fixed pattern side filtering window can be described by a filter window index as described by the following Table 10. When a fixed pattern side filtering window is applied in video coding, the syntax element "filter_window_index" can be used to indicate the filtering window type of the fixed pattern side filtering window.

[0134] Filtering window type Filter Window Index full_window (F) 0 upper_window (U) 1 lower_window (D) 2 left_window (L) 3 right_window (R) 4 northwest_window (NW) 5 northeast_window (NE) 6 southwest_window (SW) 7 southeast_window (SE) 8

[0135] Table 10: Fixed Pattern Side Filtering Window and Corresponding Filter Window Index

[0136] FIG. 8 is a flowchart of an exemplary method (800) for bidirectional filtering using a side filtering window according to some implementations of the present disclosure. The method (800) may be implemented by a video processor associated with a video encoder (20) or a video decoder (30) and may include steps (802 to 804) as described below. Some steps may be optional to perform the disclosure provided herein. Furthermore, some steps may be performed simultaneously or in a different order than shown in FIG. 8.

[0137] In step (802), the video processor can receive video frames of video for in-loop filtering.

[0138] In step (804), for a target pixel of a video frame, the video processor may perform in-loop filtering on the target pixel by selecting a bidirectional filtering window from a group of candidate filtering windows. The group of candidate filtering windows may include a plurality of side filtering windows and pool filtering windows.

[0139] In step (806), the video processor can filter target pixels of the video frame using a selected bidirectional filtering window.

[0140] FIG. 9 is a flowchart of a method (900) which is a first exemplary implementation of a method (800) for bidirectional filtering using the side filtering windows of FIG. 8, according to some implementations of the present disclosure. The method (900) may be implemented by a video processor associated with a video encoder (20) or a video decoder (30) and may include steps (902) through (912) as described below. Specifically, steps (904) through (910) of the method (800) may be performed as an exemplary implementation of step (804) of the method (800). Some steps may be optional to perform the disclosure provided herein. Furthermore, some steps may be performed simultaneously or in a different order than that shown in FIG. 9.

[0141] In step (902), the video processor can receive video frames of video for in-loop filtering.

[0142] In step (904), the video processor may obtain a plurality of filtered pixel values ​​for a target pixel of a video frame by filtering the target pixel with a plurality of side filtering windows. Implementation methods of step (904) are described in relation to FIG. 7. For example, step (904) may be performed according to step 1 of the first exemplary implementation method.

[0143] In step (906), the video processor can calculate the difference between each filtered pixel value and the raw pixel value of the target pixel to generate multiple difference values ​​for each of the multiple filtered pixel values.

[0144] In step (908), the video processor can identify the filtered pixel value associated with the minimum difference value among the multiple difference values ​​from the multiple filtered pixel values ​​as the optimal filtered pixel value.

[0145] Implementation methods of steps (906) and (908) are described in relation to FIG. 7. For example, steps (906) and (908) may be performed according to step 2 of the first exemplary implementation method.

[0146] In step (910), the video processor can select a side filtering window corresponding to the optimal filtered pixel value from a plurality of filtered pixel values ​​as a bidirectional filtering window.

[0147] In step (912), the video processor can filter target pixels of the video frame using a selected bidirectional filtering window.

[0148] Implementation methods of steps (902) and (912) are described in relation to FIG. 7. For example, steps (902) and (912) may be performed according to the return step of the first exemplary implementation method.

[0149] FIG. 10 is a flowchart of a method (1000) which is a second exemplary implementation of a method (800) for bidirectional filtering using a side filtering window of FIG. 8, according to a partial implementation of the present disclosure. The method (1000) may be implemented by a video processor associated with a video encoder (20) or a video decoder (30) and may include steps (1002) through (1018) as described below. Specifically, steps (1004) through (1016) of the method (1000) may be performed as an exemplary implementation of step (804) of the method (800). Some steps may be optional to perform the disclosure provided herein. Furthermore, some steps may be performed simultaneously or in a different order than that shown in FIG. 10.

[0150] In step (1002), the video processor can receive video frames of video for in-loop filtering.

[0151] In step (1004), the video processor may obtain a plurality of filtered pixel values ​​for a target pixel of a video frame by filtering the target pixel with a plurality of side filtering windows. Implementation methods of step (1004) are described in relation to FIG. 7. For example, step (1004) may be performed according to step 1 of the second exemplary implementation method.

[0152] In step (1006), the video processor can calculate the difference between each filtered pixel value and the raw pixel value of the target pixel to generate multiple difference values ​​for each of the multiple filtered pixel values.

[0153] In step (1008), the video processor can identify the filtered pixel value associated with the minimum difference value among the plurality of difference values ​​from the plurality of filtered pixel values ​​as the optimal filtered pixel value.

[0154] Implementation methods of steps (1006) and (1008) are described in relation to FIG. 7. For example, steps (1006) and (1008) may be performed according to step 2 of the second exemplary implementation method.

[0155] In step (1010), the video processor can calculate (a) a first difference value between a full window filtered pixel value and a raw pixel value of a target pixel, and (b) a second difference value between an optimal filtered pixel value and a raw pixel value of a target pixel.

[0156] In step (1012), the video processor can determine whether the first difference value and the second difference value satisfy a predetermined condition. If the first difference value and the second difference value satisfy the predetermined condition, the method (1000) can proceed to step (1014). Otherwise, the method (1000) can proceed to step (1016). In step (1014), the video processor can select the full filtering window as a bidirectional filtering window.

[0157] In step (1016), the video processor can select a side filtering window corresponding to the optimal filtered pixel value as a bidirectional filtering window.

[0158] Implementation methods of steps (1010) through (1016) are described in relation to FIG. 7. For example, steps (1010) through (1016) may be performed according to step 3 of the second exemplary implementation method.

[0159] In step (1018), the video processor can filter target pixels of a video frame using a selected bidirectional filtering window.

[0160] FIG. 11 is a flowchart of a method (1100) which is a third exemplary implementation of a method (800) for bidirectional filtering using the side filtering windows of FIG. 8, according to some implementations of the present disclosure. The method (1100) may be implemented by a video processor associated with a video encoder (20) or a video decoder (30) and may include steps (1102) through (1116) as described below. Specifically, steps (1104) through (1114) of the method (1100) may be performed as an exemplary implementation of step (804) of the method (800). Some steps may be optional to perform the disclosure provided herein. Furthermore, some steps may be performed simultaneously or in a different order than that shown in FIG. 11.

[0161] In step (1102), the video processor can receive video frames of video for in-loop filtering.

[0162] In step (1104), for a target pixel of a video frame, the video processor may determine a pixel similarity value for each candidate filtering window between the target pixel and adjacent pixels belonging to the candidate filtering window of the target pixel. Consequently, a group of pixel similarity values ​​is determined for each group of candidate filtering windows. A group of candidate filtering windows may include a plurality of side filtering windows and full filtering windows. An implementation method of step (1104) is described in relation to FIG. 7. For example, step (1104) may be performed according to step 1 of the third exemplary implementation method.

[0163] In step (1106), the video processor may determine a candidate filtering window corresponding to the optimal pixel similarity value among a group of pixel similarity values. An implementation method of step (1106) is described in relation to FIG. 7. For example, step (1106) may be performed according to step 2 of the third exemplary implementation method.

[0164] In step (1108), the video processor can determine whether the candidate filtering window corresponding to the optimal pixel similarity value is a side filtering window. If the candidate filtering window corresponding to the optimal pixel similarity value is a side filtering window, the method (1100) can proceed to step (1110). Otherwise, the method (1100) can proceed to step (1112).

[0165] In step (1110), the video processor can determine whether the difference between the pixel similarity value for the full filtering window and the optimal pixel similarity value is less than a predetermined threshold. If the difference between the pixel similarity value for the full filtering window and the optimal pixel similarity value is less than a predetermined threshold, the method (1110) can proceed to step (1112). Otherwise, the method (1100) can proceed to step (1114).

[0166] In step (1112), the video processor can select the full filtering window as a bidirectional filtering window.

[0167] In step (1114), the video processor can select a side filtering window corresponding to the optimal pixel similarity value as a bidirectional filtering window.

[0168] Implementation methods of steps (1108) through (1114) are described in relation to FIG. 7. For example, steps (1108) through (1114) may be performed according to step 3 of the third exemplary implementation method.

[0169] In step (1116), the video processor can filter target pixels of a video frame using a selected bidirectional filtering window.

[0170] FIG. 12 is a flowchart of an exemplary method (1200) for selecting a filtered block for a video block according to some implementations of the present disclosure. The method (1200) may be implemented by a video processor associated with a video encoder (20) or a video decoder (30) and may include steps (1202) through (1212) as described below. Some steps may be optional to perform the disclosure provided herein. Furthermore, some steps may be performed simultaneously or in a different order than shown in FIG. 12.

[0171] In step (1202), the video processor may perform side window-based bidirectional filtering on each target pixel from a plurality of target pixels within a video block to generate a corresponding first filtered pixel. As a result, a plurality of first filtered pixels are generated for a plurality of target pixels within a video block. For example, for each target pixel within a video block, the video processor may perform method (800, 900, 1000, or 1100) to filter the target pixel so that a corresponding first filtered pixel is generated for the target pixel.

[0172] In step (1204), the video processor can generate a side window filtered block containing a plurality of first filtered pixels.

[0173] In step (1206), the video processor can generate a corresponding second filtered pixel by performing bidirectional filtering on each of the multiple target pixels using the corresponding pool filtering window of the target pixel. As a result, multiple second filtered pixels are generated for the multiple target pixels within the video block.

[0174] In step (1208), the video processor can generate a full window filtered block containing a plurality of second filtered pixels.

[0175] In step (1210), the video processor may select a side window filtered block or a full window filtered block as the optimal filtered block for the video block. For example, the video processor may select a side window filtered block or a full window filtered block as the optimal filtered block based on rate distortion optimization.

[0176] In step (1212), the video processor may flag the video block based on the optimal filtered block of the video block to indicate whether the video block is full window filtered or side window filtered. For example, if the optimal filtered block is a side window filtered block, the video processor may flag the video block as side window filtered. If the optimal filtered block is a full window filtered block, the video processor may flag the video block as full window filtered.

[0177] FIG. 13 illustrates a computing environment (1310) combined with a user interface (1350) according to some implementation methods of the present disclosure. The computing environment (1310) may be part of a data processing server. The computing environment (1310) includes a processor (1320), memory (1330), and an input / output (I / O) interface (1340).

[0178] The processor (1320) generally controls the overall operations of the computing environment (1310), such as operations related to display, data acquisition, data communication, and image processing. The processor (1320) may include one or more processors for executing instructions to perform all or part of the steps of the methods described above. Furthermore, the processor (1320) may include one or more modules that facilitate interaction between the processor (1320) and other components. The processor may be a CPU (Central Processing Unit), a microprocessor, a single-chip machine, a GPU (Graphical Processing Unit), etc.

[0179] Memory (1330) is configured to store various types of data to support the operation of the computing environment (1310). Memory (1330) may include predetermined software (1332). Examples of such data include instructions for any application or method operating in the computing environment (1310), video data sets, image data, etc. Memory (1330) may be implemented by using any type of volatile or non-volatile memory device or combination thereof, such as SRAM (Static Random Access Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), ROM (Read-Only Memory), magnetic memory, flash memory, magnetic disk, or optical disk.

[0180] The I / O interface (1340) provides an interface between the processor (1320) and peripheral interface modules such as a keyboard, click wheel, buttons, etc. Buttons may include, but are not limited to, a home button, a scan start button, and a scan stop button. The I / O interface (1340) may be combined with an encoder and a decoder.

[0181] In some implementations, a non-transient computer-readable storage medium is further provided, comprising a plurality of programs for performing the methods described above, for example, in memory (1330), which are executable by a processor (1310) in a computing environment (1310). Alternatively, the non-transient computer-readable storage medium may store a bitstream or data stream containing encoded video information (e.g., video information including one or more syntax elements) that a decoder (e.g., video decoder (30) in FIG. 3) generated by an encoder (e.g., video encoder (20) in FIG. 2) using the coding method described above) uses when decoding video data. The non-transient computer-readable storage medium may be, for example, ROM, RAM (Random Access Memory), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0182] In some implementations, a computing device is further provided comprising one or more processors (e.g., processor (1320)) and a non-transient computer-readable storage medium or memory (1330) for storing a plurality of programs executable by one or more processors, wherein the one or more processors are configured to perform the methods described above when executing the plurality of programs.

[0183] In some implementations, a computer program product is further provided that includes a plurality of programs for performing the above-described methods, for example, in memory (1330), which are executable by a processor (1320) in a computing environment (1310). For example, the computer program product may include a non-transient computer-readable storage medium.

[0184] In some implementations, the computing environment (1310) may be implemented with one or more ASICs, DSPs, Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0185] The description of the present disclosure is provided for illustrative purposes only and is not intended to encompass or limit the entirety of the disclosure. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art who benefit from the implications presented in the foregoing description and the accompanying drawings.

[0186] Unless specifically stated otherwise, the order of the steps of the method according to the present disclosure is merely illustrative and is not limited to the order specifically described above, but may be modified according to practical conditions. Additionally, at least one of the steps of the method according to the present disclosure may be adjusted, combined, or omitted according to actual requirements.

[0187] The embodiments have been selected and described to explain the principles of disclosure and to enable those skilled in the art to understand the disclosure regarding various modes of implementation, and to best utilize the basic principles and various modes of implementation having various modifications suitable for the specific use being considered. Accordingly, the scope of the disclosure is not limited to specific examples of the disclosed modes of implementation, and modifications and other modes of implementation should be understood as intended to be included within the scope of the disclosure.

Claims

Claim 1 A video processing method using bidirectional filtering, characterized by comprising: receiving a video frame of video for in-loop filtering by one or more processors; performing in-loop filtering on a target pixel of the video frame by selecting a bidirectional filtering window from a group of candidate filtering windows including a plurality of side filtering windows and a pool filtering window by one or more processors for the target pixel of the video frame; and filtering the target pixel of the video frame using the selected bidirectional filtering window by one or more processors. Claim 2 A video processing method according to claim 1, wherein the step of selecting the bidirectional filtering window for the target pixel further comprises: a step of filtering the target pixel with the plurality of side filtering windows to obtain each of the plurality of filtered pixel values; and a step of selecting the side filtering window corresponding to the optimal filtered pixel value among the plurality of filtered pixel values ​​as the bidirectional filtering window. Claim 3 A video processing method according to claim 2, wherein the step of selecting a side filtering window corresponding to the optimal filtered pixel value further comprises: a step of calculating a plurality of difference values ​​corresponding to the plurality of filtered pixel values, wherein each difference value is a value between the corresponding filtered pixel value and the raw pixel value of the target pixel; and a step of identifying a filtered pixel value associated with the minimum difference value among the plurality of difference values ​​from the plurality of filtered pixel values ​​as the optimal filtered pixel value. Claim 4 A video processing method according to claim 2, wherein the step of selecting a bidirectional filtering window for the target pixel further comprises: a step of filtering the target pixel with the full filtering window to obtain a full window filtered pixel value; a step of calculating a first difference value between the full window filtered pixel value and the raw pixel value of the target pixel; a step of calculating a second difference value between the optimal filtered pixel value and the raw pixel value of the target pixel; and a step of selecting the full filtering window as the bidirectional filtering window when the first difference value and the second difference value satisfy a predetermined condition. Claim 5 A video processing method according to claim 4, wherein the predetermined condition is that the third difference value between the first difference value and the second difference value is less than a predetermined threshold value. Claim 6 A video processing method according to claim 1, wherein the step of selecting a bidirectional filtering window for the target pixel further comprises: determining a group of pixel similarity values ​​for a group of candidate filtering windows, wherein the pixel similarity value for each candidate filtering window within the group of candidate filtering windows is a pixel similarity value between the target pixel and adjacent pixels belonging within the corresponding candidate filtering window of the target pixel; and selecting a candidate filtering window corresponding to the optimal pixel similarity value among the group of pixel similarity values ​​from the group of candidate filtering windows as a bidirectional filtering window. Claim 7 A video processing method according to claim 6, characterized in that the pixel similarity value for each candidate filtering window within the group of candidate filtering windows is the average value of the difference values ​​determined between the pixel value of the target pixel and the pixel values ​​of adjacent pixels belonging within the corresponding candidate filtering window of the target pixel. Claim 8 A video processing method according to claim 6, wherein the step of selecting a bidirectional filtering window for the target pixel further includes the step of selecting the bidirectional filtering window based on a comparison of the optimal pixel similarity value and the pixel similarity value determined for the full filtering window when the candidate filtering window corresponding to the optimal pixel similarity value is a side filtering window. Claim 9 A video processing method according to claim 8, wherein the step of selecting the bidirectional filtering window based on a comparison of the optimal pixel similarity value and the pixel similarity value determined for the full filtering window further includes the step of selecting the full filtering window as the bidirectional filtering window when the difference between the pixel similarity value for the full filtering window and the optimal pixel similarity value is less than a predetermined threshold value. Claim 10 A video processing method according to claim 1, wherein the video frame comprises a plurality of video blocks, and the video processing method further comprises the step of flagging each video block to indicate whether the video block is full window filtered or side window filtered. Claim 11 A video processing method according to claim 10, characterized in that each video block is a conversion unit (TU), a coding unit (CU), or a coding tree unit (CTU). Claim 12 A video processing method according to claim 1, characterized in that each side filtering window has a filtering window type pointed to by a filtering window index. Claim 13 A video processing device utilizing bidirectional filtering, comprising: one or more processors; and a memory configured to store instructions executable by one or more processors; wherein, when the instructions are executed, the one or more processors are configured to perform a method according to any one of claims 1 to 12. Claim 14 A non-transient computer-readable storage medium storing computer-executable instructions that, when executed by one or more computer processors, cause one or more computer processors to perform a method according to any one of claims 1 through 12 and store a bitstream decoded by a method according to any one of claims 1 through 12 or a bitstream generated by a method according to any one of claims 1 through 12. Claim 15 A non-transient computer-readable storage medium storing a bitstream decoded by a method according to any one of claims 1 to 12 or a bitstream generated by a method according to any one of claims 1 to 12. Claim 16 A computer program stored in a non-transient computer-readable storage medium having instructions for storing a bitstream, wherein the bitstream comprises: encoded video data to be decoded by a method according to any one of claims 1 to 12; or encoded video data generated by a method according to any one of claims 1 to 12. Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 delete