Efficient in-loop filtering for video coding

Abstract

Various schemes for implementing efficient in-loop filtering are described, which are manifested in low latency and reduced hardware cost for in-loop filters comprising at least two filtering stages. An apparatus receives pixel data of a current block of a picture and one or more neighboring blocks of the current block, performs a filtering operation based on the pixel data and generates a filtered block comprising fully filtered sub-blocks and partially filtered sub-blocks. The apparatus further outputs an output block comprising the fully filtered sub-blocks and a respective portion of each of the partially filtered sub-blocks, wherein the respective portion is adjacent to one of the fully filtered sub-blocks.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to video coding, and more particularly, to methods and apparatus for improving operational efficiency of in-loop filtering within a video encoder or decoder. BACKGROUND

[0002] Unless otherwise stated in this document, the methods described in this section are not prior art to the claims listed below and are not admitted to be prior art by inclusion in this section.

[0003] Video coding generally involves encoding a video (i.e., an original video) into a bitstream by an encoder, transmitting the bitstream to a decoder, and decoding the video from the bitstream and processing the bitstream by the decoder to produce a reconstructed video. The encoder can employ various coding modes or tools when encoding the video, among others, with the purpose of reducing the total size of the bitstream that needs to be transmitted to the decoder, while still providing the decoder with sufficient information about the original video so that the decoder can generate a reconstructed video that is satisfactorily faithful to the original video. However, various coding tools, while achieving their respective coding purposes, can all introduce visual artifacts in the encoding and decoding process. Specifically, many of the coding tools are block-based coding tools, which means that the pictures or frames to be coded are divided into many non-overlapping rectangular regions, referred to as “blocks”. The blocks form the basic elements that the coding tools process, as often seen in the two main techniques of intra prediction and inter prediction used in video coding. Since these blocks are coded independently from each other, visual artifacts can occur when putting the coded blocks together to reconstruct a picture, for example, at the edges of the blocks.

[0004] A loop filter can be applied to the reconstructed picture to process or otherwise mitigate the above-mentioned visual artifacts, thereby improving the subjective quality of the reconstructed picture. The loop filter typically includes several individual filters that each mitigate different visual artifacts. For example, one filter can be used to smooth block edges, while another filter is used to reduce ringing artifacts or variations in sample intensity in certain regions of the picture. The individual filters can operate in a pipeline fashion, with each filter being a stage of the pipeline. That is, a first filter of the loop filter can process a first block of the reconstructed picture, and then pass the results to the next stage, which is a second filter of the loop filter. While the second filter operates on the results (e.g., pixel data) passed from the first filter, the first filter can process a second block of the reconstructed picture, and so on. The pipeline is referred to as a loop filter because it is applied to both the encoding loop of a video encoder and the decoding loop of a video decoder. Specifically, the loop filter is applied after inverse quantization (IQ) but before the picture is stored to the decoded picture buffer (DPB). At the decoding side, the improvement in reconstructed picture quality provides a more desirable reference picture for the decoding process, which translates to higher compression efficiency.

[0005] As the loop filter is used in both the encoding loop and the decoding loop, it forms an important part of a video encoder. Therefore, the efficiency of the loop filter has a significant impact on the overall codec efficiency (e.g., latency and throughput). The various techniques demonstrated in this disclosure can be used to improve the efficiency of the loop filter, and thus the efficiency of a video encoder or decoder that employs the efficient loop filter. SUMMARY

[0006] The following summary is illustrative only and is not intended to be limiting in any way. That is, the following summary is provided to introduce some of the concepts, highlights, benefits, and advantages of the novel and non-obvious technology described herein. The implementation(s) selected are further described in the detailed description below. Thus, the following summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used in determining the scope of the claimed subject matter.

[0007] It is an object of the present disclosure to provide schemes, concepts, designs, techniques, methods, and apparatuses related to in-loop filtering or subsequent processing of video picture frames after the frames are reconstructed using block-based intra- or inter-prediction but before being stored in a decoded picture buffer (DPB). It is believed that through various embodiments in the present disclosure, benefits including improved codec latency, higher codec throughput, and / or reduced hardware overhead are realized.

[0008] In one aspect, a method for performing a filtering operation on a current block of a picture is presented, where the picture can be a reconstructed picture of a video encoder or a video decoder that uses block-based intra- or inter-prediction. The method can involve receiving pixel data of the current block and pixel data of one or more neighboring blocks of the current block. In some embodiments, the one or more neighboring blocks can include a top-adjacent block that is immediately above the current block and a left-adjacent block that is immediately to the left of the current block. The method can also involve generating a filtered block by performing a filtering operation on the current block. The filtering operation can be performed using the pixel data of the current block and the pixel data of the one or more neighboring blocks. The filtered block can include one or more fully filtered sub-blocks and one or more partially filtered sub-blocks. Finally, the method can also involve outputting an output block. The output block can include a respective portion of each of the one or more fully filtered sub-blocks and the one or more partially filtered sub-blocks, where the respective portion is edge-adjacent or corner-adjacent to a respective one or more fully filtered sub-block edge. The respective portions of the partially filtered sub-blocks can collectively form an L-shaped region having a first segment and a second segment. In some embodiments, a width of each of the first segment and the second segment of the L-shaped region is related to a filtering setting used in the filtering operation. For example, depending on the filtering setting, the filtering operation can involve referring to a first number of pixels and modifying a second number of pixels. The width of the L-shaped region can be equal to a difference between the first number and the second number. In some embodiments, the filtering operation (e.g., a deblocking operation) can be performed by a first filter, and the output block can be sent to a second filter for an additive filtering operation (e.g., a deringing operation). The first and second filters are configured to process blocks of the picture in a block-based pipelined manner.

[0009] In another aspect, an apparatus is presented that includes an input module, a neighboring block buffer, a deblocking filter, a filtered block buffer, and an output module. The input module is configured to receive a current block of a picture. The neighboring block buffer is configured to store a first set of pixel data of one or more neighboring blocks of the current block. The deblocking filter is configured to perform a deblocking operation on the current block, and the deblocking operation can be performed using pixel data of the current block and the first set of pixel data of the one or more neighboring blocks. The filtered block buffer is configured to store one or more fully filtered sub-blocks and one or more partially filtered sub-blocks. Further, the output module is configured to output an output block based on the deblocking operation performed by the deblocking filter. Specifically, the output block can include the one or more fully filtered sub-blocks and a respective portion of each of the one or more partially filtered sub-blocks. Further, each of the respective portions can be adjacent (e.g., edge-adjacent or corner-adjacent) to a corresponding one of the fully filtered sub-blocks. In some embodiments, the respective portions of the partially filtered sub-blocks collectively form an L-shaped region. The L-shaped region can be formed by two rectangular segments, each having a respective width. The widths of the segments can be determined by one or more filter settings used with the deblocking operation. For example, the deblocking operation can involve referencing a first number of pixels and modifying a second number of pixels, and the widths of the segments of the L-shaped region can equal a difference between the first number and the second number. In some embodiments, the neighboring block buffer is further configured to store a second set of pixel data of the one or more neighboring blocks, and the apparatus further includes a de-ringing filter configured to perform a de-ringing operation on the one or more fully filtered sub-blocks using pixel data of the output block and the second set of pixel data of the one or more neighboring blocks. The de-ringing operation can also be performed using specific filter settings. For example, the de-ringing operation can involve referencing a specific number of pixels adjacent to each pixel of the output block in each of a left direction, a right direction, an up direction, and a down direction. The de-ringing operation can then involve modifying a respective pixel of the output block based on the reference. In some embodiments, the widths of the segments of the L-shaped region can equal the specific number. In some embodiments, the specific number also equals a difference between a first number and a second number of the filter settings used by the deblocking filter. In cases where the filter settings of the deblocking filter are different from the filter settings of the de-ringing filter, i.e., the specific number is different from the difference between the first number and the second number, the L-shaped region can be determined to be a lower value between the specific number (as used in the de-ringing operation) and the difference between the first number and the second number (as used in the deblocking operation). BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this disclosure. The drawings illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure. It is to be noted that the drawings are not necessarily drawn to scale since certain elements can be shown exaggerated in scale or in somewhat schematic form, and the specification can also refer to or rely on portions of the drawings in order to describe one or more embodiments of the present disclosure.

[0011] Figure 1 FIG. 1 shows a diagram of an example design in accordance with an embodiment of the present disclosure.

[0012] Figure 2 FIG. 2 shows a diagram of an example design in accordance with an embodiment of the present disclosure.

[0013] Figure 3 FIG. 3 shows a diagram of an example design in accordance with an embodiment of the present disclosure.

[0014] Figure 4 FIG. 4 shows a diagram of an example design in accordance with an embodiment of the present disclosure.

[0015] Figure 5 FIG. 5 shows a diagram of an example design in accordance with an embodiment of the present disclosure.

[0016] Figure 6 FIG. 6 shows a diagram of an example design in accordance with an embodiment of the present disclosure.

[0017] Figure 7 FIG. 7 shows a diagram of an example design in accordance with an embodiment of the present disclosure.

[0018] Figure 8 FIG. 8 shows a diagram of an example design in accordance with an embodiment of the present disclosure.

[0019] Figure 9 FIG. 9 shows a diagram of an example design in accordance with an embodiment of the present disclosure.

[0020] Figure 10 FIG. 10 shows a diagram of an example design in accordance with an embodiment of the present disclosure.

[0021] Figure 11 FIG. 11 shows a diagram of an example design in accordance with an embodiment of the present disclosure.

[0022] Figure 12 FIG. 12 shows a diagram of an example design in accordance with an embodiment of the present disclosure.

[0023] Figure 13 FIG. 13 shows a diagram of an example video encoder in accordance with an embodiment of the present disclosure.

[0024] Figure 14 FIG. 14 shows a diagram of an example video decoder in accordance with an embodiment of the present disclosure.

[0025] Figure 15FIG. shows a diagram illustrating an example loop filter according to embodiments of the disclosure.

[0026] Figure 16 FIG. shows a flowchart illustrating an example process according to embodiments of the disclosure.

[0027] Figure 17 FIG. shows a flowchart illustrating an example process according to embodiments of the disclosure.

[0028] Figure 18 FIG. shows a diagram illustrating an example electronic system according to embodiments of the disclosure. DETAILED DESCRIPTION

[0029] Detailed embodiments and implementations of claimed subject matter are disclosed herein. It should be understood, however, that the disclosed embodiments and implementations are merely examples of the claimed subject matter. Yet, the disclosure can be embodied in many different forms and should not be construed as limited to the exemplary embodiments and implementations set forth herein. Rather, these exemplary embodiments and implementations are provided so that this disclosure will be thorough and complete, and fully convey the scope of the disclosure to those skilled in the art. In the following description, details of well-known features and techniques can be omitted to avoid unnecessarily obscuring the presented embodiments and implementations.

[0030] Embodiments according to the disclosure relate to various techniques, methods, schemes and / or solutions related to implementing efficient loop filtering for video encoders or decoders. According to the disclosure, a number of possible solutions can be implemented individually or jointly. That is, although these possible solutions can be described individually below, two or more of these possible solutions can be implemented in one combination or another combination.

[0031] I. Block-based loop filtering

[0032] Loop filters are an essential part of video encoders or decoders. Figure 1 FIG. shows a diagram illustrating an example design according to embodiments of the disclosure, in which a loop filter 100 is shown together with its relation to some other blocks of a video encoder or decoder. As described elsewhere herein above, the loop filter can comprise several filtering stages, each configured to perform a respective filtering operation, which together operate as a block-based filtering pipeline for processing a reconstructed picture generated by intra- or inter-prediction to enhance its subjective image quality. As described elsewhere herein above, the loop filter can be configured to perform a number of different filtering operations, such as a deblocking filter, a sample adaptive offset (SAO) filter, and a Figure 1As shown, the in-loop filter 100 includes a deblocking filter 110 as a first stage of a pipeline, a sample adaptive offset (SAO) filter 120 as a second stage of the pipeline, and an adaptive loop filter (ALF) 130 as a third stage of the pipeline. The in-loop filter 100 can process a picture or frame 160 generated by an inter- or intra- prediction module 103. The inter- / intra-prediction module 103 can use various intra- and inter-prediction techniques to reconstruct the picture 160 from raw data (e.g., video pictures) to remove spatial and temporal redundancies in the raw data, respectively. The picture 160 is thus often referred to as a “reconstructed picture.” The inter- / intra-prediction module 103 can process the raw data in a block-based manner. That is, each frame can be divided into a plurality of blocks, i.e., rectangular non-overlapping regions, and the inter- / intra-prediction module 103 can process the frame by sequentially processing the blocks.

[0033] The reconstructed picture 160 can be transmitted from the inter- / intra-prediction module 103 to the in-loop filter 100 via a data bus 109. Specifically, the data bus 109 can be used to transmit the reconstructed picture 160 to the first stage of the in-loop filter 100, i.e., the deblocking filter 110. Similar to the inter- / intra-prediction module 103, the in-loop filter 100 can also process the reconstructed picture 160 in a block-based manner. However, the block partitioning used by the in-loop filter 100 can be the same or different from the block partitioning used by the inter- / intra-prediction module 103. For example, the non-overlapping blocks processed by the in-loop filter 100 can be different in size from the non-overlapping blocks processed by the inter- / intra-prediction module 103. As shown, the deblocking filter 110 can process the reconstructed picture 160 to reduce blocking artifacts in the reconstructed picture 160. The deblocking filter 110 can be configured to perform a deblocking process on the reconstructed picture 160 to reduce blocking artifacts in the reconstructed picture 160. The deblocking filter 110 can be configured to perform a deblocking process on the reconstructed picture 160 to reduce blocking artifacts in the reconstructed picture 160. The deblocking filter 110 can be configured to perform a deblocking process on the reconstructed picture 160 to reduce blocking artifacts in the reconstructed picture 160. Figure 1As shown, the reconstructed image 160 is divided into 16 blocks of the same size, including block 166, block 167 adjacent to block 166, and block 168 adjacent to block 167. The filtering pipeline of the loop filter 100 processes the reconstructed image 160 such that the three adjacent blocks are processed simultaneously by the three stages of the loop filter 100, with each of the three stages processing the three adjacent blocks sequentially over three consecutive pipeline cycles. For example, during a specific pipeline cycle, the deblocking filter 110 may be processing block 166. During the next pipeline cycle (i.e., the cycle immediately following the specific pipeline cycle), the processing result of block 166 can be transmitted from the deblocking filter 110 to the SAO filter 120 via the data bus 119 for processing by the SAO filter 120, while the deblocking filter 110 processes the next block, block 167. During the next pipeline cycle (i.e., two cycles after a specific pipeline cycle), the processing result of block 166 can be passed from SAO filter 120 to ALF 130 via data bus 129 for processing by ALF 130, and the processing result of block 167 can be passed from deblocking filter 110 to SAO filter 120 via data bus 119 for processing by SAO filter 120, while deblocking filter 110 processes the next block, namely block 168. Figure 1 As shown, during this pipeline cycle, filters 110, 120, and 130 process blocks 168, 167, and 166, respectively. From the perspective of a block, a block passes through the various stages of the loop filter 100 sequentially. For example, block 166 is processed by the deblocking filter 110 in a particular pipeline cycle, then by the SAO filter 120 in the next pipeline cycle, and then by the ALF filter 130 in the next pipeline cycle.

[0034] In a block-based pipeline process, each individual filter (i.e., each stage of the pipeline) can perform a respective filtering function using some pixel data of a current block (i.e., pixel data passed immediately from a previous stage) and some pixel data from one or more neighboring blocks adjacent to the current block. For example, when the SAO filter 120 processes the block 167, the SAO filter 120 can refer to the pixel data of the block 167 received from the deblocking filter 110 through the data bus 119. Meanwhile, the SAO filter 120 can also refer to the data of a few neighboring blocks, such as the pixel data of the blocks 162, 163, 164, and 166, which have been processed by the deblocking filter 110 in the pipeline process. Each of the blocks 162, 163, 164, and 166 is considered a neighboring block of the block 167 because it is adjacent to the block 167 (i.e., edge-adjacent or corner-adjacent to the block 167). Specifically, the blocks 163 and 166 are edge-adjacent to the block 167 because each of the blocks 163 and 166 shares a respective block edge with the block 167. On the other hand, the blocks 162 and 164 are corner-adjacent to the block 167 because each of the blocks 162 and 164 has a block corner adjacent to a respective block corner of the block 167. The same is true for the deblocking filter 110 and the ALF 130. That is, each filtering stage of the loop filter 100 can refer to the pixel data of an immediate block passed from a previous stage (i.e., the inter / intra prediction module 103 or the SAO filter 120) and specific pixel data of neighboring blocks processed before the previous stage. The immediate block that the filtering stage is currently processing can be interchangeably referred to as the "current block" of the filtering stage. The picture or frame to which the current block belongs, i.e., the picture 160, is referred to as the "current picture" or "current frame."

[0035] For each filtering stage of the in-loop filter, except for the first stage (i.e., the deblocking filter 110), the pixel data of the immediate neighboring blocks are passed from the preceding filtering stage; for the first stage (i.e., the deblocking filter 110), the pixel data of the immediate neighboring blocks can be passed from the inter / intra prediction module 103. On the other hand, the pixel data of the neighboring blocks required for processing the immediate neighboring blocks will be fetched by the respective filtering stages from the memory 180 as these blocks have already been processed by the respective filtering stages. For example, when the SAO filter 120 is processing the block 167, the pixel data of the block 167 is directly passed from the deblocking filter 110. However, the SAO filter 120 also requires the pixel data of certain neighboring blocks to be fetched from the memory 180 for processing the block 167. For processing the immediate neighboring block 167, the SAO can fetch the pixel data of the certain neighboring blocks 162, 163, 164 and 166 from the memory 180 via the data bus 125, while the pixel data of the blocks 162, 163, 164 and 166 have already been stored in the memory 180 by the deblocking filter 110 via the data bus 115 in the previous pipeline cycle (when the deblocking filter 110 processed the blocks 162, 163, 164 and 166 respectively). Similarly, each of the deblocking filter 110 and the ALF 130 can receive the pixel data of the immediate neighboring blocks from the preceding stage (i.e., from the inter / intra prediction module 103 and the SAO filter 120 respectively). Meanwhile, each of the deblocking filter 110 and the ALF 130 can fetch the pixel data of the neighboring blocks of the respective immediate neighboring blocks from the memory 180 via the data bus 115 and the data bus 135 respectively. The neighboring block pixel data fetched by the deblocking filter 110 has been stored into the memory 180 by the inter / intra prediction module 103 via the data bus 105 as a result of the inter / intra prediction module 103 previously processing the neighboring blocks. The neighboring block pixel data fetched by the ALF 130 has been stored into the memory 180 by the SAO filter 120 via the data bus 125 as a result of the SAO filter 120 previously processing the neighboring blocks.

[0036] After passing through the various stages of the in-loop filter 100, the blocks are stored in the decoded picture buffer (DPB) 190 via the data bus 139. The inter / intra prediction module 103 can access the DPB 190 via the data bus 106 to fetch the filtered pictures stored therein for performing subsequent inter and intra prediction operations. The pictures stored in the DPB 190, as they have been filtered by the in-loop filter 100 to remove certain picture artifacts, will then serve as more ideal reference picture candidates for the inter / intra prediction module 103, thereby increasing the resulting coding efficiency and the resulting subjective quality of the video.

[0037] Not all Figure 1All of the data buses in the data bus 119 and 129 are used to pass or otherwise transfer block data (i.e., pixel data of an immediately adjacent block) from a filtering stage in the filtering pipeline of the loop filter 100 to the next stage directly, and thus have a very high transfer speed. That is, there is little delay in transferring block data from one stage to the next stage within the loop filter 100. On the other hand, there is more latency in each of the data buses 115, 125, and 135. This is because the data buses 115, 125, and 135 are connected to the memory 180, which in physical implementation typically uses a relatively low speed memory, such as dynamic random-access memory (DRAM). In addition, each of the data buses 115, 125, and 135 is used to fetch pixel data of more than one adjacent block (rather than just one block), and accessing data from multiple blocks requires a longer access time. That is, the data access latency through the data buses 115, 125, and 135 is much greater than the data access latency of the data buses 119 and 129. In other words, the latency cost of accessing pixel data from the memory 180 is much higher than the latency cost of passing pixel data directly between filtering stages of the loop filter 100. Therefore, data access schemes that require less pixel data from the memory 180 are prioritized in consideration of processing latency.

[0038] II. Deblocking Filter

[0039] Figure 2 A diagram showing an example design according to an embodiment of the disclosure is shown, in which a deblocking filter scheme 200 is shown. A block that is currently being processed by a filtering stage is referred to as the "immediately block" or "current block" of that filtering stage. The deblocking filter 110 can use the deblocking filter scheme 200 to perform a deblocking operation to process the immediate block 168. As shown, each of the blocks 163, 164, 167, and 168 is a 16-pixel x 16-pixel block, i.e., a rectangular region having sixteen pixels in width (i.e., in the x-direction) and sixteen pixels in height (i.e., in the y-direction). Each pixel of the blocks 163, 164, 167, and 168 is denoted by a pair of coordinates (x, y), where x is the x-coordinate and y is the y-coordinate. The blocks 163, 164, 167, and 168 are arranged in a row, and each block is adjacent to the other blocks in the row. The blocks 163, 164, 167, and 168 are also arranged in a column, and each block is adjacent to the other blocks in the column. The blocks 163, 164, 167, and 168 are also arranged in a 2x2 grid, and each block is adjacent to the other blocks in the grid. The blocks 163, 164, 167, and 168 are also arranged in a 4x4 grid, and each block is adjacent to the other blocks in the grid. The blocks 163, 164, 167, and 168 are also arranged in a 16x16 grid, and each block is adjacent to the other blocks in the grid. Figure 2 Figure 2 ​A block 163 is represented by a small square, which is identified using unique (x, y) coordinates. For example, the first pixel of block 163, i.e., the pixel at the top-left corner of block 163, has coordinates (-16, -16), while the last pixel of block 163, i.e., the pixel at the bottom-right corner of block 163, has coordinates (-1, -1). Similarly, the current block being processed by deblocking filter 110, i.e., block 168, has a total of 256 pixels, starting with the (0, 0) pixel and ending with the (15, 15) pixel. Block 164 also has 256 pixels, and the (x, y) coordinates of the four corner pixels of block 164 are (0, -16), (15, -16), (0, -1), and (15, -1), respectively. Block 167 also has 256 pixels, and the (x, y) coordinates of the four corner pixels of block 167 are (-16, 0), (-1, 0), (-16, 15), and (-1, 15), respectively.

[0040] In this disclosure, the notation {coordinate of top-left pixel (x, y), coordinate of bottom-right pixel (x, y)} is used to refer to a rectangular region containing a plurality of pixels. In terms of this notation, Figure 2 Block 167 can be represented using {(-16, 0), (-1, 15)}, and current block 168 can be represented using {(0, 0), (15, 15)}. Similarly, blocks 163 and 164 can be referred to as {(-16, -16), (-1, -1)} and {(0, -16), (15, -1)}, respectively. A column or a row of pixels can be represented using the same notation. For example, the leftmost column of block 164 can be referred to as {(0, -16), (0, -1)}, while the bottommost row of block 163 can be referred to as {(-16, -1), (-1, -1)}.

[0041] To perform deblocking on the current block, deblocking filter 110 can process the pixel data of the current block along one or more virtual horizontal lines and / or one or more virtual vertical lines of the current block. As shown in FIG. 2A, deblocking filter 110 can perform deblocking on current block 168 along two virtual vertical lines 210 and 218, and two virtual horizontal lines 250 and 258. Lines 210, 218, 250, and 258 are "virtual" in that they do not contain any actual pixels of block 168. Rather, each of lines 210, 218, 250, and 258 is a virtual boundary used to separate two groups of pixels. For example, line 258 separates block 168 into two sub-blocks, one of which contains pixels with y-coordinates of 7 or lower, and the other of which contains pixels with y-coordinates of 8 or higher. Since deblocking is performed along lines 210, 218, 250, and 258, they are also referred to as "deblocking boundaries." Figure 2

[0042] ​Notably, some deblocking boundaries are also block boundaries or edges of the current block, while some deblocking boundaries are inside the current block. For example, deblocking boundaries 210 and 250 are exactly the left and top block boundaries (i.e., edges) of block 168, while deblocking boundaries 218 and 258 are located inside block 168, i.e., inside deblocking boundaries. Inside deblocking boundaries 218 and 258 collectively divide block 168 into four quadrants, each containing sub-blocks 281, 282, 283, or 284. Each of blocks 163, 164, 167, and 168 can be divided into a plurality of non-overlapping sub-blocks. As Figure 2 shown, block 168 is divided into sub-blocks 281, 282, 283, and 284 by deblocking boundaries 218 and 258. Likewise, block 163 is divided into sub-blocks 231, 232, 233, and 234. Block 164 is divided into sub-blocks 241, 242, 243, and 244, while block 167 is divided into sub-blocks 271, 272, 273, and 274.

[0043] Deblocking filter 110 performs a deblocking operation along a vertical deblocking boundary of the current block (e.g., boundary 210 or 218 of block 168), thereby sequentially processing each row of pixels across the vertical deblocking boundary using a horizontal filtering operation. In addition, deblocking filter 110 also performs a deblocking operation along a horizontal deblocking boundary of the current block (e.g., boundary 250 or 258 of block 168), thereby sequentially processing each column of pixels across the horizontal deblocking boundary using a vertical filtering operation. For example, deblocking filter 110 can perform a horizontal filtering operation along deblocking boundary 210 in a row-by-row manner, starting with the row of y = 0, then the row of y = 1, followed by the row of y = 2, and so on, ending with the row of y = 15.

[0044] Figure 3 A diagram showing an example design according to an embodiment of the disclosure is shown, in which horizontal filtering scheme 300 is shown for processing the row of y = 0 when deblocking filter 110 performs a deblocking operation along deblocking boundary 210. As Figure 3 shown, horizontal filtering scheme 300 involves pixels on both sides of deblocking boundary 210. Specifically, horizontal filtering scheme 300 involves the first eight pixels on both sides of deblocking boundary 210, and updates or modifies the first seven pixels on both sides of deblocking boundary 210 accordingly. The two sides of the deblocking boundary can be referred to as the p-side and the q-side, as Figure 3The marked. According to the horizontal filtering scheme 300, the deblocking filter 110 can reference the pixel data {(-8, 0), (-1, 0)} on the p-side and the pixel data {(0, 0), (7, 0)} on the q-side. Based on the referenced pixel data, i.e., the pixel row {(-8, 0), (7, 0)}, the deblocking filter 110 can update or otherwise modify the pixel values of each pixel of {(-7, 0), (6, 0)}. That is, in performing the horizontal filtering operation, the deblocking filter 110 can reference the first eight pixels on each of the p-side and the q-side, and change the values of the seven pixels on each of the p-side and the q-side accordingly. In this disclosure, the terms "update a pixel" and "modify a pixel" are used interchangeably to refer to the act of changing the pixel value of a pixel. As Figure 3 shown, the first seven pixels on the p-side are modified from the values p0, pi, p2, p3, p4, p5, and p6 to p0', pi', p2', p3', p4', p5', and p6', respectively, while the first seven pixels on the q-side are modified from the values q0, qi, q2, q3, q4, q5, and q6 to q0', qi', q2', q3', q4', q5', and q6', respectively. The eighth pixel on each side, i.e., pixels (-8, 0) and (7, 0), are referenced but not modified. The significant step at the deblocking boundary 210, which can be an artifact caused by the block-based prediction performed by the inter / intra prediction module 103, is now smoothed by replacing the pixel values {p6-p0, q0-q6} with {p6'-p0', q0'-q6'} after the horizontal filtering operation on the row {(-8, 0), (7, 0)} is performed.

[0045] The horizontal filtering scheme 300 illustrates a deblocking filter setting of (8, 7). The two parameters of the filter setting represent the number of pixels referenced and the number of pixels updated, respectively. Figure 4 FIG. 3 illustrates a horizontal filtering scheme according to an embodiment of the present disclosure, where the same row is shown as in FIG. 2. Figure 3 FIG. 4 illustrates a horizontal filtering scheme according to an embodiment of the present disclosure, where the same row is shown as in FIG. 2.

[0046] The de-blocking filter 110 can perform the horizontal filtering operation sequentially for each row that crosses the de-blocking boundary 210 using the filtering scheme 300. That is, along the de-blocking boundary 210, the de-blocking filter 110 can perform the horizontal filtering operation by referencing {(-8, 0), (7, 0)} and modifying {(-7, 0), (6, 0)}, then referencing {(-8, 1), (7, 1)} and modifying {(-7, 1), (6, 1)}, then referencing {(-8, 2), (7, 2)} and modifying {(-7, 2), (6, 2)}, and so on until the de-blocking filter 110 processes the last row that crosses the de-blocking boundary 210, i.e., referencing {(-8, 15), (7, 15)} and modifying {(-7, 15), (6, 15)}.

[0047] Similarly, in the case where the horizontal filtering scheme 400 is employed instead of the horizontal filtering scheme 300, the de-blocking filter 110 can perform the horizontal filtering operation sequentially for each row that crosses the de-blocking boundary 210. That is, along the de-blocking boundary 210, the de-blocking filter 110 can perform the horizontal filtering operation by referencing {(-8, 0), (7, 0)} and modifying {(-5, 0), (4, 0)}, then referencing {(-8, 1), (7, 1)} and modifying {(-5, 1), (4, 1)}, then referencing {(-8, 2), (7, 2)} and modifying {(-5, 2), (4, 2)}, and so on until the de-blocking filter 110 processes the last row that crosses the de-blocking boundary 210, i.e., referencing {(-8, 15), (7, 15)} and modifying {(-5, 15), (4, 15)}.

[0048] The de-blocking filter 110 can perform the horizontal filtering operation using a filtering scheme other than the filtering scheme 300 or 400. The de-blocking filter 110 can utilize a horizontal filtering scheme defined by two parameters, a first parameter for defining how many pixels from the vertical de-blocking boundary are referenced on each side of the vertical de-blocking boundary for a row, and a second parameter for defining how many pixels from the vertical de-blocking boundary are modified accordingly on each side of the vertical de-blocking boundary for the row. These two parameters constitute a filter setting for processing the block.

[0049] The de-blocking filter 110 can apply the same or similar filter setting for performing the vertical filtering operation along one or more virtual horizontal lines of the block 168, such as the de-blocking boundaries 250 and 258. Figure 5FIGS. 5 and 6 show example designs according to embodiments of the present disclosure, in which horizontal filter schemes 300 and 400 are shown for performing horizontal filter operations along deblocking boundaries 250 and 258, respectively. Similar to the vertical filter operations, in performing the horizontal filter operations, deblocking filter 110 can refer to a first number of pixels on each side of deblocking boundary 250 or 258 and modify a second number of pixels on each side of deblocking boundary 250 or 258 accordingly, while the first number and the second number are defined by two parameters in the filter settings. For example, when deblocking filter 110 performs a horizontal filter operation along deblocking boundary 250 for x = 0 row using filter settings (8, 6) according to filter scheme 300, deblocking filter 110 can refer to eight pixels in {(0, -8), (0, -1)} on the p-side and eight pixels in {(0, 0), (0, 7)} on the q-side, and then modify twelve pixels in {(0, -6), (0, 5)} accordingly. Likewise, deblocking filter 110 can perform a horizontal filter operation along deblocking boundary 258 for x = 0 row using filter settings (8, 5) according to filter scheme 400, in which deblocking filter 110 can refer to eight pixels in {(0, 0), (0, 7)} on the p-side and eight pixels in {(0, 8), (0, 15)} on the q-side, and then modify ten pixels in {(0, 3), (0, 12)} accordingly. Deblocking filter 110 can perform the horizontal filter operations along deblocking boundaries 250 and 258 column by column for each column that crosses deblocking boundaries 250 and 258, respectively. That is, along deblocking boundary 250, deblocking filter 110 can perform the horizontal filter operation by referring to {(0, -8), (0, 7)} and modifying {(0, -6), (0, 5)}, then referring to {(1, -8), (1, 7)} and modifying {(1, -6), (1, 5)}, then referring to {(2, -8), (2, 7)} and modifying {(2, -6), (2, 5)} and so on until deblocking filter 110 processes the last column that crosses deblocking boundary 250, i.e., referring to {(15, -8), (15, 7)} and modifying {(15, -6), (15, 5)}. Likewise, along deblocking boundary 258, deblocking filter 110 can perform the horizontal filter operation by referring to {(0, 0), (0, 15)} and modifying {(0, 3), (0, 12)}, then referring to {(1, 0), (1, 15)} and modifying {(1, 3), (1, 12)}, then referring to {(2, 0), (2, 15)} and modifying {(2, 3), (2, 12)} and so on until deblocking filter 110 processes the last column that crosses deblocking boundary 250, i.e., referring to {(15, 0), (15, 15)} and modifying {(15, 3), (15, 12)}.

[0050] Notably, in performing the horizontal filtering operation along deblocking boundary 210, deblocking filter 110 can operate on sub-blocks of the current block as well as sub-blocks of one or more neighboring blocks. For example, in performing the horizontal filtering operation along deblocking boundary 210 of current block 168 using horizontal filtering scheme 300, deblocking filter 110 can reference pixel data in {(-8, 0), (-1, 15)} and modify pixel data in {(-7, 0), (-1, 15)}, both of which are in neighboring block 167. At the same time, deblocking filter 110 can reference pixel data in {(0, 0), (7, 15)} and modify pixel data in {(0, 0), (6, 15)}, both of which are in current block 168.

[0051] Figure 6FIG. 6 shows an example design according to an embodiment of the disclosure, in which the deblocking filter 110 performs only horizontal filtering operations and no vertical filtering operations according to a deblocking filtering scheme 600. According to the deblocking filtering scheme 600, the deblocking filter 110 first performs horizontal filtering operations along the deblocking boundaries 610 and 618 of the block 167, and then along the deblocking boundaries 210 and 218 of the block 168. With this processing, the pixel data in the sub-blocks 272 and 274 is modified twice, once as a q-side sub-block during the horizontal filtering performed along the deblocking boundary 618, and then again as a p-side sub-block during the horizontal filtering performed along the deblocking boundary 210. That is, when the deblocking filter 110 finishes processing the block 167, the sub-blocks 272 and 274 are only "partially filtered." The sub-blocks 272 and 274 do not become "fully filtered" until the deblocking filter 110 finishes the horizontal filtering along the deblocking boundary 210. Similarly, the pixel data in the sub-blocks 281 and 283 is modified twice, once as a q-side sub-block during the horizontal filtering performed along the deblocking boundary 210, and then again as a p-side sub-block during the horizontal filtering performed along the deblocking boundary 218. Thus, when the deblocking filter 110 finishes processing the block 168 along the deblocking boundaries 210 and 218, the sub-blocks 281 and 283 are fully filtered, but the sub-blocks 282 and 284 will remain partially filtered until the deblocking filter 110 finishes processing the next block (i.e., the block immediately to the right of the block 168). In general, after the deblocking filter 110 finishes processing a block, a filtered block is produced as a result, which includes one or more fully filtered sub-blocks and one or more partially filtered sub-blocks. For example, after the deblocking filter 110 finishes processing the block 167, a filtered block is produced that consists of two fully filtered sub-blocks 271 and 273, and two partially filtered sub-blocks 272 and 274. Similarly, after the deblocking filter 110 finishes processing the block 168, a filtered block is produced that consists of two fully filtered sub-blocks 281 and 283, and two partially filtered sub-blocks 282 and 284. Note that the sub-blocks 272 and 274 are fully filtered when the deblocking filter 110 finishes processing the block 168. The deblocking filter 110 can output the fully filtered sub-blocks 272 and 274 along with the fully filtered sub-blocks 281 and 283 as an output block to the next filtering stage of the in-loop filtering pipeline 100, such as the SAO filter 120. The output block can serve as the next current block to be processed by the SAO filter 120 in the next pipeline cycle.

[0052] Although the deblocking filtering scheme 600 illustrates performing horizontal filtering only along the vertical deblocking boundaries (i.e., no vertical filtering), a deblocking filtering scheme in which vertical filtering operations are performed only along the horizontal deblocking boundaries (i.e., no horizontal filtering) can be achieved by swapping the horizontal and vertical filtering operations in the deblocking filtering scheme 600. Figure 6The rows and columns in the data can be easily derived from the deblocking filter scheme 600.

[0053] A similar scenario exists when the deblocking filter 110 performs both horizontal and vertical filtering operations. See also... Figure 2 The deblocking filter 110 performs horizontal and vertical filtering operations on each of blocks 163, 164, 167, and 168. The horizontal filtering operation is performed along two vertical deblocking boundaries, such as deblocking boundaries 210 and 218 of block 168. The vertical filtering operation is also performed along two horizontal deblocking boundaries, such as deblocking boundaries 250 and 258 of block 168. When the deblocking filter 110 finishes processing block 164, only sub-block 241 is fully filtered, while sub-blocks 242, 243, and 244 are only partially filtered because the pixel data of each of these sub-blocks may be modified again when the deblocking filter 110 processes some of the adjacent blocks of block 164. For example, when the deblocking filter 110 processes block 168 and performs a vertical filtering operation along deblocking boundary 250, the pixel values ​​of sub-blocks 243 and 244 may be modified again. In fact, sub-block 243 is not fully filtered until the deblocking filter 110 finishes processing block 168. When deblocking filter 110 completes processing of block 164, the resulting filtered block, comprising a fully filtered sub-block 241 and three partially filtered sub-blocks 242-244, is stored in memory 180 via data bus 115. This allows deblocking filter 110 to acquire at least partially filtered sub-blocks 243 and 244 when processing block 168, for example, when performing a vertical filtering operation along deblocking boundary 250. Similarly, when deblocking filter 110 completes processing of block 167, the resulting filtered block, comprising a fully filtered sub-block 271 and three partially filtered sub-blocks 272-274, is stored in memory 180 via data bus 115. This allows deblocking filter 110 to acquire at least partially filtered sub-blocks 272 and 274 when processing block 168, for example, when performing a horizontal filtering operation along deblocking boundary 210. The deblocking filter 110 completes the processing of block 168 and stores the resulting filtered block, including a fully filtered sub-block 281 and three partially filtered sub-blocks 282-284, in memory 180 via data bus 115. This allows the deblocking filter 110 to obtain any one of the partially filtered sub-blocks 282-284 when processing adjacent blocks of block 168, such as the block immediately to the right of block 168 or the block immediately below block 168.

[0054] The filter block can include sub-blocks that overlap the current block and sub-blocks that overlap one or more neighboring blocks of the current block. For example, the filter block is generated after the deblocking filter 110 completes processing of the current block 168 using the deblocking filter scheme 200. The filter block can include one or more fully filtered sub-blocks, i.e., sub-blocks 243, 272, and 281. These sub-blocks are fully filtered because their pixel values will not be modified again by the deblocking filter 110 to process any future block of the picture. Among the fully filtered sub-blocks, the sub-block 281 overlaps the current block 168, while the sub-blocks 243 and 272 overlap the neighboring blocks 164 and 167, respectively. The filter block can also include one or more partially filtered sub-blocks, i.e., sub-blocks 244, 274, 282, 283, and 284. These sub-blocks are partially filtered because their pixel values can be modified again by the deblocking filter 110 to process a future block of the picture. For example, some pixel values of the sub-block 274 can be modified again when the deblocking filter 110 processes a block immediately below the block 167. As another example, some pixel values of the sub-block 284 can be modified again when the deblocking filter 110 processes a block immediately to the right or below the block 168. Among the partially filtered sub-blocks, the sub-blocks 282-284 overlap the current block 168, while the sub-blocks 244 and 274 overlap the neighboring blocks 164 and 167, respectively.

[0055] Further, when the deblocking filter 110 completes processing of a block, the output block can be passed to the next filtering stage, where the output block includes one or more fully filtered sub-blocks. For example, when the deblocking filter 110 completes processing of the block 168, the deblocking filter 110 can output an output block that includes the fully filtered sub-blocks 234, 243, 272, and 281. The output block can be transferred from the deblocking filter 110 to the SAO filter 120 through the data bus 119, and the output block is the next current block to be processed by the SAO filter 120 in the next pipeline stage.

[0056] With Figures 2-6 The example embodiments presented together with FIG. 2 are based on the assumption that the deblocking filter 110 is an eight-tap filter, i.e., eight pixels on each side of the reference p-side and q-side are used when performing the horizontal filtering operation and / or the vertical filtering operation along the deblocking boundary. The same approach can be applied to deblocking filters of any number of taps. Assume that the deblocking filter 110 is an N ref tap filter, where N refIt is a positive integer. According to the design of this disclosure, the deblocking filter 110 can receive several blocks of a rectangular region {(-N, -N), (N-1, N-1)} surrounding the image. The rectangular region {(-N, -N), (N-1, N-1)} may include the current block {(0, 0), (N-1, N-1)}, and three adjacent blocks of the current block, namely {(-N, -N), (-1, -1)}, {(0, -N), (N-1, -1)}, and {(-N, 0), (-1, N-1)}. The deblocking filter 110 can perform horizontal and vertical filtering operations on the current block to generate a filtered block, which includes a fully filtered sub-block {(0, 0), (N-1 ... ref -1,N ref -1)} and the three partially filtered sub-blocks {(0,N ref ), (N ref -1,N-1)},{(N ref ,0), (N-1,N ref -1)} and {(N ref N ref The deblocking filter 110 can deblock the output block {(-N)}. ref -N ref ), (NN ref -1,NN ref -1)} outputs to SAO filter 120, which consists of four fully filtered sub-blocks. In Figures 2-5 In the example embodiment shown, N = 16, N ref =8.

[0057] III. SAO Filter

[0058] As described above, the output block passed from the deblocking filter 110 to the SAO filter 120 becomes the current block processed by the SAO filter 120 in the next pipeline cycle. Figure 7 A diagram illustrating an example design according to an embodiment of the present disclosure shows a SAO filtering scheme 700 for processing the current block 710 via a SAO filter 120. The current block 710 is the result output block from a deblocking filter 110, comprising fully filtered sub-blocks (from the perspective of the deblocking filter 110) 234, 243, 272, and 281. The current block 710 has been passed from the deblocking filter 110 via a data bus 119. The SAO filter 120 performs the SAO filtering operation based on a filter setting with a parameter specifying how many pixels from the pixels in the current block are referenced in each direction, updating or otherwise modifying the pixel value of the current pixel based on this parameter. That is, the SAO filtering operation is a pixel-by-pixel operation. Figure 7As shown, the SAO filtering scheme 700 uses a 1-pixel SAO filter setting. Taking pixel 760 of the current block 710 as an example, in some embodiments, the SAO filter 120 can reference one adjacent pixel of pixel 760 in each of the left, right, up, and down directions, and then modify pixel 760 accordingly. That is, the SAO filter 120 can reference pixels 761, 762, 763, and 764, and modify the value of pixel 760 based on these pixels. In some embodiments, the SAO filter 120 can further reference one adjacent pixel of pixel 760 in each of the top-left, top-right, bottom-left, and bottom-right directions. That is, the SAO filter 120 can further reference pixels 765, 766, 767, and 768 during the SAO filtering operation. Based on the pixel values ​​of pixels 761-768, the SAO filter 120 can modify the value of pixel 760 accordingly. SAO filtering can be used to reduce ringing artifacts caused by block-based inter-frame / intra-frame prediction performed by inter-frame / intra-frame prediction module 103. Therefore, SAO filter 120 can also be referred to as a "de-ringing filter".

[0059] Because the loop filter 100 is designed to operate in a pipelined manner, each stage of the loop filter 100 preferably outputs an output block of the same size. The pixel data received directly from the deblocking filter 110 is insufficient for the SAO filter 120 to generate an output block of the same size as the output block of the deblocking filter 110, i.e., the current block 710. To generate an output block of the same size as the current block 710, the SAO filter 120 needs to receive additional pixel data from one or more adjacent blocks of block 168. (Refer to...) Figure 7 To generate an output block 720 of the same size as the current block 710, the SAO filter 120 needs to receive pixel data represented by the input block 730, which includes the current block 710 and a region consisting of {(-10, -10), (7, -9)} and {(-10, -8), (-9, 7)} (i.e., an L-shaped region within the input block 730 but outside the current block 710). When the current block 710 is directly passed from the deblocking filter 110 to the SAO filter 120 via the data bus 119, the SAO filter 120 needs to retrieve the pixel data of the L-shaped region from the memory 180 via the data bus 125. As mentioned above, retrieving data from the memory 180 is a slow process. That is, the SAO filter 120 needs to retrieve the pixel data of the L-shaped region from the memory 180, resulting in a delay. The more pixel data the SAO filter 120 needs to retrieve from the memory 180 instead of receiving from the deblocking filter 110, the worse the delay becomes.

[0060] For example, a 2-pixel SAO filter setting may exacerbate the latency introduced by fetching pixel data from memory 180. Figure 8 A diagram illustrating an example design according to an embodiment of the present disclosure is shown, illustrating a SAO filtering scheme 800 for processing the current block 710 via a SAO filter 120. (See diagram for example.) Figure 8 As shown, the SAO filtering scheme 800 uses a two-pixel SAO filter setting. That is, the SAO filter 120 modifies the pixel based on the pixel values ​​of the two pixels adjacent to the pixel in each direction. Take pixel 760 of the current block 710 as an example. Figure 8 As shown, the 2-pixel SAO filter setup requires SAO filter 120 to reference two adjacent pixels in each direction relative to pixel 760, which are collectively represented as a 25-pixel square matrix, with pixel 760 located in the center of the matrix and 24 pixels surrounding pixel 760. The first layer of the matrix includes the 8 pixels immediately adjacent to pixel 760, namely pixels 761-768. The second layer of the matrix, the next pixel layer surrounding the first layer, includes an additional 16 pixels, namely pixels 861–876.

[0061] Similar to SAO filtering scheme 700, in order to generate an output block of the same size as the current block 710 using SAO filtering scheme 800, SAO filter 120 needs to receive additional pixel data from one or more neighboring blocks of block 168, in addition to the current block 710. For example... Figure 8 As shown, in order to generate an output block 820 of the same size as the current block 710, the SAO filter 120 needs to receive pixel data represented by the input block 830, which includes the current block 710 and a region consisting of {(-12, -12), (7, -9)} and {(-12, -8), (-9, 7)} (i.e., an L-shaped region within the input block 830 but outside the current block 710). Although the current block 710 is directly passed from the deblocking filter 110 to the SAO filter 120 via the data bus 119, the SAO filter 120 needs to retrieve the pixel data of the L-shaped region from the memory 180 via the data bus 125. Figure 8 The L-shaped region is larger than Figure 7 The L-shaped region. It is expected that the SAO filtering scheme 800 will introduce more delay than the SAO filtering scheme 700.

[0062] Notably, the pixel data of the L-shaped region, as described above for each SAO filtering scheme 700 and 800, is generated by the previously performed deblocking process of the deblocking filter 110 and subsequently saved in the memory 180 via the bus 115. The SAO filter 120 then retrieves the pixel data from the memory 180 via the bus 125 while the SAO filter 120 processes the current block. However, in some alternative embodiments, the pixel data of the L-shaped region retrieved from the memory 180 can already be processed by the SAO filter 120, as described below.

[0063] In some embodiments of the SAO filtering scheme 700, in addition to reading or retrieving the pixel data from the memory 180 to perform the SAO filtering operation, the SAO filter 120 can also write or store at least a portion of the generated pixel data of the output block (e.g., the output block 720 or 820) to the memory 180 for future reference, i.e., the pixel data can be retrieved by the SAO filter 120 when the SAO filter 120 processes other blocks in the future. In the SAO filtering scheme 700, after generating the output block 720, the SAO filter 120 can store an L-shaped region consisting of the rightmost column and the bottom row of the output block 720 and the rightmost column and the bottom row of the current block 710 into the memory 180 via the data bus 125. When processing future blocks, the SAO filter 120 can access the memory 180 via the data bus 125 to retrieve specific pixel data of the L-shaped region. For example, when processing a block that is immediately to the right of the block 710, the SAO filter 120 can retrieve the rightmost column of the output block 720 and the rightmost column of the current block 710. When processing a block that is immediately below the block 710, the SAO filter 120 can retrieve the bottom row of the output block 720 and the bottom row of the current block 710.

[0064] Similarly, in some embodiments of the SAO filtering scheme 800, after generating the output block 820, the SAO filter 120 can store an L-shaped region consisting of the two rightmost columns and two bottommost rows of the output block 820 and the two rightmost columns and two bottommost rows of the current block 710 into the memory 180 through the data bus 125. That is, the L-shaped region is represented by the combination of {(4, -12), (7, 7)} and {(-12, 4), (7, 7)}. The SAO filter 120 can access the memory 180 through the data bus 125 to obtain specific pixel data of the L-shaped region when processing future blocks. For example, the SAO filter 120 can obtain {(4, -12), (7, 7)} when processing the block immediately to the right of the block 710. The SAO filter 120 can obtain {(-12, 4), (7, 7)} when processing the block immediately below the block 710. The L-shaped region has two segments, the first segment being {(4, -12), (7, 7)} and the second segment being {(-12, 4), (7, 7)}. Each of these two segments has a width that is twice the SAO filter setting employed. For the SAO filtering scheme 800, the SAO filter setting = 2 (i.e., two pixels), so each of the two segments of the L-shaped region has a width of 4 pixels. For the SAO filtering scheme 700, the SAO filter setting = 1 (i.e., one pixel), so each of the two segments of the L-shaped region has a width of 2 pixels.

[0065] Further, the SAO filter 120 can output the output block 720 or 820 to the next filtering stage of the loop filtering pipeline 100, e.g., the ALF 130. The ALF 130 can use the output block 720 or 820 as the next current block to be processed in the next pipeline cycle.

[0066] With Figure 7 and Figure 8 the exemplary embodiments presented are based on the SAO filter 120 being a one-tap filter (i.e., filter setting = 1 pixel) and a two-tap filter (i.e., filter setting = 2 pixels), respectively. A one-tap SAO filter modifies a particular pixel by referring to the most adjacent one pixel in all directions, i.e., the first layer of 8 pixels around the particular pixel. A two-tap SAO filter modifies a particular pixel by referring to the most adjacent two pixels in all directions, i.e., the first layer of 8 pixels around the particular pixel plus a second layer of 16 pixels around the first layer. The same approach can be applied to an SAO filter of any number of taps. Assume the SAO filter 120 is an N f2 tap filter (i.e., filter setting = N f2 pixels), where N f2is a positive integer. According to the design of the present disclosure, the SAO filter 120 can receive the output block of the deblocking filter 110, i.e., the block {(-N ref , -N ref ), (N-N ref -1, N-N ref -1)} as the current block 710 of the SAO filter 120. The SAO filter 120 can further access the memory 180 to obtain the pixel data of the L-shaped region of the input block (e.g., the input block 730 or 830), which is {(-N ref -2N f2 , -N ref -2N f2 ), (N-N re -1 f , -N ref -1)} and {(-N ref -2N f2 , -N ref ), (-N ref -1, N-N ref -1)}. The SAO filter 120 can perform the SAO filtering operation, e.g., the SAO filtering scheme 700 or 800, and accordingly output the filtered block (e.g., the output block 720 or 820), i.e., {(-N ref -N f2 , -N ref -N f2 ), (N-N ref -1-N f2 , N-N ref -1-N f2 )}. In some embodiments, the SAO filter 120 can store the pixel data represented by the L-shaped region of the output block in the memory 180 for the SAO filter 120 to process future blocks, wherein the L-shaped region is represented by {(N-N ref -2N f2 , -N ref -2N f2 ), (N-N ref -1, N-N ref -1)} and {(-N ref -2N f2 , N-N ref -2N f2 ), (N-N ref -1-2N f2 , N-N ref -1)}.

[0067] Four, low latency data scheme

[0068] As described above, each of the SAO filter schemes 700 and 800 requires access to (e.g., reading pixel data from and / or writing pixel data to) the memory 180. It is also described above that accessing the memory 180 is a slow process, which increases the processing delay of the SAO filter 120. It is preferable to reduce the memory access in order to reduce the delay. For example, if the SAO filter 120 needs to fetch less pixel data from the memory 180, the delay can be reduced. In addition, if the SAO filter 120 needs to store less pixel data to the memory 180, the delay can also be reduced.

[0069] Figure 9 An example design is shown according to an embodiment of the disclosure, in which a data scheme 900 is presented for the combination of the deblocking filter 110 and the SAO filter 120. In the data scheme 900, the deblocking filter 110 employs the filter setting (8, 7), and the SAO filter 120 employs the filter setting of 1 pixel. The deblocking filter 110 has already performed horizontal filtering and vertical filtering on the blocks 163, 164, 167 and 168. For example, the deblocking filter 110 has performed horizontal filtering on the block 168 along the deblocking boundaries 210 and 218 using the deblocking filter scheme 300. In addition, the deblocking filter 110 has also performed vertical filtering on the block 168 along the deblocking boundaries 250 and 258 using the filter setting (8, 7). According to the data scheme 900, after the deblocking filter 110 finishes processing the block 168, the deblocking filter 110 can output an output block 910 to the SAO filter 120, which includes not only the fully filtered sub-blocks 234, 243, 272 and 281, but also an L-shaped region 911. That is, the output block 910 according to the data scheme 900 is the L-shaped region 911 that is attached to the output block 710. As described elsewhere below, including the L-shaped region 911 in the output block 910 as the next current block for the SAO filter 120 helps to reduce the amount of pixel data that the SAO filter 120 needs to fetch from the memory 180, thereby reducing the delay introduced into the SAO filter 120.

[0070] The L-shaped region 911 is composed of a respective portion of each of the partially filtered sub-blocks 244, 274, 282, 283, and 284. Specifically, the L-shaped region 911 is composed of a portion of the sub-block 274 (i.e., {(-8, 8), (-1, 8)}), a portion of the sub-block 283 (i.e., {(0, 8), (7, 8)}), a portion of the sub-block 284 (i.e., pixel (8, 8)), a portion of the sub-block 282 (i.e., {(8, 0), (8, 7)}), and a portion of the sub-block 244 (i.e., {(8, -8), (8, -1)}). Notably, each portion is adjacent to one of the fully filtered sub-blocks 234, 243, 272, and 281. Specifically, the portion of the partially filtered sub-block 274 (i.e., {(-8, 8), (-1, 8)}) is adjacent to an edge of the fully filtered sub-block 272; the portion of the partially filtered sub-block 283 (i.e., {(0, 8), (7, 8)}) is adjacent to an edge of the fully filtered sub-block 281; the portion of the partially filtered sub-block 284 (i.e., pixel (8, 8)) is adjacent to a corner of the fully filtered sub-block 281; the portion of the partially filtered sub-block 282 (i.e., {(8, 0), (8, 7)}) is adjacent to an edge of the fully filtered sub-block 281; and the portion of the partially filtered sub-block 244 (i.e., {(8, -8), (8, -1)}) is adjacent to an edge of the fully filtered sub-block 243.

[0071] Notably, the pixels of the L-shaped region 911 are not expected to be modified or otherwise updated by the deblocking filter 110 processing future blocks because the deblocking filter setting = (8, 7) is employed by the deblocking filter 110. Thus, even though the pixels of the L-shaped region 911 belong to partially filtered sub-blocks, the pixels of the L-shaped region 911 are ready to be used by the SAO filter 120. For example, even though the pixels {(0, 8), (7, 8)} of the L-shaped region 911 belong to the partially filtered sub-block 283, these pixels are not further modified by the vertical deblocking filtering operation along the deblocking boundary 950 because each of the pixels {(0, 8), (7, 8)} is the 8th pixel on the p-side of the deblocking boundary 950, which are only referenced and not modified by the vertical deblocking filtering operation along the deblocking boundary 950.

[0072] The L-shaped region 911 includes two segments, a horizontal segment with pixels having {(-8, 8), (8, 8)} and a vertical segment with pixels having {(8, -8), (8, 8)}. The difference between the two parameters of the deblocking filter setting (i.e., 8 - 7 = 1 pixel) is equal to the width of each of the horizontal and vertical segments of the L-shaped region 911. If two different filter settings are used by the deblocking filter 110 to perform the vertical and horizontal filtering operations, respectively, the width of the horizontal segment of the L-shaped region 911 can be equal to or less than the difference between the two parameters of the filter setting used for the vertical filtering operation. And the width of the vertical segment of the L-shaped region 911 can be equal to or less than the difference between the two parameters of the filter setting used for the horizontal filtering operation. For example, the deblocking filter 110 can employ the horizontal filtering scheme 400 (which has a deblocking filter setting of (8, 5)) and the vertical filtering scheme 500 (which has a deblocking filter setting of (8, 6)) for processing blocks 163, 164, 167, and 168. As such, the horizontal segment of the L-shaped region 911 can have a width of at most 8 - 6 = 2 pixels, and the vertical segment of the L-shaped region 911 can have a width of at most 8 - 5 = 3 pixels. In particular, the horizontal segment of the L-shaped region 911 can be as wide as the region of {(-8, 8), (10, 9)}, and the vertical segment of the L-shaped region 911 can be as wide as the region of {(-8, 8), (10, 9)}.

[0073] The inclusion of the L-shaped region 911 in the output block 910 of the deblocking filter 110 reduces the amount of pixel data that the SAO filter 120 needs to fetch from the memory 180. The savings in the number of pixels retrieved from the memory 180 is equal to the number of pixels included in the L-shaped region 911. As Figure 9As shown, the inclusion of L-shaped region 911 in output block 910 (output by deblocking filter 110) enables SAO filter 120 to use pixel data represented by input block 930 (input to SAO filter 120) to generate output block 920 (output by SAO filter 120). Output block 920 completely overlaps the combination of fully filtered sub-blocks 234, 243, 272, and 281, and thus has the same size as output block 720 or 820, although the position of output block 920 is offset from the position of output block 720 or 820. Input block 930 consists of output block 910 and L-shaped region 938 consisting of {(-9, -9), (-9, 8)} and {(-9, -9), (8, -9)}. From the perspective of SAO filter 120, SAO filter 120 receives input block 930 by receiving output block 910 (including output block 710 and L-shaped region 911) directly from deblocking filter 110 via data bus 119, and then receiving L-shaped region 938 from memory 180 via data bus 125. In comparison to SAO filtering scheme 700, which required SAO filter 120 to fetch pixel data for L-shaped regions {(-10, -10), (7, -9)} and {(-10, -8), (-9, 7)} from memory 180, data scheme 900 only requires SAO filter 120 to fetch pixel data for L-shaped region 938 from memory 180, which substantially reduces the amount of pixel data fetched by 50%. The 50% reduction in the amount of pixel data fetched translates to a substantially 50% reduction in the latency introduced in SAO filter 120, thereby greatly improving the processing efficiency of in-loop filter 100.

[0074] In some embodiments, SAO filter 120 can store the pixel data for L-shaped region 911 received from deblocking filter 110 into memory 180 via bus 125, such that the pixel data for L-shaped region 911 is available to be fetched again by SAO filter 120 to process future blocks. That is, in addition to receiving the pixel data for L-shaped region 911 from deblocking filter 110 and performing the SAO filtering operation using the pixel data for L-shaped region 911, SAO filter 120 can further store the pixel data for L-shaped region 911 to memory 180. However, in some alternative embodiments, SAO filter 120 can update the pixel data for L-shaped region 911 before storing the pixel data to memory 180. That is, the pixel data for L-shaped region 911 can be updated as a result of SAO filter 120 performing the SAO filtering operation for the current block, and then stored to memory 180 for future use.

[0075] Figure 10An example design is shown according to an embodiment of the disclosure, in which a data scheme 1000 is presented for the combination of deblocking filter 110 and SAO filter 120. In data scheme 1000, deblocking filter 110 employs filter setting (8, 6), and SAO filter 120 employs a filter setting of 2 pixels. Deblocking filter 110 has already performed horizontal filtering and vertical filtering on blocks 163, 164, 167, and 168. For example, deblocking filter 110 has performed horizontal filtering on block 168 along deblocking boundaries 210 and 218 using filter setting (8, 6). In addition, deblocking filter 110 has also performed vertical filtering on block 168 along deblocking boundaries 250 and 258 using filter setting (8, 6). According to data scheme 1000, after deblocking filter 110 finishes processing block 168, deblocking filter 110 can output output block 1010 to SAO filter 120, which includes not only fully filtered sub-blocks 234, 243, 272, and 281, but also L-shaped region 1011. That is, output block 1010 is L-shaped region 1011 appended to output block 710 according to data scheme 1000. Similar to data scheme 900, including L-shaped region 1011 in output block 1010 serves as the next current block for SAO filter 120, which helps to reduce the amount of pixel data that SAO filter 120 needs to fetch from memory 180, thereby reducing the latency introduced in SAO filter 120.

[0076] Similar to data scheme 900, the amount of pixel saving from fetching from memory 180 in data scheme 1000 is equal to the amount of pixels included in L-shaped region 1011. As Figure 10As shown, the inclusion of L-shaped region 1011 in output block 1010 (output by deblocking filter 110) enables SAO filter 120 to use pixel data represented by input block 1030 (input to SAO filter 120) to generate output block 920 (output by SAO filter 120). Output block 920 has the same size as output block 720 or 820, although the position of output block 920 is offset from the position of output block 720 or 820. Input block 1030 consists of output block 1010 and L-shaped region 1038 consisting of {(-10, -10), (-9, 9)} and {(-10, -10), (9, -9)}. From the perspective of SAO filter 120, input block 1030 is received by SAO filter 120 by receiving output block 1010 (including output block 710 and L-shaped region 1011) directly from deblocking filter 110 via data bus 119 and receiving L-shaped region 1038 from memory 180 via data bus 125. In comparison to SAO filtering scheme 800, in which SAO filter 120 needs to fetch pixel data of L-shaped regions {(-12, -12), (7, -9)} and {(-12, -8), (-9, 7)} from memory 180, data scheme 1000 only requires SAO filter 120 to fetch pixel data of L-shaped region 1038 from memory 180, which is more than 50% less pixel data fetched. The reduction in the amount of pixel data fetched translates to a substantial reduction in the latency introduced in SAO filter 120, thereby greatly improving the processing efficiency of in-loop filter 100. Notably, in data schemes 900 and 1000, even though the amount of extra pixel data output by deblocking filter 110 (e.g., L-shaped region 911 or 1011) is approximately the same as the reduction in the amount of pixel data that SAO filter 120 needs to fetch from memory 180, the overall latency of in-loop filter 100 is still improved by employing data schemes 900 or 1000. This is because the speed of transferring data from deblocking filter 110 to SAO filter 120 via data bus 119 is much faster than the speed of transferring data between SAO filter 120 and memory 180 via data bus 125.

[0077] Since the data schemes 900 and 1000 involve the deblocking filter 110 and the SAO filter 120, it should be noted that the size of the L-shaped region 911 and 1011 is determined not only by the filter setting employed by the deblocking filter 110 but also by the filter setting employed by the SAO filter 120. Specifically, the size of the L-shaped region 911 and 1011 is dictated by the more stringent or more restrictive requirement between the filter setting employed by the deblocking filter 110 and the filter setting employed by the SAO filter 120. For example, in the case where the deblocking filter 110 employs filter setting (8, 6) for horizontal and vertical filtering and the SAO filter 120 employs a filter setting of 1 pixel, the output block sent from the deblocking filter 110 to the SAO filter 120 via the data bus 119 will be the block 910 instead of the block 1010. This is because the block 910 already contains all the pixel data that the SAO filter 120 needs to obtain from the deblocking filter 110 for generating the output block 920, and the extra data contained in the block 1010 compared to the block 910 will not be used by the SAO filter 120 anyway. For another example, in the case where the deblocking filter 110 employs filter setting (8, 7) for horizontal and vertical filtering and the SAO filter 120 employs a filter setting of 2 pixels, the output block sent from the deblocking filter 110 to the SAO filter 120 via the data bus 119 will also be the block 910 instead of the block 1010. This is because even though the SAO filter 120 needs all the pixel data in the L-shaped region 1011 to generate the output block 920, the deblocking filter 110 cannot provide at least a portion of the pixel data in the L-shaped region 1011 to the SAO filter 120. Specifically, the deblocking filter 110 cannot provide the pixel data for the {(-8, 9), (9, 9)} or {(9, -8), (9, 9)} regions because the pixel values in these regions will be further modified by the deblocking filter 110 that will process the future block, e.g., performing a vertical filtering operation along the deblocking boundary 950. That is, the width of each of the horizontal and vertical segments of the L-shaped region 911 or 1011 can be expressed as:

[0078] W L = min(N f2 , N ref - N rev ) (1)

[0079] where N ref is the first parameter of the filter setting of the deblocking filter 110 that defines the number of pixels referenced on each of the p-side and the q-side of the deblocking boundary, where N rev is the second parameter of the filter setting of the deblocking filter 110 that defines the number of pixels modified or updated on each of the p-side and the q-side of the deblocking boundary, and where N f2is a parameter of the filter setting of the SAO filter 120, which defines the number of pixels from which a corresponding pixel is referenced in each direction for modifying the pixel value of the corresponding pixel.

[0080] As described elsewhere herein above, the inclusion of the L-shaped regions 911 and 1011 in the output blocks 910 and 1010, respectively, reduces the amount of pixel data that the SAO filter 120 needs to fetch from the memory 180. The pixel data fetched from the memory 180 translates not only into the latency improvement as described above, but also into a hardware cost saving, which is related to the scan sequence in which the loop filter 100 is used to process the various coding tree blocks of the picture, as described below. The SAO filter 120 uses a line buffer to store the pixel data fetched from the memory 180 until the pixel data is no longer needed. Thus, the reduction of pixel data fetched from the memory 180 also translates into a smaller line buffer size needed to temporarily store the pixel data.

[0081] In some embodiments, the SAO filter 120 can store the pixel data of the L-shaped region 1011 received from the deblocking filter 110 into the memory 180 via the bus 125, such that the pixel data of the L-shaped region 1011 is available to be fetched again by the SAO filter 120 to process future blocks. That is, in addition to receiving the pixel data of the L-shaped region 1011 from the deblocking filter 110 and performing the SAO filtering operation using the pixel data of the L-shaped region 1011, the SAO filter 120 can also store the pixel data of the L-shaped region 1011 into the memory 180. However, in some alternative embodiments, the SAO filter 120 can update the pixel data of the L-shaped region 1011 before storing the pixel data into the memory 180. That is, the pixel data of the L-shaped region 1011 is updated due to the SAO filter 120 performing the SAO filtering operation on the current block, and then stored into the memory 180 for future use.

[0082] Figure 11 An example design is shown according to an embodiment of the disclosure, in which two scan sequences 1110 and 1120 are shown. The picture 1150 is a reconstructed picture generated by the inter / intra prediction module 103 using block-based intra and / or inter prediction techniques, and the picture 1150 is currently being processed by the loop filter 100. The picture 1150 can be a luma picture containing luma information, or a chroma picture containing chroma information. As described elsewhere herein above, the loop filter 100 is used to process the picture 1150 in a scan sequence, in which the loop filter 100 processes the various coding tree blocks of the picture 1150 in a scan order. The scan order is determined based on the coding tree blocks of the picture 1150, and the scan order is determined based on the coding tree blocks of the picture 1150. Figure 11As shown, picture 1150 is divided into sixteen coding tree blocks (CTBs), with four CTBs in each row (i.e., a CTU row). Each CTB can be further divided into blocks that are processed sequentially by the block-based filtering performed by in-loop filter 100, such as blocks 163, 164, 167, and 168.

[0083] In some embodiments, in-loop filter 100 can employ scan sequence 1110 and process the CTBs of picture 1150 in raster scan pattern 1115. That is, in-loop filter 100 can process each CTB of a CTU row also in the positive x-direction before changing to process the next CTU row in the positive x-direction until each CTB of picture 1150 is processed. For each pixel position along the x-direction of picture 1150, the pixel data fetched by SAO filter 120 from memory 180 is stored in a line buffer until SAO filter 120 finishes processing the current CTU row and begins processing the next CTU. The employment of data schemes 900 or 1000 reduces the amount of pixel data that SAO filter 120 needs to fetch from memory 180, as described elsewhere herein above, resulting in a reduction in the size of the line buffer. Specifically, for each pixel position along the x-direction of picture 1150, data schemes 900 or 1000 enable the line buffer to reduce the number of pixels represented by parameter N saved , which is equal to the width of the horizontal segment of L-shaped region 911 or 1011, i.e., W L , as shown in equation (1). That is, for each pixel position along the x-direction of picture 1150, N saved = W L = min(N f2 , N ref -N rev ). Considering the entire picture 1150, the total reduction in line buffer size resulting from the employment of data schemes 900 or 1000 using raster scan pattern 1115 is:

[0084] Line Buffer Size Saving = N saved x W pic x B (2)

[0085] where W pic is the width of picture 1150, measured in pixels, and B is the number of bits used to represent each pixel.

[0086] In some embodiments, the in-loop filter 100 can employ the scan sequence 1120 and process the CTBs of the picture 1150 in the raster scan pattern 1125. In the scan sequence 1120, the CTBs of the picture 1150 are divided into several tiles, such as tile 1151 and 1152. The in-loop filter 100 processes each CTB of a CTU row within a tile in the positive x direction, before moving to the next CTU row in the tile in the positive x direction, and then until each CTB of the picture 1150 is processed. After processing each CTB in the tile 1151 in the raster scan pattern 1125, the in-loop filter 100 can move from the tile 1151 to the tile 1152 to continue processing the CTBs therein, also following the raster scan pattern 1125. Thus, benefiting from employing the data scheme 900 or 1000 using the raster scan pattern 1125, the total reduction in line buffer size is:

[0087] Line Buffer Size Saving = N saved x W tile x B (3)

[0088] where W tile is the width of the tiles 1151 and 1152, measured in pixels, and B is the number of bits used to represent each pixel. In the case where the tiles of the picture 1150 do not have the same width, W tile in equation (3) is determined by the maximum width of all the tiles of the picture 1150. For example, if the width of the tile 1151 is greater than the width of the tile 1152, the width of the tile 1151 is used as W tile in equation (3).

[0089] In addition to the size saving in the line buffer, the data schemes 900 and 1000 can benefit the SAO filter 120 by a size saving in the CTU height buffer. The CTU height buffer provides a similar function as the line buffer, except that the CTU height buffer is used to store the pixel data fetched by the SAO filter 120 from the memory 180 for each pixel position of each CTU row of the picture 1150 in the y direction. Thus, the raster scan pattern 1115 saves the size in the CTU height buffer as:

[0090] CTU Height Buffer Size Saving = N saved x H CTU x B (4) where H CTUis the height of each CTB of picture 1150, measured in pixels, and B is the number of bits used to represent each pixel. For raster scan mode 1125, where there are vertical tiles, data schemes 900 and 1000 also result in size savings in the vertical tile buffer. The vertical tile buffer is used to store the rightmost row of the current vertical tile for use in processing the immediately adjacent vertical tile to the right of the current vertical tile. Thus, the size saved in the vertical tile buffer for raster scan mode 1125 is:

[0091] Vertical Tile Buffer Size Saving-N saved x H tile x B (5) where H tile is the maximum height of a vertical tile of picture 1150, measured in pixels, and can be as tall as the height of picture 1150.

[0092] The idea presented in data schemes 900 and 1000 of deblocking filter 110 sending more pixel data to SAO filter 120 than block 710 also applies to the case where deblocking filter 110 only performs one of the horizontal and vertical filtering operations. For example, deblocking filter scheme 600 can be modified such that deblocking filter 110 outputs an output block to SAO filter 120 that also includes some of the partial filtered blocks. Figure 12 An example design is shown according to an embodiment of the disclosure in which data scheme 1200 is presented for the combination of deblocking filter 110 and SAO filter 120. In data scheme 1200, deblocking filter 110 employs filter setting (8, 6), and SAO filter 120 employs a filter setting of 2 pixels. Deblocking filter 110 has already performed horizontal filtering using deblocking filter scheme 600 with filter setting (8, 6), resulting in fully filtered sub-blocks 272, 274, 281, and 283, and partially filtered sub-blocks 282 and 284. According to data scheme 1200, after deblocking filter 110 completes processing of block 168, deblocking filter 110 can output output block 1210 to SAO filter 120, which includes not only fully filtered sub-blocks 272, 274, 281, and 283, but also a rectangular region 1211. Including rectangular region 1211 into output block 1210 as the next current block for SAO filter 120 is helpful in reducing the amount of pixel data that SAO filter 120 needs to fetch from memory 180, thereby reducing the latency introduced into SAO filter 120 and saving the size of the line buffer and / or CTU height buffer required by SAO filter 120, similar to data schemes 900 and 1000.

[0093] As Figure 12As shown, rectangular region 1211 contains respective portions of partially filtered sub-blocks 282 and 284 (i.e., {(8, 0), (9, 7)} and {(8, 8), (9, 15)} respectively), where the respective portions are adjacent to one of the fully filtered sub-blocks (i.e., sub-blocks 281 and 283 respectively). Further, similar to the width of L-shaped region 911 or 1011, the width of rectangular region 1211 is determined by equation (1).

[0094] In some embodiments, SAO filter 120 can store the pixel data of rectangular region 1211 received from deblocking filter 110 to memory 180 via bus 125, such that the pixel data of rectangular region 1211 can be retrieved by SAO filter 120 for processing future blocks. That is, in addition to receiving the pixel data of rectangular region 1211 from deblocking filter 110 and performing the SAO filtering operation using the pixel data of rectangular region 1211, SAO filter 120 can also store the pixel data of rectangular region 1211 to memory 180. However, in some alternative embodiments, SAO filter 120 can update the pixel data of rectangular region 1211 before storing the pixel data to memory 180. That is, the pixel data of rectangular region 1211 can be updated as a result of SAO filter 120 performing the SAO filtering operation for the current block, and then stored to memory 180 for future use.

[0095] Although data scheme 1200 is used with deblocking filter 110 that performs only horizontal filtering, a data scheme in which deblocking filter 110 performs only vertical filtering operations can be easily derived from data scheme 1200 by swapping the rows and columns in Figure 12

[0096] In some embodiments, SAO filter 120 can be replaced by a constrained directional enhancement filter (CDEF) that is capable of performing a de-ringing operation similar to that which SAO filter 120 is capable of performing. Each of the features described above with respect to SAO filter 120 can be directly applied to CDEF.

[0097] V. ILLUSTRATIVE EMBODIMENTS

[0098] Figure 13 ​An example video encoder 1300 is shown in which various embodiments, filtering schemes, and data schemes described elsewhere above can be employed. As shown, the video encoder 1300 receives an input video signal from a video source 1305 and encodes the signal into a bitstream 1395. The video encoder 1300 has several elements or modules for encoding the signal from the video source 1305, including at least some selected from among: a transform module 1310, a quantization module 1311, an inverse quantization module 1314, an inverse transform module 1315, an intra-estimation module 1320, an intra-prediction module 1325, a motion compensation module 1330, a motion estimation module 1335, a loop filter 1345, a reconstructed picture buffer 1350, a motion vector (MV) buffer 1365, an MV prediction module 1375, and an entropy encoder 1390. The motion compensation module 1330 and the motion estimation module 1335 are part of an inter-prediction module 1340. The inter-prediction module 1340 can include an integer motion estimation (IME) kernel configured to perform integer pixel searches, and a fractional motion estimation (FME) kernel configured to perform fractional pixel searches. Both integer pixel searches and fractional pixel searches are basic functions of the motion compensation module 1330 and the motion estimation module 1335.

[0099] In some embodiments, the modules 1310-1390 listed above are software instruction modules executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, the modules 1310-1390 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although the modules 1310-1390 are shown as separate modules, some modules can be combined into a single module.

[0100] The video source 1305 provides an original video signal that presents pixel data for each video frame without compression. That is, the video source 1305 provides a video stream that includes camera pictures presented in a time sequence. The subtracter 1308 computes a difference between the video data from the video source 1305 and the predicted pixel data 1313 from the motion compensation module 1330 or the intra-prediction module 1325. The transform module 1310 transforms the difference (or residual pixel data or residual signal) 1309 into transform coefficients 1316 (e.g., by performing a discrete cosine transform or DCT). The quantization module 1311 quantizes the transform coefficients 1316 into quantized data (or quantized coefficients) 1312, which is encoded into the bitstream 1395 by the entropy encoder 1390.

[0101] Inverse quantization module 1314 de-quantizes quantized data (or quantized coefficients) 1312 to obtain transform coefficients, and inverse transform module 1315 performs an inverse transform to the transform coefficients to generate a reconstructed residual 1319. Reconstructed residual 1319 is added to predicted pixel data 1313 to generate reconstructed pixel data 1317. In some embodiments, reconstructed pixel data 1317 is temporarily stored in a line buffer (not shown) for intra prediction and spatial MV prediction. The reconstructed pixels are filtered by loop filter 1345 and stored in reconstructed picture buffer 1350. In some embodiments, reconstructed picture buffer 1350 is a memory external to video encoder 1300. In some embodiments, reconstructed picture buffer 1350 is a memory internal to video encoder 1300.

[0102] Intra estimation module 1320 performs intra prediction based on reconstructed pixel data 1317 to generate intra predicted data. The intra predicted data is provided to entropy encoder 1390 to be encoded into bitstream 1395. The intra predicted data is also used by intra prediction module 1325 to generate predicted pixel data 1313.

[0103] Motion estimation module 1335 performs inter prediction by generating MVs to reference pixel data of previously decoded frames stored in reconstructed picture buffer 1350. These MVs are provided to motion compensation module 1330 to generate predicted pixel data.

[0104] Video encoder 1300 does not encode the full actual MVs in the bitstream, but instead uses MV prediction to generate predicted MVs, and the difference between the MVs used for motion compensation and the predicted MVs is encoded as residual motion data and stored in bitstream 1395.

[0105] MV prediction module 1375 generates predicted MVs based on reference MVs generated for encoding previous video frames, i.e., motion compensation MVs used to perform motion compensation. MV prediction module 1375 retrieves the reference MVs from MV buffer 1365 from previous video frames. Video encoder 1300 stores the MVs generated for the current video frame in MV buffer 1365 as reference MVs for generating predicted MVs.

[0106] MV prediction module 1375 uses the reference MVs to create predicted MVs. The predicted MVs can be calculated by spatial MV prediction or temporal MV prediction. The difference (residual motion data) between the predicted MVs and the motion compensation MVs (MC MVs) of the current frame are encoded by entropy encoder 1390 into bitstream 1395.

[0107] The entropy encoder 1390 encodes various parameters and data into the bitstream 1395 by using entropy encoding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding. The entropy encoder 1390 encodes various header elements, flags, along with quantized transform coefficients 1312 and residual motion data as syntax elements into the bitstream 1395. The bitstream 1395 is in turn stored in a storage device or transmitted to a decoder over a communication medium such as a network.

[0108] The loop filter 1345 performs filtering or smoothing operations on the reconstructed pixel data 1317 to reduce encoding artifacts, particularly at boundaries of blocks of pixels. The loop filter 1345 can be implemented by the loop filter 100. In some embodiments, the filtering operations performed include sample adaptive offset (SAO), which can be implemented by the SAO filter 120. In some embodiments, the filtering operations include adaptive loop filtering (ALF) operations, which can be implemented by the ALF 130.

[0109] Figure 14 An example video decoder 1400 is shown. As shown, the video decoder 1400 is an image decoding or video decoding circuit that receives the bitstream 1495 and decodes the contents of the bitstream 1495 into pixel data of video frames for display. The video decoder 1400 has several elements or modules for decoding the bitstream 1495, including some selected from the following: an inverse quantization module 1411, an inverse transform module 1410, an intra prediction module 1425, a motion compensation module 1430, a loop filter 1445, a decoded picture buffer 1450, an MV buffer 1465, an MV prediction module 1475, and a parser 1490. The motion compensation module 1430 is part of an inter prediction module 1440.

[0110] In some embodiments, the modules 1410-1490 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, the modules 1410-1490 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although the modules 1410-1490 are shown as separate modules, some modules can be combined into a single module.

[0111] A parser (e.g., entropy decoder) 1490 receives the bitstream 1495 and performs initial parsing according to the syntax defined by the video coding or image coding standard. The parsed syntax elements include various header elements, flags, and quantized data (or quantized coefficients) 1412. The parser 1490 parses the various syntax elements by using entropy coding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding.

[0112] An inverse quantization module 1411 de-quantizes the quantized data (or quantized coefficients) 1412 to obtain transform coefficients 1416, and an inverse transform module 1410 performs inverse transforms on the transform coefficients 1416 to produce a reconstructed residual signal 1419. The reconstructed residual signal 1419 is added to the predicted pixel data 1413 from either the intra-prediction module 1425 or the motion compensation module 1430 to produce decoded pixel data 1417. The decoded pixel data 1417 is filtered by a loop filter 1445 and stored in a decoded picture buffer 1450. In some embodiments, the decoded picture buffer 1450 is a memory external to the video decoder 1400. In some embodiments, the decoded picture buffer 1450 is a memory internal to the video decoder 1400.

[0113] The intra-prediction module 1425 receives intra-prediction data from the bitstream 1495 and, based thereon, produces predicted pixel data 1413 from the decoded pixel data 1417 stored in the decoded picture buffer 1450. In some embodiments, the decoded pixel data 1417 is also stored in a line buffer (not shown) for intra-prediction and spatial MV prediction.

[0114] In some embodiments, the content of the decoded picture buffer 1450 is used for display. A display device 1455 either takes the content of the decoded picture buffer 1450 directly for display, or takes the content of the decoded picture buffer 1450 to a display buffer. In some embodiments, the display device receives pixel values from the decoded picture buffer 1450 through pixel transport.

[0115] The motion compensation module 1430 produces predicted pixel data 1413 from the decoded pixel data 1417 stored in the decoded picture buffer 1450 according to motion compensation MVs (MC MVs). These motion compensation MVs are decoded by adding residual motion data received from the bitstream 1495 to predicted MVs received from the MV prediction module 1475.

[0116] The MV prediction module 1475 generates the predicted MV based on reference MVs (e.g., motion compensation MVs used to perform motion compensation) generated for decoding previous video frames. The MV prediction module 1475 retrieves the reference MVs for the previous video frames from the MV buffer 1465. The video decoder 1400 stores the motion compensation MVs generated for decoding the current video frame in the MV buffer 1465 as reference MVs for generating the predicted MVs.

[0117] The loop filter 1445 performs filtering or smoothing operations on the decoded pixel data 1417 to reduce coded artifacts, particularly at boundaries of blocks of pixels. The loop filter 1445 can be implemented by the loop filter 100. In some embodiments, the filtering operations performed include sample adaptive offset (SAO), which is implemented by the SAO filter 120. In some embodiments, the filtering operations include adaptive loop filter (ALF) operations, which can be implemented by the ALF 130.

[0118] Figure 15 A filter 1500 is shown, which can implement the loop filter 1345 of the video encoder 1300 or the loop filter 1445 of the video decoder 1400. The filter 1500 can also implement the loop filter 100 or the deblocking filter 110. The filter 1500 can process a picture using a block-based pipeline. The filter 1500 has several elements or modules, including some elements selected from the group consisting of an input module 1510, a neighboring block buffer 1520, a deblocking filter 1530, a de-ringing filter 1540, a filtered block buffer 1550, and an output module 1560.

[0119] The input module 1510 can be configured to receive a current block of a video picture, where the current block is a block that the filter 1500 currently starts processing. For example, the input module 1510 can be configured to receive the current block 168. The input module 1510 can also receive first set of pixel data of one or more neighboring blocks of the current block, and subsequently store the first set of pixel data in the neighboring block buffer 1520. For example, the first set of pixel data can include the sub-blocks 243, 244, 272, and 274, which belong to two neighboring blocks (i.e., the blocks 164 and 167) of the block 168.

[0120] The neighboring block buffer 1520 can be configured to store a first set of pixel data of one or more neighboring blocks of the current block. After the input module 1510 receives the first set of pixel data, the first set of pixel data of the one or more neighboring blocks is stored by the input module 1510 in the neighboring block buffer 1520. In some embodiments, the neighboring block buffer 1520 can additionally store a second set of pixel data of the one or more neighboring blocks. For example, the neighboring block buffer 1520 can additionally store pixel data of the L-shaped region 938 or 1038. The de-ringing filter 1540 can use the second set of pixel data to perform a de-ringing operation on the output block.

[0121] The deblocking filter 1530 can be configured to perform a deblocking operation on the current block using the pixel data of the current block and the first set of pixel data of the one or more neighboring blocks. For example, the deblocking filter 1530 can implement the deblocking filter 110 and can perform the deblocking operation according to the horizontal filtering scheme 300, the horizontal filtering scheme 400, the vertical filtering scheme 500, or the vertical filtering scheme 580. The deblocking filter 1530 can perform a vertical filtering operation along the deblocking boundary 250 using the pixel data of the block 168 and the pixel data of the sub-blocks 243 and 244. Alternatively or additionally, the deblocking filter 1530 can perform a horizontal filtering operation along the deblocking boundary 210 using the pixel data of the block 168 and the pixel data of the sub-blocks 272 and 274.

[0122] As a result of the deblocking operation, the deblocking filter 1530 can generate a filter block that includes one or more fully filtered sub-blocks and one or more partially filtered sub-blocks. The filter block buffer 1550 can be configured to store the filter block. For example, after the deblocking filter 1530 performs the deblocking operation along the deblocking boundaries 210, 218, 250, and 258, the deblocking filter 1530 can generate a filter block that contains the fully filtered sub-blocks 243, 272, and 281, and the partially filtered sub-blocks 244, 274, 282, 283, and 284. The fully filtered sub-blocks 243, 272, and 281 can be stored in a first portion 1552 of the filter block buffer 1550, while the partially filtered sub-blocks 244, 274, 282, 283, and 284 can be stored in a second portion 1554 of the filter block buffer 1550.

[0123] The output module 1560 can be configured to output an output block that includes one or more fully filtered sub-blocks stored in the first portion 1552 of the filter block buffer 1550 and a respective portion of each of one or more partially filtered sub-blocks stored in the second portion 1554 of the filter block buffer 1550. For example, the output module 1560 can be configured to output the output block 910 that includes the fully filtered sub-blocks 234, 243, 272, and 281 and the L-shaped region 911. The L-shaped region 911 is adjacent to the fully filtered sub-blocks 243, 272, and 281. As described elsewhere herein, the L-shaped region 911 is composed of respective portions of the partially filtered sub-blocks 244, 282, 284, 283, and 274, where each of the respective portions is adjacent to a corresponding one of the fully filtered sub-blocks 234, 243, 272, and 281.

[0124] The de-ringing filter 1540 can be configured to perform a de-ringing operation on the output block using pixel data of the output block and a second set of pixel data of one or more neighboring blocks. For example, the de-ringing filter 1540 can implement the SAO filter 120 and can perform the de-ringing operation using pixel data of the block 930 or the block 1030. The block 930 includes the output block 910 and the L-shaped region 938. The block 1030 includes the output block 1010 and the L-shaped region 1038. Each of the L-shaped regions 938 and 1038 includes pixel data from the blocks 163, 164, and 167, which are neighboring blocks of the current block 168.

[0125] In some embodiments, the de-ringing filter 1540 can be external to the filter 1500, and the output module 1560 can output the output block to the de-ringing filter 1540. In some embodiments, the filter 1500 can include the de-ringing filter 1540. The de-ringing filter 1540 can be a SAO filter or a constrained directional enhancement filter (CDEF). The de-ringing filter 1540 can implement the SAO filter 120.

[0126] Six, Illustrative Processing

[0127] Figure 16An example process 1600 in accordance with an embodiment of the present disclosure is shown. The process 1600 can be representative of one aspect of implementing the various proposed designs, concepts, schemes, systems, and methods described above. More specifically, the process 1600 can be representative of one aspect of the proposed concepts and schemes related to implementing an efficient loop filtering process in accordance with the present disclosure. The process 1600 can include one or more operations, actions, or functions as shown in one or more of blocks 1610, 1620, 1630, and 1640. Although shown as discrete blocks, the various blocks of the process 1600 can be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the desired implementation. Moreover, the blocks / sub-blocks of the process 1600 can be performed in the order shown, or in a different order. Additionally, one or more of the blocks / sub-blocks of the process 1600 can be performed repeatedly or iteratively. The process 1600 can be implemented by the device 1500, or in the device 1500 and any of its variants. For illustrative purposes only and without limitation of scope, the process 1600 is described below in the context of the device 1500. The process 1600 can begin at block 1610. Figure 16

[0128] At 1610, the process 1600 can involve the input module 1510 receiving pixel data of a current block. For example, the input module 1510 can receive pixel data of the block 168 from the inter / intra prediction module 103. From 1610, the process 1600 can proceed to 1620.

[0129] At 1620, the process 1600 can involve the input module 1510 receiving pixel data of one or more neighboring blocks of the current block. For example, the input module 1510 can receive pixel data of the sub-blocks 243 and 244 and pixel data of the sub-blocks 272 and 274 from the inter / intra prediction module 103. The sub-blocks 243 and 244 belong to the block 164, which is the immediately above neighboring block of the current block 168. The sub-blocks 272 and 274 belong to the block 167, which is the immediately left neighboring block of the current block 168. The process 1600 can also involve the input module 1510 storing the pixel data of the one or more neighboring blocks to the neighboring block buffer 1520. From 1620, the process 1600 can proceed to 1630.

[0130] ​At 1630, process 1600 can involve deblocking filter 1530 generating a filtered block by performing a filtering operation on current block. The filtering operation can be performed using pixel data of current block and pixel data of one or more neighboring blocks. The filtered block can include one or more fully filtered sub-blocks and one or more partially filtered sub-blocks. For example, deblocking filter 1530 can employ data scheme 1200 and perform a horizontal filtering operation along deblocking boundaries 210 and 218 of current block 168 using deblocking filter settings (8, 6). The horizontal filtering operation can be performed using pixel data of block 168 and pixel data of neighboring block 167 (in particular, pixel data of sub-blocks 272 and 274). Deblocking filter 1530 can accordingly generate a filtered block that includes four fully filtered sub-blocks 272, 274, 281, and 283, and two partially filtered sub-blocks 282 and 284. Process 1600 can proceed from 1630 to 1640.

[0131] At 1640, process 1600 can involve output module 1560 outputting an output block that includes a respective portion of each of one or more fully filtered sub-blocks and one or more partially filtered sub-blocks, where each respective portion is adjacent to one of the fully filtered sub-blocks. For example, output module 1560 can employ data scheme 1200 and output output block 1210. Output block 1210 includes fully filtered sub-blocks 272, 274, 281, and 283, and rectangular region 1211. Rectangular region 1211 includes a portion of partially filtered sub-block 282 and a portion of partially filtered sub-block 284. The portion of partially filtered sub-block 282, i.e., {(8, 0), (9, 7)}, is adjacent to fully filtered sub-block 281. The portion of partially filtered sub-block 284, i.e., {(8, 8), (9, 15)}, is adjacent to fully filtered sub-block 283.

[0132] Figure 17 An example process 1700 according to an embodiment of the disclosure is shown. Process 1700 can represent one aspect of implementing the various proposed designs, concepts, schemes, systems, and methods described above. More specifically, process 1700 can represent one aspect of the proposed concepts and schemes related to implementing an efficient loop filtering process according to the disclosure. Process 1700 can include one or more operations, actions, or functions as shown in one or more of blocks 1710, 1720, 1730, 1740, 1750, and 1760. Although shown as discrete blocks, each block of process 1700 can be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the desired implementation. Moreover, the blocks of process 1700 can be performed in any order, in any sequence or Figure 17The order shown is performed, or performed in a different order. Moreover, one or more blocks / sub-blocks of the process 1700 can be performed repeatedly or iteratively. The process 1700 can be performed by the device 1500 or implemented in the device 1500 and any of its variants. For illustrative purposes only and without limitation of scope, the process 1700 is described below in the context of the device 1500. The process 1700 can begin at block 1710.

[0133] At 1710, the process 1700 can involve the input module 1510 receiving pixel data of a current block. For example, the input module 1510 can receive the pixel data of the block 168 from the inter / intra prediction module 103. The process 1700 can proceed from 1710 to 1720.

[0134] At 1720, the process 1700 can involve the input module 1510 receiving first pixel data of one or more neighboring blocks of the current block. For example, the input module 1510 can receive the pixel data of the sub-blocks 243 and 244 and the pixel data of the sub-blocks 272 and 274 from the inter / intra prediction module 103. The sub-blocks 243 and 244 belong to the block 164, which is the immediately neighboring block above the current block 168. The sub-blocks 272 and 274 belong to the block 167, which is the immediately neighboring block to the left of the current block 168. The process 1700 can also involve the input module 1510 storing the pixel data of the one or more neighboring blocks into the neighboring block buffer 1520. The process 1700 can proceed from 1720 to 1730.

[0135] At 1730, the process 1700 can involve the deblocking filter 1530 generating a filtered block by performing a filtering operation on the current block. The filtering operation can be performed using the pixel data of the current block and the pixel data of the one or more neighboring blocks. The filtered block can include one or more fully filtered sub-blocks and one or more partially filtered sub-blocks. For example, the deblocking filter 1530 can employ the data scheme 1000 and perform a horizontal filtering operation and a vertical filtering operation using the deblocking filter setting (8, 6). The deblocking filter 1530 can perform the horizontal filtering operation along the deblocking boundaries 210 and 218 and the vertical filtering operation along the deblocking boundaries 250 and 258. The horizontal filtering operation can be performed using the pixel data of the block 168 and the pixel data of the neighboring block 167, in particular, the pixel data of the sub-blocks 272 and 274. The vertical filtering operation can be performed using the pixel data of the block 168 and the pixel data of the neighboring block 164, in particular, the pixel data of the sub-blocks 243 and 244. The deblocking filter 1530 can accordingly generate a filtered block that includes three fully filtered sub-blocks 243, 272, and 281 and five partially filtered sub-blocks 244, 282, 284, 283, and 274. The process 1700 can proceed from 1730 to 1740.

[0136] At 1740, process 1700 can involve output module 1560 outputting an output block including one or more fully filtered sub-blocks and a respective portion of each of one or more partially filtered sub-blocks, where each respective portion is adjacent to one of the fully filtered sub-blocks. For example, output module 1560 can employ data scheme 1000 and output output block 1010. Output block 1010 includes fully filtered sub-blocks 234, 243, 272, and 281, and L-shaped region 1011. L-shaped region 1011 includes a portion of partially filtered sub-block 244, a portion of partially filtered sub-block 282, a portion of partially filtered sub-block 284, a portion of partially filtered sub-block 283, and a portion of partially filtered sub-block 274. The portion of partially filtered sub-block 244, i.e., {(8, -8), (9, -1)}, is adjacent to an edge of fully filtered sub-block 243. The portion of partially filtered sub-block 282, i.e., {(8, 0), (9, 7)}, is adjacent to an edge of fully filtered sub-block 281. The portion of partially filtered sub-block 284, i.e., {(8, 8), (9, 9)}, is adjacent to a corner of fully filtered sub-block 281. The portion of partially filtered sub-block 283, i.e., {(0, 8), (7, 9)}, is adjacent to an edge of fully filtered sub-block 281. The portion of partially filtered sub-block 274, i.e., {(-8, 8), (-1, 9)}, is adjacent to an edge of fully filtered sub-block 272. Process 1700 can proceed from 1740 to 1750.

[0137] At 1750, process 1700 can involve input module 1510 receiving a second set of pixel data for one or more neighboring blocks of the current block. For example, de-ringing filter 1540 can employ data scheme 1000 and receive pixel data for L-shaped region 1038 from inter / intra prediction module 103 as the second set of pixel data. L-shaped region 1038 includes pixel data for blocks 163, 164, and 167, which are neighboring blocks of current block 168. Process 1700 can also involve input module 1510 storing the second set of pixel data to neighboring block buffer 1520. Process 1700 can further involve de-ringing filter 1540 receiving the second set of pixel data from neighboring block buffer 1520. Process 1700 can proceed from 1750 to 1760.

[0138] At 1760, process 1700 can involve de-ringing filter 1540 performing a second filtering operation on each pixel of one or more fully filtered sub-blocks using the output block's pixel data, the second set of pixel data, or both. For example, de-ringing filter 1540 can perform a SAO filtering operation on each pixel of block 710 using a 2-pixel SAO filter setting. De-ringing filter 1540 can perform the SAO filtering operation using pixel data within block 1030, which includes block 1010 and L-shaped region 1038. For each pixel of block 710, de-ringing filter 1540 can reference two pixels in each of a left direction, a right direction, a top direction, and a bottom direction and modify the value of the respective pixel accordingly. In some embodiments, de-ringing filter 1540 can also reference two pixels from the respective pixel in each of a top-left direction, a top-right direction, a bottom-left direction, and a bottom-right direction and modify the value of the respective pixel accordingly.

[0139] Seven, illustrative electronic system

[0140] Many of the above-described features and applications are implemented as software processes that are specified as a set of instructions recorded on a computer readable storage medium (also referred to as computer readable medium). When these instructions are executed by one or more computational or processing unit(s) (e.g., one or more processors, cores of processors, or other processing units), they cause the processing unit(s) to perform the actions indicated in the instructions. Examples of computer readable media include, but are not limited to, read only memory (ROM), flash memory, volatile memory, non-volatile memory, EPROM, EEPROM, etc. Computer readable media do not include carrier waves and electronic signals over wired or wireless computer networks.

[0141] In this specification, the term "software" is meant to include firmware residing in read-only memory or applications stored in magnetic storage which can be read into memory for processing by a processor. Also, in some embodiments, multiple software applications can be implemented as sub-parts of a larger program while remaining distinct software applications. In some embodiments, multiple software applications can also be implemented as separate programs. Finally, any combination of separate programs that together implement a software application described here is within the scope of the application. In some embodiments, the software programs, when installed to operate on one or more electronic systems, define one or more specific machine implementations that execute and perform the operations of the software programs.

[0142] Figure 18 An electronic system 1800 implementing some embodiments of the present disclosure is conceptually illustrated. The electronic system 1800 can be a computer (e.g., a desktop computer, a personal computer, a tablet computer, etc.), a telephone, a personal digital assistant (PDA), or any other kind of electronic device. Such an electronic system includes various types of computer readable media and interfaces for various other types of computer readable media. The electronic system 1800 includes a bus 1805, a processing unit 1810, a graphics-processing unit (GPU) 1815, a system memory 1820, a network 1825, a read-only memory 1830, a persistent storage 1835, input devices 1840, and output devices 1845.

[0143] The bus 1805 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the electronic system 1800. For instance, the bus 1805 communicatively connects the processing unit 1810 with the GPU 1815, the read-only memory 1830, the system memory 1820, and the persistent storage 1835.

[0144] From these various memory units, the processing unit 1810 retrieves instructions to execute and data to process in order to execute the processes of the present disclosure. The processing unit can be a single processor or a multi-core processor in different embodiments. Some instructions are passed to and executed by the GPU 1815. The GPU 1815 can offload various computations or supplement the image processing provided by the processing unit 1810.

[0145] Read only memory (ROM) 1830 stores static data and instructions that are used by processing unit(s) 1810 and other modules of the electronic system. On the other hand, permanent storage device 1835 is a read-and-write memory device. This device is a non-volatile memory unit that stores instructions and data even when electronic system 1800 is off. Some embodiments of the disclosure use a mass-storage device (such as a magnetic or optical disk and its corresponding disk drive) as permanent storage device 1835.

[0146] Other embodiments use a removable storage device (such as a floppy disk, flash drive, etc., and its corresponding disk drive) as permanent storage device. Like permanent storage device 1835, system memory 1820 is a read-and-write memory device. However, unlike storage device 1835, system memory 1820 is a volatile read-and-write memory, such as a random access memory. System memory 1820 stores some of the instructions and data that the processor needs at runtime. In some embodiments, the processes according to the disclosure are stored in system memory 1820, permanent storage device 1835, and / or read-only memory 1830. For example, according to some embodiments, various memory units include instructions for processing a multimedia clip. From these various memory units, processing unit(s) 1810 fetches the instructions to be executed and the data to be processed, in order to execute the processes of some embodiments.

[0147] Bus 1805 also connects to input and output devices 1840 and 1845. Input device(s) 1840 enable the user to communicate information and select commands to the electronic system. Input device(s) 1840 include alphanumeric keyboards and pointing devices (also called “cursor control devices”), cameras (e.g., webcams), microphones or similar devices for receiving voice commands, etc. Output device(s) 1845 display

[0148] Finally, as Figure 18As shown, bus 1805 also couples electronic system 1800 to a network (not shown) through a network adapter (not shown). The computer can be part of a network of computers such as a local area network ("LAN"), a wide area network ("WAN"), or an Intranet, or a network of networks, such as the Internet. Any or all components of electronic system 1800 can be used in conjunction with the subject disclosure.

[0149] Some embodiments include electronic components, such as microprocessors, storage and memory that store computer program instructions in a machine- readable or computer-readable (alternatively referred to as computer-readable storage media, machine-readable media, or machine- readable storage media) a form of which includes RAM, ROM, EEPROM, flash memory or other memory technology, CD ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information in a form readable by a computing device. For example, the computer readable (storage) media can include any type of disk including floppy disks, optical disks, DVDs, Blu-Ray, hard disks, solid state drives, or tape. Such computer readable storage media, when carrying computer readable instructions, represent embodiments of carrier waves used to program computers to perform particular tasks. The computer program instructions can be executed by one or more processing units to cause a computer or other electronic device to perform the tasks.

[0150] While the above discussion primarily refers to microprocessor or multi-core processors that execute software, many of the above-described functions and applications are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the integrated circuits themselves. In addition, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0151] As used in the description of the application and in any claims, the terms "computer", "server", "processor", and "memory" all refer to electronic or other technological devices. These terms do not encompass humans or groups of humans. For illustrative purposes, the term "display" or "display device" refers to display on an electronic device. As used in the description of the application and in any claims, the terms "computer readable medium", "computer readable media", and "machine readable medium" are entirely restricted to tangible, physical objects, which store information in a computer readable format. These terms do not encompass any wireless signals, wired download signals, and other such transitory, propagating signals.

[0152] While the application has been described in connection with what is presently considered to be the most practical and preferred embodiments, it is to be understood that the application is not to be limited to the disclosed embodiments, but on the contrary, is intended to cover various modifications and equivalent arrangements.

[0153] Additional Description

[0154] The herein described subject matter sometimes illustrates different features or elements being in a combination of some embodiments. It is to be understood that the described features or elements can be combined with each other in any manner within the scope of the application unless the combination of certain features is explicitly prohibited. The descriptions as used herein provide examples that can help the reader visualize various aspects or embodiments of the application. However, the described examples should not be deemed to narrow the scope of the application, but merely to provide examples that aid one skilled in the art to visualize aspects of the application. Thus, the scope and spirit of the application should be judged in terms of the claims that follow, rather than the above descriptions.

[0155] Furthermore, with respect to the use of substantially any plural and / or singular term herein, those having ordinary skill in the art can withhold to a singular and / or plural interpretation as is appropriate to the context and / or application. The present application explicitly disclaims any limitations that might result from any such attached interpretation to a singular and / or plural meaning, encompassed by any single term used herein.

[0156] Furthermore, those skilled in the art will recognize that, generally, the terms used herein, particularly in the claims, are generally intended as "open" terms (e.g., the term "including" is to be interpreted as "including but not limited to," the term "having" is to be interpreted as "having at least," the term "includes" is to be interpreted as "includes, but is not limited to," etc.). Those skilled in the art will further recognize that where terms are used in accordance with their ordinary usage, those ordinary meanings are intended; and that ordinary usage terms generally function as "open" terms. It will be further understood by those within the art that any or all specific examples of structures and / or methods previously expressed herein are intended as illustrative only and not limiting as to the scope of the application. For example, while the application has been described herein in the context of a particular implementation in a particular environment and / or with particular components, it will be apparent to those of ordinary skill in the art that the features, objects, and / or operations described herein can be implemented with a substantially different combination of hardware and that according to the application, those features, objects, and / or operations need not be implemented in any specific sequence, nor according to any specific architecture. Accordingly, those skilled in the art will recognize that the systems and / or methods described herein in connection with the present application can be embodied in a variety of different forms and that each of the various implementations and / or examples disclosed herein are to be considered equivalent in terms of the scope of the application. Furthermore, the application described herein is intended to cover and encompass all known and available forms of the systems and / or methods as well as their equivalents. Additionally, the application described herein is intended to cover and encompass all ways known to those skilled in the art to use the systems and / or methods in connection with any possible application. Also, to the extent that any patent, publication, or other disclosure is identified herein as being "incorporated by reference," the present application incorporates each such patent, publication, or other disclosure in its entirety. Moreover, in the event that any provisions of the following claims conflict, then these sections shall be construed as contributing to the fullest extent allowed by law, to the collection of rights to which the claims as a whole are entitled. Furthermore, those skilled in the art will recognize that terms such as "above," "below," "up," "down," "top," "bottom," "front," "back," "leading," "trailing," and the like, are used herein for descriptive purposes only, and can not necessarily be construed as referring to anything in particular.

[0157] From the foregoing, it will be appreciated that, for purposes of illustration, various embodiments of the application have been described in terms that connect those embodiments to a particular application. After reading this description, it will be apparent to a person having ordinary skill in the art how the various embodiments can be implemented in different kinds of applications. It will also be apparent that various modifications can be made within the scope and spirit of the application. Thus, the various embodiments described above are not intended to limit the true scope and spirit of the application, which is defined by the appended claims.

Claims

1. A method for processing the current block of an image, comprising: Receive the pixel data of the current block; Receive pixel data from one or more adjacent blocks of the current block; A filtered block is generated by performing a filtering operation on the current block using the pixel data of the current block and the pixel data of one or more neighboring blocks. The filtered block includes one or more fully filtered sub-blocks and one or more partially filtered sub-blocks. as well as The output block comprises one or more fully filtered sub-blocks and a corresponding portion of each of the one or more partially filtered sub-blocks, the corresponding portion being adjacent to one of the fully filtered sub-blocks. The filtering operation includes a deblocking operation. The images include reconstructed images using block-based intra-frame or inter-frame prediction. The filtering operation includes one or both of the following: Perform a horizontal filtering operation along at least one vertical deblocking boundary of the current block, and Perform a vertical filtering operation along at least one horizontal deblocking boundary of the current block. The horizontal filtering operation includes, on each side of at least one vertical deblocking boundary, referencing a first number of pixels and modifying a second number of pixels, and The vertical filtering operation includes referencing a third number of pixels and modifying a fourth number of pixels on each side of at least one horizontal deblocking boundary.

2. The method for processing the current block of an image as described in claim 1, characterized in that, The one or more adjacent blocks include a first adjacent block immediately above the current block and a second adjacent block immediately to the left of the current block. The one or more partial filter sub-blocks include three partial filter sub-blocks overlapping the current block, and two partial filter sub-blocks overlapping the first adjacent block and the second adjacent block, respectively. The corresponding portions of the multiple partially filtered sub-blocks together form an L-shaped region.

3. The method for processing the current block of an image as described in claim 1, characterized in that, The one or more adjacent blocks include the adjacent block immediately to the left of the current block. The one or more partially filtered sub-blocks include a partially filtered sub-block that overlaps with the current block, and The corresponding portion of the partially filtered sub-block includes a rectangular region.

4. The method for processing the current block of an image as described in claim 1, characterized in that, The execution of the filtering operation includes performing the horizontal filtering operation. The corresponding portion includes a rectangular region with a fifth horizontal dimension, and The fifth quantity is equal to the difference between the first quantity and the second quantity.

5. The method for processing the current block of an image as described in claim 1, characterized in that, The execution of the filtering operation includes performing the vertical filtering operation. The corresponding portion includes a rectangular region with a sixth vertical dimension, and The sixth quantity is equal to the difference between the third quantity and the fourth quantity.

6. The method for processing the current block of an image as described in claim 1, characterized in that, Performing the filtering operation includes performing the horizontal filtering operation and the vertical filtering operation; The corresponding portions of the multiple partially filtered sub-blocks collectively form an L-shaped region, the L-shaped region comprising a first segment and a second segment, the first segment having a width of a fifth number of pixels, and the second segment having a width of a sixth number of pixels. The fifth quantity is equal to the difference between the first quantity and the second quantity, and The sixth quantity is equal to the difference between the third quantity and the fourth quantity.

7. The method for processing the current block of an image as described in claim 1, characterized in that, The pixel data of one or more adjacent blocks includes a first set of pixel data, and the filtering operation includes a first filtering operation. The method further includes: Receive the second set of pixel data from the one or more adjacent blocks. A second filtering operation is performed on each pixel of the one or more fully filtered sub-blocks using the pixel data of the output block, the second set of pixel data, or both, wherein the second filtering operation includes: Referencing a seventh number of pixels adjacent to the corresponding pixel of the one or more fully filtered sub-blocks in each of the left, right, up, and down directions; and The corresponding pixels of the one or more fully filtered sub-blocks are modified based on the reference.

8. The method for processing the current block of an image as described in claim 7, characterized in that, The corresponding portions of the partially filtered sub-blocks collectively form an L-shaped region, the L-shaped region comprising a first segment and a second segment, the first segment having a width of a fifth number of pixels, and the second segment having a width of a sixth number of pixels. The second filtering operation further includes referencing a seventh number of pixels in the upper-left, upper-right, lower-left, and lower-right directions, said pixels being adjacent to the corresponding pixels of the one or more fully filtered sub-blocks, and The seventh quantity is equal to the fifth quantity and the sixth quantity.

9. The method for processing the current block of an image as described in claim 7, characterized in that, The first filtering operation includes a deblocking operation performed by the first filter. The second filtering operation includes a deringing operation performed by the second filter. The second filter includes a sample adaptive offset filter or a constraint direction enhancement filter, and The first filter and the second filter are configured to process the image using block-based pipelined processing.

10. An apparatus for processing a current block of an image, comprising: The input module is set as the current block for receiving images; The adjacent block buffer is configured to store the first set of pixel data of one or more adjacent blocks of the current block; A deblocking filter is configured to perform a deblocking operation on the current block using pixel data of the current block and the first set of pixel data of one or more adjacent blocks; The filter block buffer is configured to store one or more fully filtered sub-blocks and one or more partially filtered sub-blocks; as well as The output module is configured to output an output block, the output block comprising the one or more fully filtered sub-blocks, and a corresponding portion of each of the one or more partially filtered sub-blocks, the corresponding portion being adjacent to one of the fully filtered sub-blocks. The deblocking filter is configured to perform the deblocking operation along at least one vertical or horizontal deblocking boundary of the current block by referencing a first number of pixels and modifying a second number of pixels on each side of at least one vertical or horizontal deblocking boundary.

11. The apparatus as claimed in claim 10, characterized in that, The one or more adjacent blocks include a first adjacent block immediately above the current block and a second adjacent block immediately to the left of the current block. The deblocking operation includes horizontal filtering and vertical filtering. The one or more partial filter sub-blocks include three partial filter sub-blocks overlapping the current block, and two partial filter sub-blocks overlapping the first adjacent block and the second adjacent block, respectively. The corresponding portions of the multiple partially filtered sub-blocks together form an L-shaped region.

12. The apparatus as claimed in claim 10, characterized in that, The one or more adjacent blocks include the adjacent block immediately to the left of the current block. The deblocking operation includes a horizontal filtering operation. The one or more partially filtered sub-blocks include a partially filtered sub-block that overlaps with the current block, and The corresponding portion of the partially filtered sub-block includes a rectangular region.

13. The apparatus as claimed in claim 10, characterized in that, The corresponding portion includes a rectangular region having a fifth width or length, and The fifth quantity is equal to the difference between the first quantity and the second quantity.

14. The apparatus as claimed in claim 13, characterized in that, The plurality of corresponding portions of the plurality of partially filtered sub-blocks together form an L-shaped region including a first segment and a second segment, wherein each of the first segment and the second segment has a width of the fifth number of pixels.

15. The apparatus as claimed in claim 10, characterized in that, The adjacent block buffer is also configured to store a second set of pixel data for the one or more adjacent blocks. The device further includes: The deringing filter is configured to perform a deringing operation on each pixel of the one or more fully filtered sub-blocks using pixel data from the output block and a second set of pixel data from the one or more adjacent blocks.

16. The apparatus as claimed in claim 15, characterized in that, The de-ringing filter is configured to perform the de-ringing operation by referencing a seventh number of pixels adjacent to each pixel of the one or more fully filtered sub-blocks in each of the left, right, top, and bottom directions, and to modify the corresponding pixels of the one or more fully filtered sub-blocks based on the reference.

17. The apparatus as claimed in claim 16, characterized in that, The de-ringing filter is configured to perform the de-ringing operation by further referencing a seventh number of pixels adjacent to the corresponding pixels of the one or more fully filtered sub-blocks in each of the upper left, upper right, lower left, and lower right directions.

18. The apparatus as claimed in claim 15, characterized in that, The ringing removal filter includes a sample adaptive offset filter or a constraint direction enhancement filter, and The deblocking filter and the deringing filter are configured to process the image using block-based pipelined processing.

Images