Amplification filtering of video data

Abstract

An apparatus for amplifying and filtering video data includes one or more processors configured to generate a number of amplified pixels that is an integer multiple of half a size of a filter used to amplify and filter video data. The device may then store remaining pixels to be used to amplify and filter a subsequent right adjacent block. In this manner, the apparatus may avoid generating amplified pixels that would result in misalignment with the block and may not be output due to misalignment.

Description

[0001] Cross - Reference to Related Applications

[0002] This application is a PCT and claims the benefit of U.S. utility patent application Ser. No. 18 / 069,076, filed Dec. 20, 2022, the entire content of which is incorporated herein by reference. Technical Field

[0003] This disclosure relates to video coding and decoding, and more particularly, to filtering video data. Background Art

[0004] Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital live systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e - book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radiotelephones, so - called "smart phones", video teleconferencing devices, video streaming devices, and the like. Digital video devices implement video coding techniques such as those described in standards defined by MPEG - 2, MPEG - 4, ITU - T H.263, ITU - T H.264 / MPEG - 4 Part 10 Advanced Video Coding (AVC), ITU - T H.265 / High Efficiency Video Coding (HEVC), ITU - T H.266 / Versatile Video Coding (VVC), and extensions of such standards, as well as proprietary video codecs / formats such as AOMedia Video 1 (AV1) developed by the Alliance for Open Media. Video devices can more efficiently transmit, receive, encode, decode, and / or store digital video information by implementing such video coding techniques.

[0005] Video coding techniques include spatial (intra - picture) prediction and / or temporal (inter - picture) prediction to reduce or remove redundancy inherent in a video sequence. For block - based video coding, a video slice (e.g., a video picture or a portion of a video picture) can be partitioned into video blocks, which may also be referred to as coding tree units (CTUs), coding units (CUs), and / or coding nodes. Intra - coded (I) slices of video blocks in a picture are encoded using spatial prediction with respect to reference samples in adjacent blocks in the same picture. Video blocks in inter - coded (P or B) slices of a picture can use spatial prediction with respect to reference samples in adjacent blocks in the same picture or temporal prediction with respect to reference samples in other reference pictures. A picture can be referred to as a frame, and a reference picture can be referred to as a reference frame. Summary of the Invention

[0006] In general, the present disclosure describes techniques for efficiently upscaling decoded video data. In some cases, the captured video data may be downscaled before encoding and then upscaled after decoding, which can improve compression efficiency relative to encoding and decoding full-scale video data. Video data is typically processed in raster scan order (i.e., from left to right). Decoding and upscaling filtering can be performed in parallel, such that after a block has been decoded, upscaling filtering can be performed on that block while other blocks are being decoded. This results in the pixels of the right adjacent block of the block not being available for the upscaling filter. Thus, according to the techniques of the present disclosure, pixels of a block for which upscaling cannot be performed from the current block (e.g., the rightmost pixels whose number corresponds to half of the filter width) can be stored and used for upscaling filtering the remaining pixels of that block and the pixels of the right adjacent block. In this way, the techniques of the present disclosure can improve the efficiency of upscaling filtering by avoiding generating upscaled pixels that are not aligned with the block width and can more accurately upscale filter pictures of video data.

[0007] In one example, a method of upscaling filtering video data includes: determining a filter scaling value that represents half of the number of support pixels to be used by an upscaling filter to upscale filter a video data block; determining an upscaling ratio of the video data block; calculating a size multiplier value as a product of the upscaling ratio and a difference between a size of the video data block and the filter scaling value; calculating a rounding value based on the size multiplier value such that the rounding value is the largest integer multiple of the filter scaling value that does not exceed the size multiplier value; for each row of the video data block, generating upscaled pixels whose number is equal to the rounding value, and not generating upscaled pixels for the following number of pixels starting from the right edge of the video block, the number being equal to a difference between a product of the size of the block and the upscaling ratio and the rounding value; for each row of the video data block, storing values of the number of pixels starting from the right edge of the video block of the video data; and using the values of the number of pixels starting from the right edge of the video block to upscale filter pixels of a right adjacent video data block of the video data block.

[0008] In another example, an apparatus for performing upsampling filtering on video data, the apparatus comprising: a memory configured to store video data; and one or more processors implemented in circuitry and configured to: determine a filter scaling value, the filter scaling value representing half of the number of support pixels to be used by an upsampling filter to perform upsampling filtering on a video data block; determine an upsampling ratio of the video data block; calculate a size multiplier value as a product of the upsampling ratio and a difference between a size of the video data block and the filter scaling value; calculate a rounding value based on the size multiplier value such that the rounding value is the largest integer multiple of the filter scaling value that does not exceed the size multiplier value; generate, for each row of the video data block, upsampled pixels the number of which is equal to the rounding value, without generating upsampled pixels for the following number of pixels starting from a right edge of the video block, the number being equal to a difference between a product of the size of the block and the upsampling ratio and the rounding value; for each row of the video data block, store values of the number of pixels starting from the right edge of the video block of the video data; and use the values of the number of pixels starting from the right edge of the video block that are stored to perform upsampling filtering on pixels of a right adjacent video data block of the video data block.

[0009] In another example, a computer-readable storage medium having instructions stored thereon that, when executed, cause a processor to: determine a filter scaling value, the filter scaling value representing half of the number of support pixels to be used by an upsampling filter to perform upsampling filtering on a video data block; determine an upsampling ratio of the video data block; calculate a size multiplier value as a product of the upsampling ratio and a difference between a size of the video data block and the filter scaling value; calculate a rounding value based on the size multiplier value such that the rounding value is the largest integer multiple of the filter scaling value that does not exceed the size multiplier value; generate, for each row of the video data block, upsampled pixels the number of which is equal to the rounding value, without generating upsampled pixels for the following number of pixels starting from a right edge of the video block, the number being equal to a difference between a product of the size of the block and the upsampling ratio and the rounding value; for each row of the video data block, store values of the number of pixels starting from the right edge of the video block of the video data; and use the values of the number of pixels starting from the right edge of the video block that are stored to perform upsampling filtering on pixels of a right adjacent video data block of the video data block.

[0010] In another example, an apparatus for performing upsampling filtering on video data, the apparatus comprising: means for determining a filter scaling value, the filter scaling value representing half the number of support pixels to be used by an upsampling filter to perform upsampling filtering on a video data block; means for determining an upsampling ratio of the video data block; means for calculating a size multiplier value as a product of the upsampling ratio and a difference between a size of the video data block and the filter scaling value; means for calculating a rounding value based on the size multiplier value such that the rounding value is the largest integer multiple of the filter scaling value that does not exceed the size multiplier value; means for generating, for each row of the video data block, upsampled pixels the number of which is equal to the rounding value, without generating upsampled pixels for the following number of pixels starting from a right edge of the video block, the number being equal to a difference between a product of a size of the block and the upsampling ratio and the rounding value; means for storing, for each row of the video data block, values of the number of pixels starting from the right edge of the video block of the video data; and means for performing upsampling filtering on pixels of a right adjacent video data block of the video data block using the values of the number of pixels starting from the right edge of the video block that are stored.

[0011] Details of one or more examples are set forth in the accompanying drawings and the following description. Other features, objects, and advantages will be apparent from the description, the drawings, and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A block diagram illustrating an example video encoding and decoding system that can execute the techniques of the present disclosure.

[0013] Figure 2A AND 2B A conceptual diagram illustrating pixels that can be used when performing upsampling filtering on a video data block.

[0014] Figure 3 A block diagram illustrating an example video encoder that can execute the techniques of the present disclosure.

[0015] Figure 4 A block diagram illustrating an example video decoder that can execute the techniques of the present disclosure.

[0016] Figure 5 A block diagram illustrating an example of a set of components that can be included in an upsampling filter unit.

[0017] Figure 6A AND 6B A conceptual diagram illustrating an input block of video data and a corresponding upsampled block generated by upsampling filtering.

[0018] Figure 7Conceptual diagram for illustrating a technique for performing upsampling filtering on a video data block.

[0019] Figure 8 Conceptual diagram for illustrating an example technique for performing upsampling filtering on a video data block according to the techniques of the present disclosure.

[0020] Figure 9 Flowchart for illustrating an example method for encoding a current block according to the techniques of the present disclosure.

[0021] Figure 10 Flowchart for illustrating an example method for decoding a current block according to the techniques of the present disclosure.

[0022] Figure 11 Flowchart for illustrating an example method for performing upsampling filtering on a video data block according to the techniques of the present disclosure. Detailed Description

[0023] A video encoder (or preprocessing unit) may receive raw unencoded video data to be encoded and apply a downsampling filter to the received video data to downsample the video data spatially. After downsampling the video data, the video encoder may encode the downsampled video data. Similarly, a video decoder may decode the downsampled video data and then apply an upsampling filter to the decoded video data to form upsampled video data. The upsampling filter may be, for example, an eight-tap filter that uses the values of the current pixel, three pixels to the left of the current pixel, and four pixels to the right of the current pixel to form one or more upsampled pixels. The number of upsampled pixels may correspond to an upsampling ratio, which may be, for example, 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, 2, or other values within this range or greater than 2.

[0024] The video decoder (or postprocessing unit) may perform upsampling filtering in parallel with decoding such that only the samples of the current block and the left adjacent block are available for upsampling filtering. Thus, the video decoder may avoid generating upsampled pixels for the pixels on the far right of the block that would require accessing the pixels of the right adjacent block.

[0025] Additionally, the techniques of the present disclosure include generating upsampled pixels that are aligned with the block width rather than generating upsampled pixels that are not aligned with the block width. That is, the number of upsampled pixels generated for the current block may be a multiple of half of the filter width. Upsampled pixels may be generated for the remaining pixels of the current block while generating upsampled pixels for subsequent blocks in the same row as the current block. To generate upsampled pixels for the remaining pixels, a certain number of unfiltered pixel values of the current block may be stored for reference by the upsampling filter, where the number may be, for example, 1 less than half of the filter width.

[0026] Thus, for example, the filter can be an 8-tap filter with 8 support pixels, including three pixels to the left of the current pixel, the current pixel, and four pixels to the right of the current pixel. Accordingly, the values of the three pixels to the left of the four rightmost pixels of the current block will be stored along with the four rightmost pixels such that the four rightmost pixels can be upsampled and filtered while also filtering the right-adjacent block of the current block. Although the techniques of the present disclosure are described with respect to an 8-tap filter, these techniques can also be applied to filters of other sizes, such as 16-tap, 32-tap, or the like.

[0027] In this way, the techniques of the present disclosure can improve the efficiency of upsampling and filtering video data. Heuristic testing of these techniques on 4K video data has shown improved performance at all tested upsampling ratios, with improvements ranging from 6.67% to 11%, as summarized in Table 1 below:

[0028] Table 1

[0029] Magnification ratio Performance improvement 1.125 11% 1.25 10% 1.375 9.09% 1.5 8.33% 1.625 7.69% 1.75 7.14% 1.875 6.67%

[0030] Figure 1 is a block diagram illustrating an example video encoding and decoding system 100 that can implement the techniques of the present disclosure. The techniques of the present disclosure generally relate to decoding (encoding and / or decoding) video data. Generally speaking, video data includes any data for processing video. Thus, video data can include raw, unencoded video, encoded video, decoded (e.g., reconstructed) video, and video metadata, such as signaling data.

[0031] As Figure 1 shown, in this example, system 100 includes a source device 102 that provides encoded video data to be decoded and displayed by a destination device 116. Specifically, source device 102 provides video data to destination device 116 via a computer-readable medium 110. Source device 102 and destination device 116 can include any of a wide range of devices, including desktop computers, notebook (i.e., laptop) computers, mobile devices, tablet computers, set-top boxes, telephone handsets such as smart phones, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, broadcast receiver devices, or the like. In some cases, source device 102 and destination device 116 can be equipped for wireless communication and can thus be referred to as wireless communication devices.

[0032] In Figure 1In the example, the source device 102 includes a video source 104, a memory 106, a video encoder 200, and an output interface 108. The destination device 116 includes an input interface 122, a video decoder 300, a memory 120, and a display device 118. According to the present disclosure, the video encoder 200 of the source device 102 and the video decoder 300 of the destination device 116 may be configured to apply techniques for performing upsampling filtering on video data. Thus, the source device 102 represents an example of a video encoding device, while the destination device 116 represents an example of a video decoding device. In other examples, the source device and the destination device may include other components or arrangements. For example, the source device 102 may receive video data from an external video source (such as an external camera). Similarly, the destination device 116 may interface with an external display device instead of including an integrated display device.

[0033] As Figure 1 shown, the system 100 is merely an example. Generally, any digital video encoding and / or decoding device may perform techniques for performing upsampling filtering on video data. The source device 102 and the destination device 116 are merely examples of such encoding devices, where the source device 102 generates encoded video data for transmission to the destination device 116. The present disclosure refers to an "encoding" device as a device that performs encoding (encoding and / or decoding) of data. Thus, the video encoder 200 and the video decoder 300 represent examples of encoding devices, particularly a video encoder and a video decoder. In some examples, the source device 102 and the destination device 116 may operate in a substantially symmetric manner such that each of the source device 102 and the destination device 116 includes video encoding and decoding components. Thus, the system 100 may support one-way or two-way video transmission between the source device 102 and the destination device 116, such as for video streaming, video playback, video broadcasting, or video telephony.

[0034] Typically, video source 104 represents a source of video data (i.e., raw, undecoded video data) and provides a series of consecutive pictures (also referred to as "frames") of the video data to video encoder 200, which encodes the data of the pictures. The video source 104 of source device 102 may include a video capture device, such as a camera, a video archive containing previously captured raw video, and / or a video feed interface for receiving video from a video content provider. As another alternative, the video source 104 may generate computer graphics-based data as the source video, or a combination of live video, archived video, and computer-generated video. In each case, the video encoder 200 encodes the captured, pre-captured, or computer-generated video data. The video encoder 200 may reorder the pictures from the received order (sometimes referred to as the "display order") into a decoding order for decoding. The video encoder 200 may generate a bitstream including the encoded video data. The source device 102 may then output the encoded video data to a computer-readable medium 110 via output interface 108 for reception and / or retrieval by an input interface 122 of, for example, a destination device 116.

[0035] The memory 106 of the source device 102 and the memory 120 of the destination device 116 represent general-purpose memories. In some examples, the memories 106, 120 may store raw video data, e.g., raw video from the video source 104 and raw decoded video data from the video decoder 300. Additionally or alternatively, the memories 106, 120 may store software instructions that may be executed, for example, by the video encoder 200 and the video decoder 300, respectively. Although the memory 106 and the memory 120 are shown separately from the video encoder 200 and the video decoder 300 in this example, it should be understood that the video encoder 200 and the video decoder 300 may also include internal memories for functionally similar or equivalent purposes. Further, the memories 106, 120 may store, for example, the encoded video data output from the video encoder 200 and input to the video decoder 300. In some examples, portions of the memories 106, 120 may be allocated as one or more video buffers, e.g., to store raw, decoded, and / or encoded video data.

[0036] Computer-readable medium 110 may represent any type of medium or device capable of transmitting encoded video data from source device 102 to destination device 116. In one example, computer-readable medium 110 represents a communication medium that enables source device 102 to send the encoded video data directly to destination device 116 in real time, for example, via a radio frequency network or a computer-based network. According to a communication standard (e.g., a wireless communication protocol), output interface 108 may modulate a transmission signal including the encoded video data, and input interface 122 may demodulate the received transmission signal. The communication medium may include any wireless or wired communication medium, such as the radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network (e.g., a local area network, a wide area network, or a global network such as the Internet). The communication medium may include routers, switches, base stations, or any other device that may be used to facilitate communication from source device 102 to destination device 116.

[0037] In some examples, source device 102 may output the encoded data from output interface 108 to storage device 112. Similarly, destination device 116 may access the encoded data from storage device 112 via input interface 122. Storage device 112 may include any one of a variety of distributed or locally accessible data storage media, such as a hard disk drive, a Blu-ray disc, a DVD, a CD-ROM, a flash memory, a volatile or non-volatile memory, or any other suitable digital storage media for storing the encoded video data.

[0038] In some examples, source device 102 may output the encoded video data to file server 114 or another intermediate storage device that may store the encoded video data generated by source device 102. Destination device 116 may access the stored video data from file server 114 via streaming or downloading.

[0039] File server 114 may be any type of server device capable of storing the encoded video data and sending the encoded video data to destination device 116. File server 114 may represent a web server (e.g., for a website), a server configured to provide a file transfer protocol service (e.g., File Transfer Protocol (FTP) or File Delivery over Unidirectional Transport (FLUTE) protocol), a content delivery network (CDN) device, a Hypertext Transfer Protocol (HTTP) server, a Multimedia Broadcast Multicast Service (MBMS) or Enhanced MBMS (eMBMS) server, and / or a Network Attached Storage (NAS) device. File server 114 may additionally or alternatively implement one or more HTTP streaming protocols, such as HTTP Dynamic Adaptive Streaming (DASH), HTTP Live Streaming (HLS), Real-Time Streaming Protocol (RTSP), HTTP Dynamic Streaming, or the like.

[0040] The destination device 116 can access the encoded video data from the file server 114 via any standard data connection, including an Internet connection. This can include a wireless channel (e.g., Wi-Fi connection), a wired connection (e.g., Digital Subscriber Line (DSL), cable modem, etc.), or a combination of both, suitable for accessing the encoded video data stored on the file server 114. The input interface 122 can be configured to operate according to any one or more of the various protocols for retrieving or receiving media data from the file server 114 as discussed above, or other such protocols for retrieving media data.

[0041] The output interface 108 and the input interface 122 can represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In an example where the output interface 108 and the input interface 122 include wireless components, the output interface 108 and the input interface 122 can be configured to transmit data (e.g., encoded video data) according to a cellular communication standard (e.g., 4G, 4G-LTE (Long Term Evolution), LTE Advanced, 5G, or the like). In some examples where the output interface 108 includes a wireless transmitter, the output interface 108 and the input interface 122 can be configured to transmit data (e.g., encoded video data) according to other wireless standards (e.g., IEEE 802.11 specifications, IEEE 802.15 specifications (e.g., ZigBee™), Bluetooth™ standards, or the like). In some examples, the source device 102 and / or the destination device 116 can include corresponding System-on-Chip (SoC) devices. For example, the source device 102 can include an SoC device for performing the functions attributed to the video encoder 200 and / or the output interface 108, and the destination device 116 can include an SoC device for performing the functions attributed to the video decoder 300 and / or the input interface 122.

[0042] The techniques of the present disclosure can be applied to video decoding supporting any of a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, Internet streaming video transmission (e.g., HTTP Dynamic Adaptive Streaming over HTTP (DASH)), digital video encoded onto a data storage medium, decoding of digital video stored on a data storage medium, or other applications.

[0043] The input interface 122 of the destination device 116 receives an encoded video bitstream from a computer-readable medium 110 (e.g., a communication medium, a storage device 112, a file server 114, or the like). The encoded video bitstream can include signaling information defined by the video encoder 200, which is also used by the video decoder 300, such as syntax elements having values that describe the characteristics and / or processing of video blocks or other decoding units (e.g., slices, pictures, groups of pictures, sequences, etc.). The display device 118 displays decoded pictures of the decoded video data to the user. The display device 118 can represent any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0044] Although Figure 1 not shown, in some examples, the video encoder 200 and the video decoder 300 can each be integrated with an audio encoder and / or an audio decoder and can include appropriate MUX-DEMUX units or other hardware and / or software to process a multiplexed stream that includes both audio and video in a common data stream.

[0045] The video encoder 200 and the video decoder 300 can each be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the techniques are implemented partially in software, the device can store instructions for the software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform the techniques of the present disclosure. Each of the video encoder 200 and the video decoder 300 can be included in one or more encoders or decoders, and any of the encoders or decoders can be integrated as part of a combined encoder / decoder (CODEC) in a corresponding device. Devices including the video encoder 200 and / or the video decoder 300 can include integrated circuits, microprocessors, and / or wireless communication devices, such as cellular telephones.

[0046] Video encoder 200 and video decoder 300 may operate according to a video coding standard, such as ITU-T H.265, also known as High Efficiency Video Coding (HEVC) or extensions thereof, such as multi-view and / or scalable video coding extensions. Alternatively, video encoder 200 and video decoder 300 may operate according to other proprietary or industry standards, such as ITU-T H.266, also known as Versatile Video Coding (VVC). In other examples, video encoder 200 and video decoder 300 may operate according to a proprietary video codec / format, such as AOMedia Video 1 (AV1), extensions of AV1, and / or successor versions of AV1 (e.g., AV2). In other examples, video encoder 200 and video decoder 300 may operate according to other proprietary formats or industry standards. However, the techniques of the present disclosure are not limited to any particular coding standard or format. Generally, video encoder 200 and video decoder 300 may be configured to perform the techniques of the present disclosure in conjunction with any video coding technique that performs upsampling filtering on video data.

[0047] Generally, video encoder 200 and video decoder 300 may perform block-based coding of pictures. The term "block" generally refers to a structure that includes data to be processed (e.g., encoded, decoded, or otherwise used during encoding and / or decoding). For example, a block may include a two-dimensional matrix of samples of luminance and / or chrominance data. Generally, video encoder 200 and video decoder 300 may code video data represented in YUV (e.g., Y, Cb, Cr) format. That is, video encoder 200 and video decoder 300 may code a luminance component and chrominance components, rather than coding red, green, and blue (RGB) data of samples of a picture, where the chrominance components may include both a red chrominance component and a blue chrominance component. In some examples, video encoder 200 converts the received RGB format data to YUV representation before encoding, and video decoder 300 converts the YUV representation to RGB format. Alternatively, a preprocessing and postprocessing unit (not shown) may perform these conversions.

[0048] The present disclosure may generally relate to coding (e.g., encoding and decoding) of pictures to include a process of encoding or decoding data of a picture. Similarly, the present disclosure may relate to coding of blocks of a picture to include a process of encoding or decoding data of a block, such as prediction and / or residual coding. An encoded video bitstream generally includes a series of values for representing coding decisions (e.g., coding modes) and syntax elements that partition a picture into blocks. Thus, a reference to coding a picture or a block should generally be understood as coding values of syntax elements that form the picture or the block.

[0049] HEVC defines various blocks, including coding units (CUs), prediction units (PUs), and transform units (TUs). According to HEVC, a video decoder (such as video encoder 200) divides a coding tree unit (CTU) into CUs according to a quadtree structure. That is, the video decoder divides the CTU and CUs into four equal, non-overlapping squares, and each node of the quadtree has zero or four child nodes. A node without child nodes may be referred to as a "leaf node", and the CU of such a leaf node may include one or more PUs and / or one or more TUs. The video decoder may further divide the PUs and TUs. For example, in HEVC, a residual quadtree (RQT) represents the division of TUs. In HEVC, PUs represent inter-prediction data, while TUs represent residual data. Intra-predicted CUs include intra-prediction information, such as an intra-mode indication.

[0050] As another example, video encoder 200 and video decoder 300 may be configured to operate according to VVC. According to VVC, a video decoder (such as video encoder 200) divides a picture into a plurality of coding tree units (CTUs). Video encoder 200 may divide a CTU according to a tree structure (such as a quadtree-binary tree (QTBT) structure or a multi-type tree (MTT) structure). The QTBT structure removes the concept of multiple division types, such as the separation between CUs, PUs, and TUs in HEVC. The QTBT structure includes two levels: a first level divided according to quadtree division and a second level divided according to binary tree division. The root node of the QTBT structure corresponds to the CTU. The leaf nodes of the binary tree correspond to coding units (CUs).

[0051] In the MTT division structure, quadtree (QT) division, binary tree (BT) division, and one or more types of ternary tree (TT) (also referred to as a trinary tree (TT)) division can be used to divide a block. Ternary tree or trinary tree division is a division in which a block is split into three sub-blocks. In some examples, the ternary tree or trinary tree division divides a block into three sub-blocks without splitting the original block through the center. The division types in the MTT (e.g., QT, BT, and TT) can be symmetric or asymmetric.

[0052] When operating according to the AV1 codec, the video encoder 200 and the video decoder 300 can be configured to decode video data in blocks. In AV1, the largest coding block that can be processed is called a superblock. In AV1, a superblock can be 128×128 luma samples or 64×64 luma samples. However, in a successor video coding format (e.g., AV2), a superblock can be defined by a different (e.g., larger) luma sample size. In some examples, a superblock is the top level of a block quadtree. The video encoder 200 can also divide a superblock into smaller decoding blocks. The video encoder 200 can divide a superblock and other coding blocks into smaller blocks using square or non-square partitioning. Non-square blocks can include N / 2×N, N×N / 2, N / 4×N, and N×N / 4 blocks. The video encoder 200 and the video decoder 300 can perform separate prediction and transform processing on each decoding block.

[0053] AV1 also defines tiles of video data. A tile is a rectangular array of superblocks that can be decoded independently of other tiles. That is, the video encoder 200 and the video decoder 300 can encode and decode the coding blocks within a tile, respectively, without using video data from other tiles. However, the video encoder 200 and the video decoder 300 can perform filtering across tile boundaries. The size of a tile can be uniform or non-uniform. Tile-based coding can enable parallel processing and / or multi-threading for encoder and decoder implementations.

[0054] In some examples, the video encoder 200 and the video decoder 300 can use a single QTBT or MTT structure to represent each of the luma and chroma components, while in other examples, the video encoder 200 and the video decoder 300 can use two or more QTBT or MTT structures, such as one QTBT / MTT structure for the luma component and another QTBT / MTT structure for the two chroma components (or two QTBT / MTT structures for the respective chroma components).

[0055] The video encoder 200 and the video decoder 300 can be configured to use quadtree partitioning, QTBT partitioning, MTT partitioning, superblock partitioning, or other partitioning structures.

[0056] In some examples, a CTU includes a coding tree block (CTB) of luma samples, two corresponding CTBs of chroma samples for a picture having three sample arrays, or a CTB of samples for a monochrome picture or a picture coded using three separate color planes and syntax structures for coding the samples. For a certain value of N, a CTB may be a block of N×N samples such that the partitioning of components into CTBs is a segmentation. A component may be an array or a single sample from one of the three arrays (luma and two chromas) of a picture in 4:2:0, 4:2:2, or 4:4:4 color format, or an array or a single sample of an array for a picture in monochrome format. In some examples, for some values of M and N, a coding block is a block of M×N samples such that the partitioning of a CTB into coding blocks is a segmentation.

[0057] Blocks (e.g., CTUs or CUs) may be grouped in a picture in various ways. As an example, a tile may refer to a rectangular region of CTU rows within a particular tile in a picture. A tile may be a rectangular region of CTUs within a particular tile column and a particular tile row in a picture. A tile column refers to a rectangular region of CTUs having a height equal to the height of the picture and a width specified by a syntax element (e.g., in a picture parameter set). A tile row refers to a rectangular region of CTUs having a height specified by a syntax element (e.g., in a picture parameter set) and a width equal to the width of the picture.

[0058] In some examples, a tile may be divided into a plurality of tiles, each of the plurality of tiles may include one or more CTU rows within the tile. A tile that is not divided into a plurality of tiles may also be referred to as a tile. However, a tile that is a proper subset of a tile may not be referred to as a tile. Tiles in a picture may also be arranged in slices. A slice may be an integral number of tiles of a picture that may be exclusively included in a single network abstraction layer (NAL) unit. In some examples, a slice includes a contiguous sequence of several complete tiles or complete tiles of only one tile.

[0059] The present disclosure may interchangeably use "N×N" and "N by N" to refer to the sample size of a block (e.g., a CU or other video block) in terms of vertical and horizontal dimensions, such as 16×16 samples or 16 by 16 samples. Generally, a 16×16 CU will have 16 samples in the vertical direction (y = 16) and 16 samples in the horizontal direction (x = 16). Similarly, an N×N CU typically has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. Samples in a CU may be arranged in rows and columns. Additionally, a CU does not necessarily need to have the same number of samples in the horizontal direction as in the vertical direction. For example, a CU may include N×M samples, where M does not necessarily equal N.

[0060] Video encoder 200 encodes video data of a CU representing prediction and / or residual information and other information. The prediction information indicates how to predict the CU to form a prediction block of the CU. The residual information generally represents the sample-by-sample difference between the samples of the CU before encoding and the prediction block.

[0061] To predict a CU, video encoder 200 can generally form a prediction block of the CU by inter-frame prediction or intra-frame prediction. Inter-frame prediction generally refers to predicting the CU from data of a previous decoded picture, while intra-frame prediction generally refers to predicting the CU from previously decoded data of the same picture. To perform inter-frame prediction, video encoder 200 can use one or more motion vectors to generate a prediction block. Video encoder 200 can generally perform a motion search to identify a reference block that closely matches the CU, for example, in terms of the difference between the CU and the reference block. Video encoder 200 can calculate a difference metric using sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), mean squared difference (MSD), or other such difference calculations to determine whether the reference block closely matches the current CU. In some examples, video encoder 200 can use uni-directional prediction or bi-directional prediction to predict the current CU.

[0062] Some examples of VVC also provide an affine motion compensation mode, which can be regarded as an inter-frame prediction mode. In the affine motion compensation mode, video encoder 200 can determine two or more motion vectors representing non-translational motion (such as zooming in or out, rotation, perspective motion, or other irregular motion types).

[0063] To perform intra-frame prediction, video encoder 200 can select an intra-frame prediction mode to generate a prediction block. Some examples of VVC provide sixty-seven intra-frame prediction modes, including various directional modes, as well as a planar mode and a DC mode. Generally, video encoder 200 selects an intra-frame prediction mode that describes adjacent samples of the current block (e.g., the block of the CU), and predicts the samples of the current block from the intra-frame prediction mode. Assuming that video encoder 200 decodes CTUs and CUs in a raster scan order (from left to right, from top to bottom), such samples can generally be above the current block, above and to the left of the current block, or to the left in the same picture as the current block.

[0064] Video encoder 200 encodes data representing the prediction mode of the current block. For example, for an inter-frame prediction mode, video encoder 200 can encode data indicating which one of the various available inter-frame prediction modes is used and the motion information of the corresponding mode. For example, for uni-directional or bi-directional inter-frame prediction, video encoder 200 can use advanced motion vector prediction (AMVP) or a merge mode to encode the motion vector. Video encoder 200 can use a similar mode to encode the motion vectors for the affine motion compensation mode.

[0065] AV1 includes two general techniques for encoding and decoding the decoding blocks of video data. The two general techniques are intra prediction (e.g., intra-frame prediction or spatial prediction) and inter prediction (e.g., inter-frame prediction or temporal prediction). In the context of AV1, when predicting a block of the current frame of video data using an intra prediction mode, the video encoder 200 and the video decoder 300 do not use the video data from other frames of the video data. For most intra prediction modes, the video encoder 200 encodes the block of the current frame based on the difference between the sample values in the current block and the predicted values generated from the reference samples in the same frame. The video encoder 200 determines the predicted values generated from the reference samples based on the intra prediction mode.

[0066] After prediction (such as intra prediction or inter prediction of a block), the video encoder 200 may calculate the residual data of the block. The residual data (e.g., residual block) represents the per-sample difference between the block and the predicted block for the block, which is formed using the corresponding prediction mode. The video encoder 200 may apply one or more transforms to the residual block to generate transform data in the transform domain rather than the sample domain. For example, the video encoder 200 may apply a discrete cosine transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform to the residual video data. Additionally, the video encoder 200 may apply a secondary transform after the first transform, such as a mode-dependent non-separable second-order transform (MDNSST), a signal-dependent transform, a Karhunen-Loeve transform (KLT), etc. The video encoder 200 generates transform coefficients after applying one or more transforms.

[0067] As described above, after any transform to generate transform coefficients, the video encoder 200 may perform quantization of the transform coefficients. Quantization generally refers to the process of quantizing the transform coefficients to potentially reduce the amount of data used to represent the transform coefficients, thereby providing further compression. By performing the quantization process, the video encoder 200 may reduce the bit depth associated with some or all of the transform coefficients. For example, the video encoder 200 may round an n-bit value down to an m-bit value during quantization, where n is greater than m. In some examples, to perform quantization, the video encoder 200 may perform a bitwise right shift of the value to be quantized.

[0068] After quantization, the video encoder 200 may scan the transform coefficients to generate a one-dimensional vector from a two-dimensional matrix including the quantized transform coefficients. The scan may be designed to place higher energy (and thus lower frequency) transform coefficients at the front of the vector and lower energy (and thus higher frequency) transform coefficients at the back of the vector. In some examples, the video encoder 200 may use a predefined scan order to scan the quantized transform coefficients to generate a serialized vector, and then entropy encode the quantized transform coefficients of the vector. In other examples, the video encoder 200 may perform adaptive scanning. After scanning the quantized transform coefficients to form a one-dimensional vector, the video encoder 200 may entropy encode the one-dimensional vector, for example, according to context-adaptive binary arithmetic coding (CABAC). The video encoder 200 may also entropy encode the values of syntax elements that describe metadata associated with the encoded video data for use by the video decoder 300 when decoding the video data.

[0069] To perform CABAC, the video encoder 200 may assign a context within a context model to the symbol to be sent. The context may relate to, for example, whether the neighboring values of the symbol are zero values. Probability determination may be based on the context assigned to the symbol.

[0070] The video encoder 200 may also generate syntax data, such as block-based syntax data, picture-based syntax data, and sequence-based syntax data, for the video decoder 300 in, for example, a picture header, a block header, a slice header, or other syntax data (such as a sequence parameter set (SPS), a picture parameter set (PPS), or a video parameter set (VPS)). The video decoder 300 may likewise decode such syntax data to determine how to decode the corresponding video data.

[0071] In this way, the video encoder 200 may generate a bitstream including the encoded video data, for example, syntax elements that describe the segmentation of a picture into blocks (e.g., CUs) and the prediction and / or residual information of the blocks. Ultimately, the video decoder 300 may receive the bitstream and decode the encoded video data.

[0072] Generally, the video decoder 300 performs processing opposite to that performed by the video encoder 200 to decode the encoded video data of the bitstream. For example, the video decoder 300 may use CABAC to decode the values of the syntax elements of the bitstream in a manner that is substantially similar but inverse to the CABAC encoding process of the video encoder 200. The syntax elements may define segmentation information for dividing a picture into CTUs and dividing each CTU according to a corresponding segmentation structure (e.g., QTBT structure) to define the CUs of the CTU. The syntax elements may further define the prediction and residual information of the blocks (e.g., CUs) of the video data.

[0073] Residual information may be represented by, for example, quantized transform coefficients. The video decoder 300 may inverse quantize and inverse transform the quantized transform coefficients of a block to reproduce the residual block of the block. The video decoder 300 uses the signaling prediction mode (intra or inter prediction) and associated prediction information (e.g., motion information for inter prediction) to form a prediction block of the block. Then, the video decoder 300 may combine the prediction block and the residual block (on a sample-by-sample basis) to reproduce the original block. The video decoder 300 may perform additional processing, such as performing a deblocking process to reduce visual artifacts along the boundaries of the block.

[0074] The present disclosure may generally refer to "signaling" certain information, such as syntax elements. The term "signaling" may generally refer to the communication of values of syntax elements and / or other data for decoding encoded video data. That is, the video encoder 200 may signal the value of a syntax element in the bitstream. Generally speaking, signaling refers to generating a value in the bitstream. As mentioned above, the source device 102 may transmit the bitstream to the destination device 116 substantially in real time or non-real time, such as may occur when storing the syntax element in the storage device 112 for later retrieval by the destination device 116.

[0075] According to the techniques of the present disclosure, after video data has been decoded (i.e., reproduced) by the video encoder 200 or the video decoder 300, the video encoder 200 and the video decoder 300 may perform upsampling filtering on the decoded video data according to the techniques of the present disclosure. The techniques of the present disclosure are described mainly with respect to the video decoder 300, but it should be understood that the video encoder 200 may also perform these techniques after decoding / reproducing the video data.

[0076] After decoding a video data block, the video decoder 300 may start decoding the right-adjacent block of the block and also perform upsampling filtering on the decoded video data block according to the techniques of the present disclosure. Alternatively, a post-processing unit ( Figure 1 not shown) may receive the decoded block from the video decoder 300 and perform the upsampling filtering process described in the present disclosure.

[0077] The upsampling filter may be an 8-tap filter, i.e., eight support pixels may be used to generate one or more upsampled pixels (which may also include modifying the value of one or more of the support pixels). Thus, when performing upsampling filtering on the pixels of the current block, the upsampling filter may avoid generating upsampled-filtered pixels for the four rightmost pixels of the current block (i.e., the pixels equal to half the width of the filter in number). Since three pixels to the left of these four rightmost pixels will be needed to perform upsampling filtering on the remaining four rightmost pixels, all seven of these pixels may be stored for later use when the right-adjacent block is ready for upsampling filtering.

[0078] Figure 2A and 2B is a conceptual diagram of pixels that can be used when performing upsampling filtering on video data blocks. Figure 2A Depicts a block 130 of 16×16 pixels to be horizontally upsampled. The white non-shadowed pixels of block 130 can be upsampled using the left adjacent pixels 132, the white pixel itself, the pixel to be stored 134, and the unfiltered pixel 136. The unfiltered pixel 136 is not filtered together with the other pixels of block 130 because the pixels of the right adjacent block may not be available. Thus, after the right adjacent block has been decoded, the unfiltered pixel 136 and the pixel to be stored 134 can be stored for filtering the unfiltered pixel 136 together with the pixels of the right adjacent block (not shown) of block 130.

[0079] Figure 2B Depicts the support pixels 140 for an 8-tap upsampling filter, including the center pixel 142 (also referred to as the "current pixel"). The 8-tap upsampling filter can be used to form one or more output pixels, including the center pixel 142 and Figure 2B the other pixels shown in, i.e., three left adjacent pixels and four right adjacent pixels. Video coding and filtering standards (such as AOMedia Video 1 (AV1)) support various upsampling ratios, such as 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2. The upsampled output can be generated in a 4×4 pixel format because the loop filter parameters can be defined at a 4×4 level. Thus, the last four columns of pixels (e.g., Figure 2A the unfiltered pixel 136) are not filtered until the right adjacent block has been decoded and is available for reference.

[0080] The following equations can be used for the upsampling filter:

[0081] For a pixel shift in the output, the horizontal (X position) shift in the input pixel domain:

[0082]

[0083] The X position in the input pixel domain to generate the first output pixel;

[0084]

[0085]

[0086] Center pixel position:

[0087]

[0088] Upsampling filter equation:

[0089]

[0090] Figure 3 is a block diagram illustrating an example video encoder 200 that can implement the techniques of the present disclosure. It is provided for purposes of explanation Figure 3 and should not be construed as a limitation on the techniques widely illustrated and described in the present disclosure. For purposes of explanation, the present disclosure describes a video encoder 200 for techniques according to VVC (ITU-T H.266, in development) and HEVC (ITU-T H.265). However, the techniques of the present disclosure can be performed by a video coding device configured for other video coding standards and video coding formats (e.g., AV1 and successors to the AV1 video coding format).

[0091] In Figure 3 the example, video encoder 200 includes a video data memory 230, a mode selection unit 202, a residual generation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a filter unit 216, a decoded picture buffer (DPB) 218, an entropy coding unit 220, and an amplification filter unit 228. Any one or all of the video data memory 230, the mode selection unit 202, the residual generation unit 204, the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the filter unit 216, the DPB 218, the entropy coding unit 220, and the amplification filter unit 228 can be implemented in one or more processors or in processing circuitry. For example, the units of video encoder 200 can be implemented as one or more circuits or logic elements, as part of a hardware circuit, or as part of a processor, ASIC, or FPGA. Additionally, video encoder 200 can include additional or alternative processors or processing circuitry to perform these and other functions.

[0092] The video data memory 230 can store video data to be encoded by components of video encoder 200. Video encoder 200 can receive the video data from, for example, a video source 104 ( Figure 1)(Receive the video data stored in the video data memory 230. The DPB 218 can act as a reference picture memory that stores reference video data for predicting subsequent video data by the video encoder 200. The video data memory 230 and the DPB 218 can be formed of any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The video data memory 230 and the DPB 218 can be provided by the same memory device or separate memory devices. In various examples, the video data memory 230 can be on-chip with other components of the video encoder 200, as illustrated, or off-chip relative to those components.)

[0093] In the present disclosure, a reference to the video data memory 230 should not be construed as limited to memory internal to the video encoder 200 unless specifically so described, or as limited to memory external to the video encoder 200 unless specifically so described. Instead, a reference to the video data memory 230 should be understood as a reference memory that stores video data received by the video encoder 200 for encoding (e.g., video data of a current block to be encoded). Figure 1 The memory 106 can also provide temporary storage of outputs from various units of the video encoder 200.

[0094] Figure 3 Various units are shown to assist in understanding the operations performed by the video encoder 200. The units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. A fixed-function circuit is a circuit that provides specific functionality and is preset in the operations it can perform. A programmable circuit is a circuit that can be programmed to perform various tasks and provides flexible functionality in the operations it can perform. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by instructions of the software or firmware. A fixed-function circuit can execute software instructions (e.g., to receive parameters or output parameters), but the type of operations performed by the fixed-function circuit is generally immutable. In some examples, one or more of the units can be different circuit blocks (fixed-function or programmable), and in some examples, one or more of the units can be integrated circuits.)

[0095] The video encoder 200 can include an arithmetic logic unit (ALU) basic function unit (EFU), digital circuits, analog circuits, and / or a programmable core formed of programmable circuits. In an example where software executed by programmable circuits is used to perform the operations of the video encoder 200, the memory 106 ( Figure 1)(e.g., object code), or another memory (not shown) within the video encoder 200 may store such instructions. The instruction storage may store instructions of software received and executed by the video encoder 200.

[0096] The video data memory 230 is configured to store received video data. The video encoder 200 may retrieve pictures of the video data from the video data memory 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202. The video data in the video data memory 230 may be the original video data to be encoded.

[0097] The mode selection unit 202 includes a motion estimation unit 222, a motion compensation unit 224, and an intra prediction unit 226. The mode selection unit 202 may include additional functional units for performing video prediction according to other prediction modes. As an example, the mode selection unit 202 may include a palette unit, a block copy unit (which may be part of the motion estimation unit 222 and / or the motion compensation unit 224), an affine unit, a linear model (LM) unit, or the like.

[0098] The mode selection unit 202 generally coordinates multiple encoding passes to test combinations of encoding parameters and the resulting rate-distortion values of these combinations. The encoding parameters may include the CTU-to-CU partitioning, the prediction mode of the CU, the transform type of the residual data of the CU, the quantization parameter of the residual data of the CU, and so on. The mode selection unit 202 may ultimately select a combination of encoding parameters having a better rate-distortion value than other tested combinations.

[0099] The video encoder 200 may divide a picture retrieved from the video data memory 230 into a series of CTUs and encapsulate one or more CTUs within a slice. The mode selection unit 202 may divide the CTUs of the picture according to a tree structure (e.g., an MTT structure, a QTBT structure, a superblock structure, or the quadtree structure described above). As described above, the video encoder 200 may form one or more CUs by dividing the CTUs according to a tree structure. This CU may generally also be referred to as a "video block" or a "block".

[0100] Generally, the mode selection unit 202 also controls its components (e.g., the motion estimation unit 222, the motion compensation unit 224, and the intra prediction unit 226) to generate a prediction block for a current block (e.g., a current CU, or in HEVC, the overlapping portion of a PU and a TU). For inter prediction of a current block, the motion estimation unit 222 may perform a motion search to identify one or more reference blocks in one or more reference pictures (e.g., one or more previously decoded pictures stored in the DPB 218) that closely match. Specifically, the motion estimation unit 222 may calculate a value representing how similar a potential reference block is to the current block, for example, according to the sum of absolute differences (SAD), the sum of squared differences (SSD), the mean absolute difference (MAD), the mean squared difference (MSD), or the like. The motion estimation unit 222 typically may perform these calculations using the sample-by-sample differences between the current block and the considered reference block. The motion estimation unit 222 may identify the reference block having the lowest value generated by these calculations, indicating the reference block that most closely matches the current block.

[0101] The motion estimation unit 222 may form one or more motion vectors (MVs) that define the position of the reference block in the reference picture relative to the position of the current block in the current picture. The motion estimation unit 222 may then provide the motion vectors to the motion compensation unit 224. For example, for uni-directional inter prediction, the motion estimation unit 222 may provide a single motion vector, and for bi-directional inter prediction, the motion estimation unit 222 may provide two motion vectors. The motion compensation unit 224 may then use the motion vectors to generate the prediction block. For example, the motion compensation unit 224 may retrieve the data of the reference block using the motion vectors. As another example, if the motion vectors have fractional sample precision, then the motion compensation unit 224 may interpolate the values of the prediction block according to one or more interpolation filters. Additionally, for bi-directional inter prediction, the motion compensation unit 224 may retrieve the data of the two reference blocks identified by the respective motion vectors and combine the retrieved data (e.g., by sample-by-sample averaging or weighted averaging).

[0102] When operating according to the AV1 video coding format, the motion estimation unit 222 and the motion compensation unit 224 may be configured to use translational motion compensation, affine motion compensation, overlapping block motion compensation (OBMC), and / or combined inter-intra prediction to encode decoded blocks (e.g., both luminance and chrominance decoded blocks) of video data.

[0103] As another example, for intra prediction or intra prediction decoding, the intra prediction unit 226 may generate a prediction block from samples adjacent to the current block. For example, for the directional mode, the intra prediction unit 226 may typically mathematically combine the values of adjacent samples and fill in these computed values in a defined direction on the current block to produce a prediction block. As another example, for the DC mode, the intra prediction unit 226 may compute the average value of adjacent samples of the current block and generate a prediction block to include this resulting average value for each sample of the prediction block.

[0104] When operating according to the AV1 video coding format, the intra prediction unit 226 may be configured to encode decoded blocks of video data (e.g., luminance and chrominance decoded blocks) using directional intra prediction, non-directional intra prediction, recursive filter intra prediction, chroma format for luma (CFL) prediction, intra block copy (IBC), and / or palette mode. The mode selection unit 202 may include additional functional units for performing video prediction according to other prediction modes.

[0105] The mode selection unit 202 provides the prediction block to the residual generation unit 204. The residual generation unit 204 receives the original undecoded version of the current block from the video data memory 230 and receives the prediction block from the mode selection unit 202. The residual generation unit 204 computes the sample-by-sample difference between the current block and the prediction block. The resulting sample-by-sample difference defines the residual block of the current block. In some examples, the residual generation unit 204 may also determine the differences between the sample values in the residual block to generate the residual block using residual differential pulse code modulation (RDPCM). In some examples, the residual generation unit 204 may be formed using one or more subtractor circuits that perform binary subtraction.

[0106] In an example where the mode selection unit 202 partitions a CU into PUs, each PU may be associated with a luminance prediction unit and a corresponding chrominance prediction unit. The video encoder 200 and the video decoder 300 may support PUs of various sizes. As indicated above, the size of a CU may refer to the size of the luminance decoded block of the CU, and the size of a PU may refer to the size of the luminance prediction unit of the PU. Assuming a particular CU has a size of 2N×2N, the video encoder 200 may support PU sizes of 2N×2N or N×N for intra prediction, and symmetric PU sizes of 2N×2N, 2N×N, N×2N, N×N, or similar for inter prediction. The video encoder 200 and the video decoder 300 may also support asymmetric partitions of PU sizes of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter prediction.

[0107] In an example where the mode selection unit 202 does not further split a CU into PUs, each CU may be associated with a luminance decoding block and a corresponding chrominance decoding block. As described above, the size of a CU may refer to the size of the luminance decoding block of the CU. The video encoder 200 and the video decoder 300 may support CU sizes of 2N×2N, 2N×N, or N×2N.

[0108] For other video decoding techniques such as intra block copy mode decoding, affine mode decoding, and linear model (LM) mode decoding, as some examples, the mode selection unit 202 generates a prediction block of the current block being encoded via a corresponding unit associated with the decoding technique. In some examples such as palette mode decoding, the mode selection unit 202 may not generate a prediction block but instead generates a syntax element indicating a way to reconstruct a block based on a selected palette. In such modes, the mode selection unit 202 may provide these syntax elements to the entropy encoding unit 220 for encoding.

[0109] As described above, the residual generation unit 204 receives the video data of the current block and the corresponding prediction block. The residual generation unit 204 then generates a residual block of the current block. To generate the residual block, the residual generation unit 204 calculates the per-sample difference between the prediction block and the current block.

[0110] The transform processing unit 206 applies one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). The transform processing unit 206 may apply various transforms to the residual block to form the transform coefficient block. For example, the transform processing unit 206 may apply a discrete cosine transform (DCT), an oriented transform, a Karhunen-Loeve transform (KLT), or a conceptually similar transform to the residual block. In some examples, the transform processing unit 206 may perform multiple transforms on the residual block, e.g., a first transform and a second transform such as a rotation transform. In some examples, the transform processing unit 206 does not apply a transform to the residual block.

[0111] When operating according to AV1, the transform processing unit 206 may apply one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). The transform processing unit 206 may apply various transforms to the residual block to form the transform coefficient block. For example, the transform processing unit 206 may apply a horizontal / vertical transform combination that may include a discrete cosine transform (DCT), an asymmetric discrete sine transform (ADST), a flipped ADST (e.g., ADST in reverse order), and an identity transform (IDTX). When using the identity transform, the transform is skipped in one of the vertical or horizontal directions. In some examples, the transform processing may be skipped.

[0112] Quantization unit 208 may quantize the transform coefficients in a transform coefficient block to produce a quantized transform coefficient block. Quantization unit 208 may quantize the transform coefficients of the transform coefficient block according to a quantization parameter (QP) value associated with the current block. Video encoder 200 (e.g., via mode selection unit 202) may adjust the degree of quantization applied to the transform coefficient block associated with the current block by adjusting the QP value associated with the CU. Quantization may introduce information loss, and thus, the quantized transform coefficients may have lower precision than the original transform coefficients generated by transform processing unit 206.

[0113] Inverse quantization unit 210 and inverse transform processing unit 212 may apply inverse quantization and inverse transform, respectively, to the quantized transform coefficient block to reconstruct a residual block from the transform coefficient block. Reconstruction unit 214 may generate a reconstructed block corresponding to the current block (although there may be a certain degree of distortion) based on the reconstructed residual block and the prediction block generated by mode selection unit 202. For example, reconstruction unit 214 may add the samples of the reconstructed residual block to the corresponding samples of the prediction block generated by mode selection unit 202 to generate the reconstructed block.

[0114] Filter unit 216 may perform one or more filter operations on the reconstructed block. For example, filter unit 216 may perform a deblocking operation to reduce blocking artifacts along the edges of the CU. In some examples, the operation of filter unit 216 may be skipped.

[0115] When operating according to AV1, filter unit 216 may perform one or more filter operations on the reconstructed block. For example, filter unit 216 may perform a deblocking operation to reduce blocking artifacts along the edges of the CU. In other examples, filter unit 216 may apply a Constrained Directional Enhancement Filter (CDEF), which may be applied after deblocking and may include applying a non-separable non-linear low-pass directional filter based on an estimated edge direction. Filter unit 216 may also include a loop restoration filter applied after CDEF and may include a separable symmetric normalized Wiener filter or a bi-self-guiding filter.

[0116] Video encoder 200 stores the reconstructed block in DPB 218. For example, in an example where the operation of filter unit 216 is not performed, reconstruction unit 214 may store the reconstructed block into DPB 218. In an example where the operation of filter unit 216 is performed, filter unit 216 may store the filtered reconstructed block into DPB 218. Motion estimation unit 222 and motion compensation unit 224 may retrieve reference pictures formed by the reconstructed (and potentially filtered) blocks from DPB 218 for inter prediction of blocks in subsequent encoded pictures. Additionally, intra prediction unit 226 may use the reconstructed blocks in DPB 218 of the current picture for intra prediction of other blocks in the current picture.

[0117] Generally, the entropy coding unit 220 can perform entropy coding on syntax elements received from other functional components of the video encoder 200. For example, the entropy coding unit 220 can perform entropy coding on the quantized transform coefficient blocks from the quantization unit 208. As another example, the entropy coding unit 220 can perform entropy coding on the prediction syntax elements from the mode selection unit 202 (e.g., motion information for inter prediction or intra mode information for intra prediction). The entropy coding unit 220 can perform one or more entropy coding operations on the syntax elements (which are another example of video data) to generate entropy coded data. For example, the entropy coding unit 220 can perform context adaptive variable length coding (CAVLC) operations, CABAC operations, variable-to-variable (V2V) length coding operations, syntax-based context adaptive binary arithmetic coding (SBAC) operations, probability interval partitioning entropy (PIPE) coding operations, exponential Golomb coding operations, or another type of entropy coding operation on the data. In some examples, the entropy coding unit 220 can operate in a bypass mode, where the syntax elements are not entropy coded.

[0118] The video encoder 200 can output a bitstream that includes the entropy coded syntax elements required to reconstruct the blocks of a slice or picture. Specifically, the entropy coding unit 220 can output the bitstream.

[0119] According to AV1, the entropy coding unit 220 can be configured as a symbol-to-symbol adaptive multi-symbol arithmetic coder. The syntax elements in AV1 include an alphabet of N elements, and the context (e.g., probability model) includes a set of N probabilities. The entropy coding unit 220 can store the probabilities as an n-bit (e.g., 15-bit) cumulative distribution function (CDF). The entropy coding unit 220 performs recursive scaling with an update factor based on the alphabet size to update the context.

[0120] The operations described above are described in terms of blocks. Such a description should be understood as operations for a luminance coding block and / or a chrominance coding block. As described above, in some examples, the luminance coding block and the chrominance coding block are the luminance and chrominance components of a CU. In some examples, the luminance coding block and the chrominance coding block are the luminance and chrominance components of a PU.

[0121] In some examples, it is not necessary to repeat the operations performed on the luminance coding block for the chrominance coding block. As an example, there is no need to repeat the operations of identifying the motion vector (MV) and reference picture of the luminance coding block to identify the MV and reference picture of the chrominance block. In fact, the MV of the luminance coding block can be scaled to determine the MV of the chrominance block, and the reference picture can be the same. As another example, the intra prediction process can be the same for the luminance coding block and the chrominance coding block.

[0122] Video encoder 200 also includes an amplification filter unit 228. The amplification filter unit 228 represents a post-loop filter that can execute the techniques of the present disclosure to perform amplification filtering on decoded video data. Although depicted as part of the video encoder 200, in some examples, the amplification filter unit 228 can form part of a separate post-processing. Additionally, a preprocessing unit (not shown) can initially reduce the video data before storing the reduced video data in the video data memory 230.

[0123] Figure 4 is a block diagram illustrating an example video decoder 300 that can execute the techniques of the present disclosure. It is provided for purposes of explanation Figure 4 and Figure 4 does not limit the techniques widely illustrated and described in the present disclosure. For purposes of explanation, the present disclosure describes a video decoder 300 according to the techniques of VVC (ITU-T H.266, in development) and HEVC (ITU-T H.265). However, the techniques of the present disclosure can be executed by a video decoding device configured for other video coding standards.

[0124] In Figure 4 the example, video decoder 300 includes a coded picture buffer (CPB) memory 320, an entropy decoding unit 302, a prediction processing unit 304, an inverse quantization unit 306, an inverse transform processing unit 308, a reconstruction unit 310, a filter unit 312, a decoded picture buffer (DPB) 314, and an amplification filter unit 322. Any one or all of the CPB memory 320, entropy decoding unit 302, prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310, filter unit 312, DPB 314, and amplification filter unit 322 can be implemented in one or more processors or in processing circuitry. For example, the units of video decoder 300 can be implemented as one or more circuits or logic elements, as part of a hardware circuit, or as part of a processor, ASIC, or FPGA. Additionally, video decoder 300 can include additional or alternative processors or processing circuitry to perform these and other functions.

[0125] The prediction processing unit 304 includes a motion compensation unit 316 and an intra prediction unit 318. The prediction processing unit 304 can include additional units that perform prediction according to other prediction modes. As an example, the prediction processing unit 304 can include a palette unit, a block copy unit (which can form part of the motion compensation unit 316), an affine unit, a linear model (LM) unit, or the like. In other examples, video decoder 300 can include more, fewer, or different functional components.

[0126] When operating according to AV1, compensation unit 316 may be configured to decode coded blocks of video data (e.g., both luminance and chrominance coded blocks) using translational motion compensation, affine motion compensation, OBMC, and / or combined inter-intra prediction as described above. Intra prediction unit 318 may be configured to decode coded blocks of video data (e.g., both luminance and chrominance coded blocks) using directional intra prediction, non-directional intra prediction, recursive filter intra prediction, CFL, intra block copy (IBC), and / or palette mode as described above.

[0127] CPB memory 320 may store video data to be decoded by components of video decoder 300, such as an encoded video bitstream. The video data stored in CPB memory 320 may be obtained, for example, from computer-readable medium 110 ( Figure 1 ). CPB memory 320 may include a CPB that stores encoded video data (e.g., syntax elements) from the encoded video bitstream. Additionally, CPB memory 320 may store video data other than syntax elements of decoded pictures, such as temporary data representing outputs from various units of video decoder 300. DPB 314 generally stores decoded pictures that video decoder 300 may output and / or use as reference video data when decoding subsequent data or pictures in the encoded video bitstream. CPB memory 320 and DPB 314 may be formed from any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. CPB memory 320 and DPB 314 may be provided by the same memory device or separate memory devices. In various examples, CPB memory 320 may be on-chip with other components of video decoder 300 or off-chip relative to those components.

[0128] Additionally or alternatively, in some examples, video decoder 300 may retrieve decoded video data from memory 120 ( Figure 1 ). That is, memory 120 may store data as discussed above with respect to CPB memory 320. Similarly, when some or all of the functions of video decoder 300 are implemented in software to be executed by the processing circuitry of video decoder 300, memory 120 may store instructions to be executed by video decoder 300.

[0129] Figure 4 The various units shown in ( Figure 3, A fixed-function circuit refers to a circuit that provides specific functionality and is pre-set in terms of the operations it can perform. A programmable circuit refers to a circuit that can be programmed to perform various tasks and provides flexible functionality among the operations it can perform. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. A fixed-function circuit can execute software instructions (e.g., to receive parameters or output parameters), but the type of operations performed by the fixed-function circuit is generally immutable. In some examples, one or more of the units can be different circuit blocks (fixed-function or programmable), and in some examples, one or more of the units can be an integrated circuit.

[0130] Video decoder 300 may include an ALU, an EFU, digital circuits, analog circuits, and / or a programmable core formed by a programmable circuit. In an example where the operation of video decoder 300 is performed by software executed on the programmable circuit, on-chip or off-chip memory may store the instructions (e.g., object code) of the software that video decoder 300 receives and executes.

[0131] Entropy decoding unit 302 may receive encoded video data from the CPB and perform entropy decoding on the video data to reproduce syntax elements. Prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310, and filter unit 312 may generate decoded video data based on the syntax elements extracted from the bitstream.

[0132] Generally, video decoder 300 reconstructs pictures on a block-by-block basis. Video decoder 300 may perform the reconstruction operation on each block individually (where the block that is currently being reconstructed (i.e., decoded) may be referred to as the "current block").

[0133] Entropy decoding unit 302 may perform entropy decoding on the syntax elements that define the quantization transform coefficients of the quantization transform coefficient block and the transform information (such as quantization parameter (QP) and / or transform mode indication). Inverse quantization unit 306 may use the QP associated with the quantization transform coefficient block to determine the degree of quantization, and similarly, determine the degree of inverse quantization applied by inverse quantization unit 306. Inverse quantization unit 306 may, for example, perform a bitwise left shift operation to inverse-quantize the quantization transform coefficients. Inverse quantization unit 306 may thereby form a transform coefficient block including transform coefficients.

[0134] After inverse quantization unit 306 forms the transform coefficient block, inverse transform processing unit 308 may apply one or more inverse transforms to the transform coefficient block to generate a residual block associated with the current block. For example, inverse transform processing unit 308 may apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotation transform, an inverse orientation transform, or another inverse transform to the transform coefficient block.

[0135] In addition, the prediction processing unit 304 generates a prediction block based on the prediction information syntax element entropy decoded by the entropy decoding unit 302. For example, if the prediction information syntax element indicates that the current block is inter-frame predicted, the motion compensation unit 316 may generate a prediction block. In this case, the prediction information syntax element may indicate the reference picture in the DPB 314 from which the reference block is retrieved, and the motion vector identifying the position of the reference block in the reference picture relative to the current block in the current picture. The motion compensation unit 316 generally may perform the inter-frame prediction process in a manner similar to that described with respect to the motion compensation unit 224 ( Figure 3 ).

[0136] As another example, if the prediction information syntax element indicates that the current block is intra-frame predicted, the intra-frame prediction unit 318 may generate a prediction block according to the intra-frame prediction mode indicated by the prediction information syntax element. Again, the intra-frame prediction unit 318 generally may perform the intra-frame prediction process in a manner similar to that described with respect to the intra-frame prediction unit 226 ( Figure 3 ). The intra-frame prediction unit 318 may retrieve data of adjacent samples of the current block from the DPB 314.

[0137] The reconstruction unit 310 may use the prediction block and the residual block to reconstruct the current block. For example, the reconstruction unit 310 may add the samples of the residual block to the corresponding samples of the prediction block to reconstruct the current block.

[0138] The filter unit 312 may perform one or more filter operations on the reconstructed block. For example, the filter unit 312 may perform a deblocking operation to reduce blocking artifacts along the edges of the reconstructed block. The operation of the filter unit 312 is not necessarily performed in all examples.

[0139] The video decoder 300 may store the reconstructed block in the DPB 314. For example, in an example where the operation of the filter unit 312 is not performed, the reconstruction unit 310 may store the reconstructed block into the DPB 314. In an example where the operation of the filter unit 312 is performed, the filter unit 312 may store the filtered reconstructed block into the DPB 314. As discussed above, the DPB 314 may provide reference information (e.g., samples of the current picture for intra-frame prediction and samples of the previously decoded picture for subsequent motion compensation) to the prediction processing unit 304. In addition, the video decoder 300 may output the decoded picture (e.g., decoded video) from the DPB 314 for subsequent presentation on a display device (such as Figure 1 display device 118).

[0140] Video decoder 300 further includes an upsampling filter unit 322. The upsampling filter unit 322 may perform the techniques of the present disclosure, as discussed in more detail below, to horizontally upsample decoded video data. In some examples, the upsampling filter unit 322 may alternatively form part of a separate post-processing unit (not shown).

[0141] In this manner, video decoder 300 represents an example of an apparatus for upsampling filtering video data, the apparatus including: a memory configured to store video data; and one or more processors implemented in circuitry and configured to: determine a filter scaling value that represents half the number of support pixels to be used by an upsampling filter to upsample filter a video data block; determine an upsampling ratio for the video data block; calculate a size multiplier value as a product of the upsampling ratio and a difference between a size of the video data block and the filter scaling value; calculate a rounding value based on the size multiplier value such that the rounding value is the largest integer multiple of the filter scaling value that does not exceed the size multiplier value; generate, for each row of the video data block, upsampled pixels the number of which is equal to the rounding value, without generating upsampled pixels for the following number of pixels starting from a right edge of the video block, the number being equal to a difference between a product of the size of the block and the upsampling ratio and the rounding value; store values of the number of pixels starting from the right edge of the video block of the video data for each row of the video data block; and use the stored values of the number of pixels starting from the right edge of the video block to upsample filter pixels of a right adjacent video data block of the video data block.

[0142] Figure 5 is a block diagram illustrating a set of example components of upsampling filter unit 148. Figure 3 The upsampling filter unit 228 and / or Figure 4 The upsampling filter unit 322 may include components similar to those of upsampling filter unit 148. In this example, the components of upsampling filter unit 148 include a filter coefficient selection unit 150, filter coefficients 152, and a collection 156A - 156D of various filter kernels (filter kernel 156). Input pixels 154 are provided to filter kernel 156, and filter kernel 156 applies corresponding filter coefficients to form upsampled pixels 158. Filter kernel 156 may operate on input pixels 154 in parallel to generate upsampled pixels 158.

[0143] Figure 6A and 6B is a conceptual diagram illustrating an example of an input block of video data and a corresponding upsampled block generated by upsampling filtering. Specifically, Figure 6ADepicts an example of input block 160, which is a 64×64 pixel block and is filtered by an upsampling filter with an upsampling ratio of 1.875 to generate an upsampled block 162, which is a 120×64 pixel block. That is, only the first 60 pixels are upsampled and filtered, resulting in 120 upsampled and filtered pixels. Figure 6B Depicts an example of input block 164, which is a 64×64 pixel block and is filtered by an upsampling filter with an upsampling ratio of 1.375 to generate an upsampled block 166, which is an 83×64 pixel block. That is, only the first 60 pixels are upsampled and filtered, resulting in 83 upsampled and filtered pixels.

[0144] Figure 6A and 6B The example of corresponds to a conventional high-end architecture implementation using superblocks (SBs), where in AV1, a superblock can be 64×64 or 128×128. In this conventional method, all possible pixels in the current SB are upsampled and filtered, regardless of whether the width of the upsampled SB is a multiple of 4.

[0145] For a 64×64 pixel SB, 60 pixel columns of the current SB are used as the central pixels for upsampling and filtering. The last four pixels are not used because the right adjacent data may not be available for those pixels. Similarly, for a 128×128 pixel SB, only the first 124 pixel columns of the current SB are used as the central pixels for upsampling.

[0146] For upsampling ratios other than 2, this method may result in unaligned output generation (i.e., the output upsampled width is not aligned with 4 pixels). For a 4K width input frame with an input block of 64 pixels and an upsampling ratio of 1.375, only the first 60 pixels are filtered as central pixels, resulting in an output block with a sample width of 83, which is not a multiple of 4.

[0147] Since the upsampling hardware may include multiple cores (e.g., 16 cores that process all pixels of a 4×4 block in parallel), if the generated output is not aligned with a 4-pixel width, not all cores will be fully utilized, resulting in a performance loss.

[0148] If the frames (pictures) of the video data have each SB column as a tile, there may be misaligned output generation after magnifying the first SB (i.e., the output magnification width is not aligned with 4 pixels), which is not emitted on the output interface. In this case, the remaining pixels of the 4 × 4 block will be generated when processing the next SB. Due to this constraint, the 4×4 block generated using the SB boundary pixels will require twice the amount of clock cycles compared to other 4×4 blocks, resulting in performance degradation. For a 4K width-reduced frame with each SB column as a tile and a magnification ratio of 1.375, pixel columns 81 to 83 and pixel column 84 are generated in separate clock cycles, resulting in performance degradation.

[0149] Additional processing logic is also used to align the next SB output, which depends on the number of misaligned pixels (1, 2, or 3) in the previous SB. Therefore, in these conventional techniques, 7 to 10 rows will need to be stored between blocks anywhere. Relative to the technology of the present disclosure, the variable nature of the number of rows to be stored increases the logic requirements and reduces the performance.

[0150] Assume that each 4×4 block requires n cycles to be processed:

[0151] Reduced width, DWwidth

[0152] Magnification width, UPwidth = DWwidth UpscaleRatio

[0153] The number of SB blocks in the luminance frame width = NumSB = DWwidth / SBWidth

[0154] The number of SB block boundaries = (NumSB - 1) = (DWwidth / SBWidth - 1)

[0155] The number of 4×4 in the luminance frame height = FrameHeight / 4

[0156] The number of luminance 4×4 at the SB boundary = (NumSB - 1) FrameHeight / 4 = ((DWwidth UpscaleRatio) / SBWidth - 1) FrameHeight / 4

[0157] The number of 4×4 in the luminance frame width = UPwidth / 4 = (DWwidth UpscaleRatio) / 4

[0158] Total luminance 4×4 blocks = NumLuma = (DWwidth UpscaleRatio) / 4 FrameHeight / 4

[0159] Total cycles for luminance processing =

[0160] Total cycles for chrominance processing =

[0161] Total cycles for frame processing =

[0162] Figure 7 is a conceptual diagram illustrating an example of a conventional technique for performing upsampling filtering on a video data block. In this example, a 64×64 block includes 60 columns of available pixels 170, and the available pixels 170 can be upsampled via a filter with an upsampling ratio of 1.375 to form 38 upsampled pixels 176. For a 64-pixel block, the final goal is to form 88 upsampled pixels. Therefore, in this example, 5 columns of pixels that have not been upsampled will be retained. In order to form upsampled pixels for the pixels 180 that have not been upsampled, three columns of upsampled misaligned pixels 178, and three columns of original adjacent pixels 172 to the unused pixels 174, and the unused pixels 174 themselves will need to be saved, for a total of ten columns of saved pixels to be upsampled filtered, and the subsequent block 182.

[0163] Figure 8 is a conceptual diagram illustrating an example technique for performing upsampling filtering on a video data block according to the technology of the present disclosure. In this example, initially only those pixels required to form aligned upsampled pixels are used, rather than generating misaligned upsampled pixels (such as Figure 7 the misaligned pixels 178). That is, the number of upsampled pixels equal to a multiple of four (in this example, or more generally, half of the filter width) and not exceeding the width of the final upsampled block is formed. Therefore, for an upsampling filter with an upsampling ratio of 1.375, 80 columns of pixels can be generated from, for example, 58 central pixels of the original block. That is, the upsampled pixels 196 can include 80 columns of pixels generated from the available pixels 190. This results in 8 columns of pixels 198 that have not been upsampled, which can subsequently be generated from the adjacent pixels 192 and the unused pixels 194 and the subsequent block 199.

[0164] Therefore, according to the technology of the present disclosure, assuming that each 4×4 block is processed in n cycles:

[0165] Reduced width = DWwidth

[0166] Upsampled width = UPwidth = DWwidth UpscaleRatio

[0167] Number of SB blocks in the luminance frame width = NumSB = DWwidth / SBWidth

[0168] Number of SB block boundaries = (NumSB - 1) = (DWwidth / SBWidth - 1)

[0169] Number of 4x4 in the luminance frame height = FrameHeight / 4

[0170] Number of luminance 4x4 at the SB boundary =

[0171] Number of 4x4 in the luminance frame width =

[0172] Total luminance 4x4 blocks =

[0173] Total number of cycles for luminance processing =

[0174] Total number of cycles for chrominance processing =

[0175] Total number of cycles for frame processing =

[0176] Performance improvement =

[0177] Figure 9 is a flowchart illustrating an example method for encoding a current block according to the techniques of the present disclosure. The current block may include a current CU. Although described with respect to video encoder 200 ( Figure 1 and 3 ), it should be understood that other devices may be configured to perform methods similar to Figure 9 .

[0178] In this example, video encoder 200 initially predicts the current block (350). For example, video encoder 200 may form a predicted block of the current block. Then, video encoder 200 may calculate the residual block of the current block (352). To calculate the residual block, video encoder 200 may calculate the difference between the original uncoded block and the predicted block of the current block. Then, video encoder 200 may transform the residual block and quantize the transform coefficients of the residual block (354). Next, video encoder 200 may scan the quantized transform coefficients of the residual block (356). During or after the scan, video encoder 200 may perform entropy coding on the transform coefficients (358). For example, video encoder 200 may use CAVLC or CABAC to encode the transform coefficients. Then, video encoder 200 may output the entropy-coded data of the block (360).

[0179] Video encoder 200 may also decode the current block after encoding the current block to use the decoded version of the current block as reference data for subsequent encoded data (e.g., in an inter-frame or intra-frame prediction mode). Thus, video encoder 200 may inverse quantize and inverse transform the coefficients to reproduce the residual block (362). Video encoder 200 may combine the residual block with the prediction block to form the decoded block (364). Then, video encoder 200 may store the decoded block in DPB 218 (366). After decoding the block, video encoder 200 or a post-processing unit may perform the techniques of the present disclosure to perform upsampling filtering on the block.

[0180] Figure 10 is a flowchart illustrating an example method for decoding a current block of video data according to the techniques of the present disclosure. The current block may include a current CU. Although described with respect to video decoder 300 ( Figure 1 and 4 ), it should be understood that other devices may be configured to perform methods similar to Figure 10 .

[0181] Video decoder 300 may receive the entropy-coded data of the current block, such as the entropy-coded prediction information and entropy-coded data of the transform coefficients of the residual block corresponding to the current block (370). Video decoder 300 may entropy-decode the entropy-coded data to determine the prediction information of the current block and reproduce the transform coefficients of the residual block (372). Video decoder 300 may predict the current block, for example, using an intra-frame or inter-frame prediction mode indicated by the prediction information of the current block (374), to calculate the prediction block of the current block. Then, video decoder 300 may reverse-scan the reproduced transform coefficients (376) to create a quantized transform coefficient block. Then, video decoder 300 may inverse quantize the transform coefficients and apply an inverse transform to the transform coefficients to generate a residual block (378). Video decoder 300 may finally decode the current block by combining the prediction block and the residual block (380). After decoding the block, video decoder 300 or a post-processing unit may perform the techniques of the present disclosure to perform upsampling filtering on the block.

[0182] Figure 11 is a flowchart illustrating an example method for performing upsampling filtering on a block of video data according to the techniques of the present disclosure. The method regarding Figure 5 's upsampling filter unit 148 is explained Figure 11 . This method or a similar method may be performed by Figure 4 's upsampling filter unit 322, Figure 3 's upsampling filter unit 228, or a post-processing unit separate from video encoder 200 or video decoder 300.

[0183] Initially, the upscale filter unit 148 receives a decoded video data block. The upscale filter unit 148 determines a filter scaling value (400), which represents half of the number of support pixels to be used by the upscale filter to perform upscale filtering on the decoded video data block. Additionally or alternatively, the filter scaling value may correspond to a fixed unit number of pixels processed by the filter kernel 156, such as four pixels.

[0184] The upscale filter unit 148 may also determine an upscale ratio (402). The upscale ratio may be determined by configuration data provided to the upscale filter unit 148. In some examples, the configuration data may be provided by a user. In some examples, the configuration data may be provided in a bitstream including encoded video data, such as in a supplementary enhancement information (SEI) message or other high-level signaling data. The upscale ratio may be determined from a set of available upscale ratios, such as 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2.

[0185] The upscale filter unit 148 may then calculate a size multiplier value (404). The size multiplier value may correspond to the product of the upscale ratio and the difference between the size of the block and the filter scaling value. That is, if the upscale ratio is R, the size of the block is S, and the filter scaling value is FS, the multiplier value may be calculated according to R (S - FS). Thus, for example, if the upscale ratio is 1.375, the block size is 64, and the filter scaling value is 4, the multiplier value will be 1.375 60 = 83.

[0186] The upscale filter unit 148 may further calculate a rounding value (406) from the size multiplier value. The rounding value may be the largest possible integer multiple of the filter scaling value that does not exceed the size multiplier value. For example, if the filter scaling value is 4 and the multiplier value is 83, the rounding value will be 80.

[0187] The upscale filter unit 148 may then generate upscale pixels for each row of the block, the number of which is equal to the rounding value (408). Specifically, the upscale filter unit 148 may generate only the above-mentioned number of upscale pixels for the block at this time (before the right adjacent block has been decoded), and not generate upscale pixels for the following number of pixels starting from the right edge of the block, the number of which is equal to the difference between the product of the size of the block and the upscale ratio and the rounding value. Thus, if the size of the block is 64, the upscale ratio is 1.375, and the rounding value is 80, then the number of upscale pixels that will not be generated will be equal to 8 ((64 1.375) - 80).

[0188] The upsampling filter unit 148 can then store the values of the adjacent pixels of the unused pixels and the unused pixels themselves (410). For example, if the unused pixels include seven columns of pixels and the adjacent pixels include three columns of pixels (assuming an 8-tap filter), then ten columns of pixels can be stored for each row. After the next block (i.e., the right adjacent block of the current block) has been decoded, the upsampling filter unit 148 can use the same method to upsample the unused pixels and the pixels of the next block (412).

[0189] In this way, Figure 11 The method of represents an example of a method for upsampling filtering video data, which includes: determining a filter scaling value, the filter scaling value representing half of the number of support pixels to be used by an upsampling filter to upsample filter a video data block; determining an upsampling ratio of the video data block; calculating a size multiplication value as a product of the difference between the and the filter scaling value; calculating a rounding value according to the size multiplication value such that the rounding value is the largest integer multiple of the filter scaling value not exceeding the size multiplication value; for each row of the video data block, generating upsampled pixels the number of which is equal to the rounding value, and not generating upsampled pixels for the following number of pixels starting from the right edge of the video block, the number being equal to the difference between the product of the size of the block and the upsampling ratio and the rounding value; for each row of the video data block, storing the values of the number of pixels starting from the right edge of the video block of the video data; and using the values of the number of pixels starting from the right edge of the video block to upsample filter the pixels of the right adjacent video data block of the video data block.

[0190] Certain examples of the technology of the present disclosure are outlined in the following clauses:

[0191] Clause 1: A method for performing upsampling filtering on video data, the method comprising: determining a filter scaling value, the filter scaling value representing half of the number of support pixels to be used by an upsampling filter to perform upsampling filtering on a video data block; determining an upsampling ratio of the video data block; calculating a size multiplier value as a product of the upsampling ratio and a difference between the size of the video data block and the filter scaling value; calculating a rounding value based on the size multiplier value such that the rounding value is the largest integer multiple of the filter scaling value not exceeding the size multiplier value; for each row of the video data block, generating upsampled pixels the number of which is equal to the rounding value, without generating upsampled pixels for the following number of pixels starting from the right edge of the video block, the number being equal to a difference between a product of the size of the block and the upsampling ratio and the rounding value; for each row of the video data block, storing values of the number of pixels starting from the right edge of the video block of the video data; and using the values of the number of pixels starting from the right edge of the video block stored, to perform upsampling filtering on pixels of a right adjacent video data block of the video data block.

[0192] Clause 2: The method according to Clause 1, wherein the filter scaling value is the value 4.

[0193] Clause 3: The method according to Clause 1, wherein the upsampling ratio is selected from a set of upsampling ratios comprising 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2.

[0194] Clause 4: The method according to Clause 1, wherein the size of the video data block is one of 64 pixels or 128 pixels.

[0195] Clause 5: The method according to Clause 1, wherein the rounding value is one of 64, 72, 80, 88, 96, 104, 112, or 120.

[0196] Clause 6: The method according to Clause 1, wherein the number of pixels starting from the right edge of the video block is 1 less than the filter scaling value.

[0197] Clause 7: The method according to Clause 1, wherein: the number of support pixels is 8, the filter scaling value is 4, the magnification ratio is selected from the set of magnification ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2, when the magnification ratio is 1.125, the rounding value is 64, when the magnification ratio is 1.25, the rounding value is 72, when the magnification ratio is 1.375, the rounding value is 80, when the magnification ratio is 1.5, the rounding value is 88, when the magnification ratio is 1.625, the rounding value is 96, when the magnification ratio is 1.75, the rounding value is 104, when the magnification ratio is 1.875, the rounding value is 112, or when the magnification ratio is 2, the rounding value is 120, and wherein the number of pixels starting from the right edge of the video block is 3.

[0198] Clause 8: The method according to Clause 1, further comprising decoding the coded block to form the video data block.

[0199] Clause 9: The method according to Clause 8, further comprising decoding the coded version of the right adjacent block after generating the number of magnified pixels for each row of the block.

[0200] Clause 10: The method according to Clause 1, further comprising: encoding the original block to form a coded block; and decoding the coded block to form the video data block.

[0201] Clause 11: An apparatus for performing upsampling filtering on video data, the apparatus comprising: a memory configured to store video data; and one or more processors implemented in circuitry and configured to: determine a filter scaling value that represents half the number of support pixels to be used by an upsampling filter to perform upsampling filtering on a video data block; determine an upsampling ratio of the video data block; calculate a size multiplier value as a product of the upsampling ratio and a difference between a size of the video data block and the filter scaling value; calculate a rounding value based on the size multiplier value such that the rounding value is the largest integer multiple of the filter scaling value that does not exceed the size multiplier value; generate, for each row of the video data block, upsampled pixels the number of which is equal to the rounding value, without generating upsampled pixels for the following number of pixels starting from a right edge of the video block, the number being equal to a difference between a product of a size of the block and the upsampling ratio and the rounding value; store values of the number of pixels starting from the right edge of the video block of the video data for each row of the video data block; and perform upsampling filtering on pixels of a right adjacent video data block of the video data block using the stored values of the number of pixels starting from the right edge of the video block.

[0202] Clause 12: The apparatus according to clause 11, wherein the filter scaling value is the value 4.

[0203] Clause 13: The apparatus according to clause 11, wherein the upsampling ratio is selected from a set of upsampling ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2.

[0204] Clause 14: The apparatus according to clause 11, wherein the size of the video data block is one of 64 pixels or 128 pixels.

[0205] Clause 15: The apparatus of clause 11, wherein the rounding value is one of 64, 72, 80, 88, 96, 104, 112, or 120.

[0206] Clause 16: The apparatus according to clause 11, wherein the number of pixels starting from the right edge of the video block is 1 less than the filter scaling value.

[0207] Clause 17: The apparatus according to Clause 11, wherein: the number of supported pixels is 8, the filter scaling value is 4, the size of the video data block is 64, the magnification ratio is selected from the set of magnification ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2, when the magnification ratio is 1.125, the rounding value is 64, when the magnification ratio is 1.25, the rounding value is 72, when the magnification ratio is 1.375, the rounding value is 80, when the magnification ratio is 1.5, the rounding value is 88, when the magnification ratio is 1.625, the rounding value is 96, when the magnification ratio is 1.75, the rounding value is 104, when the magnification ratio is 1.875, the rounding value is 112, or when the magnification ratio is 2, the rounding value is 120, and wherein the number of pixels starting from the right edge of the video block is 3.

[0208] Clause 18: The apparatus according to Clause 11, wherein the one or more processors are further configured to decode an encoded block to form the video data block.

[0209] Clause 19: The apparatus according to Clause 18, wherein the one or more processors are further configured to decode an encoded version of a right adjacent block after generating the number of magnified pixels for each row of the block.

[0210] Clause 20: The apparatus according to Clause 11, wherein the one or more processors are further configured to: encode an original block to form an encoded block; and decode the encoded block to form the video data block.

[0211] Clause 21: The apparatus according to Clause 11, wherein the apparatus includes one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

[0212] Clause 22: A computer-readable storage medium storing instructions which, when executed, cause a processor to: determine a filter scaling value, which represents half the number of support pixels to be used by an upsampling filter to perform upsampling filtering on a video data block; determine an upsampling ratio of the video data block; calculate a size multiplier value as a product of the upsampling ratio and a difference between a size of the video data block and the filter scaling value; calculate a rounding value based on the size multiplier value such that the rounding value is the largest integer multiple of the filter scaling value not exceeding the size multiplier value; generate, for each row of the video data block, upsampled pixels the number of which is equal to the rounding value, without generating upsampled pixels for the following number of pixels starting from a right edge of the video block, the number being equal to a difference between a product of a size of the block and the upsampling ratio and the rounding value; store values of the number of pixels starting from the right edge of the video block of the video data for each row of the video data block; and use the values of the number of pixels starting from the right edge of the video block stored, to perform upsampling filtering on pixels of a right adjacent video data block of the video data block.

[0213] Clause 23: The computer-readable storage medium according to Clause 22, wherein the filter scaling value is the value 4.

[0214] Clause 24: The computer-readable storage medium according to Clause 22, wherein the upsampling ratio is selected from a set of upsampling ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2.

[0215] Clause 25: The computer-readable storage medium according to Clause 22, wherein the size of the video data block is one of 64 pixels or 128 pixels.

[0216] Clause 26: The computer-readable storage medium according to Clause 22, wherein the rounding value is one of 64, 72, 80, 88, 96, 104, 112, or 120.

[0217] Clause 27: The computer-readable storage medium according to Clause 22, wherein the number of pixels starting from the right edge of the video block is 1 less than the filter scaling value.

[0218] Clause 28: The computer-readable storage medium according to Clause 22, wherein: the number of support pixels is 8, the filter scaling value is 4, the size of the video data block is 64, the magnification ratio is selected from the set of magnification ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2, when the magnification ratio is 1.125, the rounding value is 64, when the magnification ratio is 1.25, the rounding value is 72, when the magnification ratio is 1.375, the rounding value is 80, when the magnification ratio is 1.5, the rounding value is 88, when the magnification ratio is 1.625, the rounding value is 96, when the magnification ratio is 1.75, the rounding value is 104, when the magnification ratio is 1.875, the rounding value is 112, or when the magnification ratio is 2, the rounding value is 120, and wherein the number of pixels starting from the right edge of the video block is 3.

[0219] Clause 29: The computer-readable storage medium according to Clause 22, further comprising instructions for causing the processor to decode the coded block to form the video data block.

[0220] Clause 30: The computer-readable storage medium according to Clause 22, wherein the one or more processors are further configured to: encode an original block to form a coded block; and decode the coded block to form the video data block.

[0221] Clause 31: An apparatus for magnifying and filtering video data, the apparatus comprising: means for determining a filter scaling value, the filter scaling value representing half of the number of support pixels to be used by a magnification filter to magnify and filter a video data block; means for determining a magnification ratio of the video data block; means for calculating a size multiplier value as a product of the magnification ratio and a difference between the size of the video data block and the filter scaling value; means for calculating a rounding value based on the size multiplier value such that the rounding value is the largest integer multiple of the filter scaling value not exceeding the size multiplier value; means for generating, for each row of the video data block, magnified pixels the number of which is equal to the rounding value, without generating magnified pixels for the following number of pixels starting from the right edge of the video block, the number being equal to a difference between a product of the size of the block and the magnification ratio and the rounding value; means for storing, for each row of the video data block, values of the number of pixels starting from the right edge of the video block of the video data; and means for magnifying and filtering pixels of a right adjacent video data block of the video data block using the stored values of the number of pixels starting from the right edge of the video block.

[0222] Clause 32: A method for performing upsampling filtering on video data, the method comprising: determining a filter scaling value, the filter scaling value representing half of the number of support pixels to be used by an upsampling filter to perform upsampling filtering on a video data block; determining an upsampling ratio of the video data block; calculating a size multiplier value as a product of the upsampling ratio and a difference between the size of the video data block and the filter scaling value; calculating a rounding value based on the size multiplier value such that the rounding value is the largest integer multiple of the filter scaling value that does not exceed the size multiplier value; for each row of the video data block, generating upsampled pixels the number of which is equal to the rounding value, without generating upsampled pixels for the following number of pixels starting from the right edge of the video block, the number being equal to a difference between a product of the size of the block and the upsampling ratio and the rounding value; for each row of the video data block, storing values of the number of pixels starting from the right edge of the video block of the video data; and using the values of the number of pixels starting from the right edge of the video block, performing upsampling filtering on pixels of a right adjacent video data block of the video data block.

[0223] Clause 33: The method according to clause 32, wherein the filter scaling value is the value 4.

[0224] Clause 34: The method according to any one of clauses 32 and 33, wherein the upsampling ratio is selected from a set of upsampling ratios comprising 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2.

[0225] Clause 35: The method according to any one of clauses 32 to 34, wherein the size of the video data block is one of 64 pixels or 128 pixels.

[0226] Clause 36: The method according to any one of clauses 32 - 35, wherein the rounding value is one of 64, 72, 80, 88, 96, 104, 112, or 120.

[0227] Clause 37: The method according to any one of clauses 32 - 36, wherein the number of pixels starting from the right edge of the video block is 1 less than the filter scaling value.

[0228] Clause 38: The method according to Clause 1, wherein: the number of the support pixels is 8, the filter scaling value is 4, the size of the video data block is 64, the magnification ratio is selected from the set of magnification ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2, when the magnification ratio is 1.125, the rounding value is 64, when the magnification ratio is 1.25, the rounding value is 72, when the magnification ratio is 1.375, the rounding value is 80, when the magnification ratio is 1.5, the rounding value is 88, when the magnification ratio is 1.625, the rounding value is 96, when the magnification ratio is 1.75, the rounding value is 104, when the magnification ratio is 1.875, the rounding value is 112, or when the magnification ratio is 2, the rounding value is 120, and wherein the number of pixels starting from the right edge of the video block is 3.

[0229] Clause 39: The method according to any one of Clauses 32 - 38, further comprising decoding the coded block to form the video data block.

[0230] Clause 40: The method according to Clause 39, further comprising decoding the coded version of the right - adjacent block after generating the number of magnified pixels for each row of the block.

[0231] Clause 41: The method according to any one of Clauses 32 - 40, further comprising: encoding the original block to form a coded block; and decoding the coded block to form the video data block.

[0232] Clause 42: An apparatus for performing upsampling filtering on video data, the apparatus comprising: a memory configured to store video data; and one or more processors implemented in circuitry and configured to: determine a filter scaling value, the filter scaling value representing half of the number of support pixels to be used by an upsampling filter to perform upsampling filtering on a video data block; determine an upsampling ratio of the video data block; calculate a size multiple value as a product of the upsampling ratio and a difference between a size of the video data block and the filter scaling value; calculate a rounding value based on the size multiple value such that the rounding value is the largest integer multiple of the filter scaling value that does not exceed the size multiple value; generate, for each row of the video data block, upsampled pixels the number of which is equal to the rounding value, without generating upsampled pixels for the following number of pixels starting from a right edge of the video block, the number being equal to a difference between a product of the size of the block and the upsampling ratio and the rounding value; store values of the number of pixels starting from the right edge of the video block of the video data for each row of the video data block; and perform upsampling filtering on pixels of a right adjacent video data block of the video data block using the stored values of the number of pixels starting from the right edge of the video block.

[0233] Clause 43: The apparatus according to clause 42, wherein the filter scaling value is the value 4.

[0234] Clause 44: The apparatus according to any one of clauses 42 and 43, wherein the upsampling ratio is selected from a set of upsampling ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2.

[0235] Clause 45: The apparatus according to any one of clauses 42 to 44, wherein the size of the video data block is one of 64 pixels or 128 pixels.

[0236] Clause 46: The apparatus according to any one of clauses 42 - 45, wherein the rounding value is one of 64, 72, 80, 88, 96, 104, 112, or 120.

[0237] Clause 47: The apparatus according to any one of clauses 42 - 46, wherein the number of pixels starting from the right edge of the video block is 1 less than the filter scaling value.

[0238] Clause 48: The apparatus according to Clause 11, wherein: the number of support pixels is 8, the filter scaling value is 4, the size of the video data block is 64, the magnification ratio is selected from the set of magnification ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2, when the magnification ratio is 1.125, the rounding value is 64, when the magnification ratio is 1.25, the rounding value is 72, when the magnification ratio is 1.375, the rounding value is 80, when the magnification ratio is 1.5, the rounding value is 88, when the magnification ratio is 1.625, the rounding value is 96, when the magnification ratio is 1.75, the rounding value is 104, when the magnification ratio is 1.875, the rounding value is 112, or when the magnification ratio is 2, the rounding value is 120, and wherein the number of pixels starting from the right edge of the video block is 3.

[0239] Clause 49: The apparatus according to any one of Clauses 42 - 48, wherein the one or more processors are further configured to decode an encoded block to form the video data block.

[0240] Clause 50: The apparatus according to Clause 49, wherein the one or more processors are further configured to decode an encoded version of a right - adjacent block after generating the number of magnified pixels for each row of the block.

[0241] Clause 51: The apparatus according to any one of Clauses 42 - 50, wherein the one or more processors are further configured to: encode an original block to form an encoded block; and decode the encoded block to form the video data block.

[0242] Clause 52: The apparatus according to any one of Clauses 42 - 51, wherein the apparatus includes one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set - top box.

[0243] Clause 53: A computer-readable storage medium storing instructions which, when executed, cause a processor to: determine a filter scaling value, the filter scaling value representing half the number of support pixels to be used by an upsampling filter to perform upsampling filtering on a video data block; determine an upsampling ratio of the video data block; calculate a size multiplier value as a product of the upsampling ratio and a difference between the size of the video data block and the filter scaling value; calculate a rounding value based on the size multiplier value such that the rounding value is the largest integer multiple of the filter scaling value not exceeding the size multiplier value; generate, for each row of the video data block, upsampled pixels the number of which is equal to the rounding value, without generating upsampled pixels for the following number of pixels starting from the right edge of the video block, the number being equal to a difference between a product of the size of the block and the upsampling ratio and the rounding value; store values of the number of pixels starting from the right edge of the video block of the video data; and use the values of the number of pixels starting from the right edge of the video block to perform upsampling filtering on pixels of a right-adjacent video data block of the video data block.

[0244] Clause 54: The computer-readable storage medium according to Clause 53, wherein the filter scaling value is the value 4.

[0245] Clause 55: The computer-readable storage medium according to any one of Clauses 53 and 54, wherein the upsampling ratio is selected from a set of upsampling ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2.

[0246] Clause 56: The computer-readable storage medium according to any one of Clauses 53-55, wherein the size of the video data block is one of 64 pixels or 128 pixels.

[0247] Clause 57: The computer-readable storage medium according to any one of Clauses 53-56, wherein the rounding value is one of 64, 72, 80, 88, 96, 104, 112, or 120.

[0248] Clause 58: The computer-readable storage medium according to any one of Clauses 53-57, wherein the number of pixels starting from the right edge of the video block is 1 less than the filter scaling value.

[0249] Clause 59: The computer-readable storage medium according to Clause 53, wherein: the number of support pixels is 8, the filter scaling value is 4, the size of the video data block is 64, the magnification ratio is selected from the set of magnification ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2, when the magnification ratio is 1.125, the rounding value is 64, when the magnification ratio is 1.25, the rounding value is 72, when the magnification ratio is 1.375, the rounding value is 80, when the magnification ratio is 1.5, the rounding value is 88, when the magnification ratio is 1.625, the rounding value is 96, when the magnification ratio is 1.75, the rounding value is 104, when the magnification ratio is 1.875, the rounding value is 112, or when the magnification ratio is 2, the rounding value is 120, and wherein the number of pixels starting from the right edge of the video block is 3.

[0250] Clause 60: The computer-readable storage medium according to any one of Clauses 53-59, further comprising instructions for causing the processor to decode the coded block to form the video data block.

[0251] Clause 61: The computer-readable storage medium according to any one of Clauses 53-60, wherein the one or more processors are further configured to: encode an original block to form a coded block; and decode the coded block to form the video data block.

[0252] Clause 62: An apparatus for performing magnification filtering on video data, the apparatus comprising: means for determining a filter scaling value, the filter scaling value representing half of the number of support pixels to be used by a magnification filter to perform magnification filtering on a video data block; means for determining a magnification ratio of the video data block; means for calculating a size multiplier value as a product of the magnification ratio and a difference between the size of the video data block and the filter scaling value; means for calculating a rounding value based on the size multiplier value such that the rounding value is the largest integer multiple of the filter scaling value not exceeding the size multiplier value; means for generating, for each row of the video data block, a number of magnified pixels equal to the rounding value, without generating magnified pixels for the following number of pixels starting from the right edge of the video block, the number being equal to a difference between a product of the size of the block and the magnification ratio and the rounding value; means for storing, for each row of the video data block, values of the number of pixels starting from the right edge of the video block of the video data; and means for performing magnification filtering on pixels of a right adjacent video data block of the video data block using the stored values of the number of pixels starting from the right edge of the video block.

[0253] It should be recognized that depending on the example, certain actions or events of any of the techniques described herein may be performed in a different order, may be added, combined, or entirely omitted (e.g., not all the described actions or events are required to practice the described techniques). Additionally, in certain examples, actions or events may be performed concurrently rather than sequentially, such as by multithreading, interrupt processing, or multiple processors.

[0254] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored on or transmitted via a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium including any medium that facilitates transfer of a computer program from one place to another, such as according to a communication protocol. In this manner, a computer-readable medium generally may correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementations of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0255] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but rather are actually directed to non-transitory tangible storage media. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0256] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, as used herein, the terms "processor" and "processing circuitry" may refer to any of the foregoing structures or any other structure suitable for implementation in the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated in a combined codec. Further, these techniques may be fully implemented in one or more circuits or logic elements.

[0257] The techniques of the present disclosure may be implemented in a wide variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs) or IC collections (e.g., chip sets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but need not be implemented by different hardware units. Rather, as described above, various units may be combined in a codec hardware unit with appropriate software and / or firmware, or provided by a collection of interoperating hardware units, including one or more processors as described above.

[0258] Various examples have been described. These and other examples are within the scope of the appended claims.

Claims

1. A method for performing upsampling filtering on video data, the method comprising: Determining a filter scaling value, the filter scaling value representing half of the number of support pixels to be used by an upsampling filter to perform upsampling filtering on a video data block; Determining an upsampling ratio of the video data block; Calculating a size multiplier value as a product of the upsampling ratio and a difference between the size of the video data block and the filter scaling value; Calculating a rounding value according to the size multiplier value such that the rounding value is the largest integer multiple of the filter scaling value not exceeding the size multiplier value; For each row of the video data block, generating upsampled pixels the number of which is equal to the rounding value, without generating upsampled pixels for the following number of pixels starting from the right edge of the video block, the number being equal to a difference between a product of the size of the block and the upsampling ratio and the rounding value; For each row of the video data block, storing values of the number of pixels starting from the right edge of the video block of the video data; And Using the values of the number of pixels starting from the right edge of the video block that are stored, performing upsampling filtering on pixels of a right adjacent video data block of the video data block.

2. The method according to claim 1, further comprising: Storing unprocessed values of pixels of a block to the left of the number of pixels starting from the right edge of the video block; And Using the unprocessed values of the pixels of the block that are stored, performing upsampling filtering on the number of pixels starting from the right edge of the video block.

3. The method according to claim 1, wherein The filter scaling value is the value 4.

4. The method according to claim 1, wherein the upsampling ratio is selected from a set of upsampling ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2.

5. The method according to claim 1, wherein the size of the video data block is one of 64 pixels or 128 pixels.

6. The method according to claim 1, wherein the rounding value is one of 64, 72, 80, 88, 96, 104, 112, or 120.

7. The method according to claim 1, wherein, The number of pixels starting from the right edge of the video block is 1 less than the filter scaling value.

8. The method according to claim 1, wherein: The number of support pixels is 8, The filter scaling value is 4, The size of the video data block is 64, The upsampling ratio is selected from a set of upsampling ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2, When the upsampling ratio is 1.125, the rounding value is 64, When the upsampling ratio is 1.25, the rounding value is 72, When the upsampling ratio is 1.375, the rounding value is 80, When the upsampling ratio is 1.5, the rounding value is 88, When the upsampling ratio is 1.625, the rounding value is 96, When the upsampling ratio is 1.75, the rounding value is 104, When the upsampling ratio is 1.875, the rounding value is 112, or When the magnification ratio is 2, the rounding value is 120, and wherein the number of pixels starting from the right edge of the video block is 3.

9. The method according to claim 1, further comprising decoding an encoded block to form the video data block.

10. The method according to claim 9, further comprising, after generating the number of magnified pixels for each row of the block, decoding an encoded version of a right adjacent block.

11. The method according to claim 1, further comprising: encoding an original block to form an encoded block; and decoding the encoded block to form the video data block.

12. An apparatus for performing upsampling filtering on video data, the apparatus comprising: a memory configured to store video data; and one or more processors implemented in circuitry and configured to: determine a filter scaling value that represents half of the number of support pixels to be used by an upsampling filter to perform upsampling filtering on a video data block; determine an upsampling ratio of the video data block; calculate a size multiplication value as a product of the upsampling ratio and a difference between a size of the video data block and the filter scaling value; calculate a rounding value according to the size multiplication value such that the rounding value is the largest integer multiple of the filter scaling value not exceeding the size multiplication value; generate, for each row of the video data block, an upsampled pixel whose number is equal to the rounding value, without generating upsampled pixels for the following number of pixels starting from the right edge of the video block, the number being equal to a difference between a product of a size of the block and the upsampling ratio and the rounding value; store, for each row of the video data block, values of the number of pixels starting from the right edge of the video block of the video data; and perform upsampling filtering on pixels of a right adjacent video data block of the video data block using the stored values of the number of pixels starting from the right edge of the video block.

13. The apparatus according to claim 12, wherein the one or more processors are further configured to: store unprocessed values of pixels of a block to the left of the number of pixels starting from the right edge of the video block; and perform upsampling filtering on the number of pixels starting from the right edge of the video block using the stored unprocessed values of the pixels of the block.

14. The apparatus according to claim 12, wherein the filter scaling value is the value 4.

15. The apparatus according to claim 12, wherein the upsampling ratio is selected from a set of upsampling ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2.

16. The apparatus according to claim 12, wherein the size of the video data block is one of 64 pixels or 128 pixels.

17. The apparatus according to claim 12, wherein the rounding value is one of 64, 72, 80, 88, 96, 104, 112, or 120.

18. The apparatus according to claim 12, wherein the number of pixels starting from the right edge of the video block is 1 less than the filter scaling value.

19. The apparatus according to claim 12, wherein: the number of support pixels is 8, the filter scaling value is 4, the size of the video data block is 64, the magnification ratio is selected from the set of magnification ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2, when the magnification ratio is 1.125, the rounding value is 64, when the magnification ratio is 1.25, the rounding value is 72, when the magnification ratio is 1.375, the rounding value is 80, when the magnification ratio is 1.5, the rounding value is 88, when the magnification ratio is 1.625, the rounding value is 96, when the magnification ratio is 1.75, the rounding value is 104, when the magnification ratio is 1.875, the rounding value is 112, or when the magnification ratio is 2, the rounding value is 120, and wherein the number of pixels starting from the right edge of the video block is 3.

20. The apparatus according to claim 12, wherein the one or more processors are further configured to decode an encoded block to form the video data block.

21. The apparatus according to claim 20, wherein the one or more processors are further configured to decode an encoded version of a right adjacent block after generating the number of magnified pixels for each row of the block.

22. The apparatus according to claim 12, wherein the one or more processors are further configured to: encode an original block to form an encoded block; and decode the encoded block to form the video data block.

23. The apparatus according to claim 12, wherein the apparatus comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

24. A computer-readable storage medium having instructions stored thereon, which when executed cause a processor to: determine a filter scaling value, the filter scaling value representing half of the number of support pixels to be used by a magnification filter to magnify-filter a video data block; determine a magnification ratio of the video data block; calculate a size multiplier value as a product of the magnification ratio and a difference between the size of the video data block and the filter scaling value; calculate a rounding value based on the size multiplier value such that the rounding value is the largest integer multiple of the filter scaling value not exceeding the size multiplier value; generate, for each row of the video data block, a number of magnified pixels equal to the rounding value, without generating magnified pixels for the following number of pixels starting from the right edge of the video block, the number being equal to a difference between a product of the size of the block and the magnification ratio and the rounding value; store, for each row of the video data block, values of the number of pixels starting from the right edge of the video block of the video data; and Using the values of the number of pixels starting from the right edge of the video block stored, perform upsampling filtering on the pixels of the right adjacent video data block of the video data block.

25. The computer-readable storage medium according to claim 24, further comprising instructions for causing the processor to perform the following operations: Store the unprocessed values of the pixels of the block to the left of the number of pixels starting from the right edge of the video block; and Using the unprocessed values of the pixels of the block stored, perform upsampling filtering on the number of pixels starting from the right edge of the video block.

26. The computer-readable storage medium according to claim 24, wherein the filter scaling value is the value 4.

27. The computer-readable storage medium according to claim 24, wherein the upsampling ratio is selected from the set of upsampling ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2.

28. The computer-readable storage medium according to claim 24, wherein the size of the video data block is one of 64 pixels or 128 pixels.

29. The computer-readable storage medium according to claim 24, wherein the rounding value is one of 64, 72, 80, 88, 96, 104, 112, or 120.

30. The computer-readable storage medium according to claim 24, wherein the number of pixels starting from the right edge of the video block is 1 less than the filter scaling value.

31. The computer-readable storage medium according to claim 24, wherein: the number of support pixels is 8, the filter scaling value is 4, the size of the video data block is 64, the upsampling ratio is selected from the set of upsampling ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2, when the upsampling ratio is 1.125, the rounding value is 64, when the upsampling ratio is 1.25, the rounding value is 72, when the upsampling ratio is 1.375, the rounding value is 80, when the upsampling ratio is 1.5, the rounding value is 88, when the upsampling ratio is 1.625, the rounding value is 96, when the upsampling ratio is 1.75, the rounding value is 104, when the upsampling ratio is 1.875, the rounding value is 112, or when the upsampling ratio is 2, the rounding value is 120, and wherein the number of pixels starting from the right edge of the video block is 3.

32. The computer-readable storage medium according to claim 24, further comprising instructions for causing the processor to decode an encoded block to form the video data block.

33. The computer-readable storage medium according to claim 24, wherein the one or more processors are further configured to: Encode an original block to form an encoded block; and Decode the encoded block to form the video data block.

34. An apparatus for performing upsampling filtering on video data, the apparatus comprising: A component for determining a filter scaling value, which represents half of the number of support pixels to be used by an upsampling filter to perform upsampling filtering on a video data block; A component for determining an upsampling ratio of the video data block; A component for calculating a size multiplication value as a product of the upsampling ratio and a difference between a size of the video data block and the filter scaling value; A component for calculating a rounding value according to the size multiplication value such that the rounding value is the largest integer multiple of the filter scaling value not exceeding the size multiplication value; A component for generating, for each row of the video data block, upsampled pixels the number of which is equal to the rounding value, without generating upsampled pixels for the following number of pixels starting from a right edge of the video block, the number being equal to a difference between a product of a size of the block and the upsampling ratio and the rounding value; A component for storing, for each row of the video data block, values of the number of pixels starting from the right edge of the video block of the video data; And A component for performing upsampling filtering on pixels of a right adjacent video data block of the video data block by using the stored values of the number of pixels starting from the right edge of the video block.

Images