Adaptive Resolution Management Predictive Rescaling

Abstract

To provide adaptive resolution management predictive rescaling.SOLUTION: A method includes receiving a reference frame, determining a scaling constant for a current block, determining a scaled reference block using the reference frame and the scaling constant, determining a scaled prediction block using the scaled reference block, and reconstructing pixel data of the current block using the rescaled prediction block. Related apparatuses, systems, techniques, and articles are also described.SELECTED DRAWING: Figure 2

Description

TECHNICAL FIELD

[0001] (Cross-Reference to Related Applications) This application claims the benefit of priority of U.S. Provisional Patent Application No. 62 / 883,454, filed on August 6, 2019 and entitled "ADAPTIVE RESOLUTION MANAGEMENT PREDICTION RESCALING", which is hereby incorporated herein by reference in its entirety.

[0002] The present invention relates generally to the field of video compression. In particular, the present invention is directed to adaptive resolution management prediction rescaling.

BACKGROUND ART

[0003] Video codecs may include electronic circuits or software that compress or decompress digital video. This may convert uncompressed video into a compressed format and vice versa. In the context of video compression, a device that compresses video and / or performs certain functions thereof may typically be referred to as an encoder, and a device that decompresses video and / or performs certain functions thereof may be referred to as a decoder.

[0004] The format of the compressed data may conform to standard video compression specifications. Compression may be lossy in that the compressed video may lack certain information present in the original video. This result may include that the decompressed video may have lower quality than the original uncompressed video, as there may be insufficient information to accurately reconstruct the original video.

[0005] There may be a complex relationship between video quality and the amount of data used to represent the video, as determined for example by bitrate, the complexity of encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access, end-to-end delay such as but not limited to latency, and equivalents.

[0006] Motion compensation may involve an approach to predict a video frame or part thereof, based on reference frames such as the preceding and / or future frames, by taking into account the motion of the camera and / or objects in the video. This may be employed in encoding and decoding video data for video compression, for example, in encoding and decoding using the Advanced Video Coding (AVC) standard of the Video Professional Group (MPEG), also known as H.264. Motion compensation may describe a picture in terms of the conversion of a reference picture to the current picture. The reference picture may be temporally preceding and / or from the future when compared to the current picture. [Overview of the project] [Means for solving the problem]

[0007] In one aspect, the decoder includes a network configured to receive a reference frame, determine a scaling constant with respect to the current block, use the reference frame and the scaling constant to determine a scaled reference block, use the scaled reference block to determine a scaled predicted block, and use the rescaled predicted block to reconstruct the pixel data of the current block.

[0008] In another aspect, the method includes receiving a reference frame, determining a scaling constant with respect to the current block, determining a scaled reference block using the reference frame and the scaling constant, determining a scaled predicted block using the scaled reference block, and reconstructing the pixel data of the current block using the rescaled predicted block.

[0009] Details of one or more modifications of the subject matter described herein are shown in the accompanying drawings and the description below. Other features and advantages of the subject matter described herein will be evident from the description and drawings and from the claims. The present invention provides, for example, the following: (Item 1) A decoder, wherein the decoder is It is a network, Receiving the reference frame, For the current block, determine the scaling constant, Using the aforementioned reference frame and the scaling constant, determine the scaled reference block, Using the aforementioned scaled reference block, a scaled prediction block is determined, Reconstruct the pixel data of the current block using the rescaled prediction block. A network configured to perform A decoder equipped with a decoder. (Item 2) The decoder according to item 1, further configured to determine the scaled prediction block by determining scaled motion information. (Item 3) The decoder according to item 2, wherein the scaled motion information is determined by multiplying at least the horizontal component of the motion vector by the scaling constant and the vertical component of the motion vector by the scaling constant. (Item 4) The decoder according to item 1, further configured to determine the scaled reference block by determining the position of the scaled reference block. (Item 5) The decoder according to item 4, wherein the position is determined by multiplying the horizontal component of the position of the reference block by a scaling factor and multiplying the vertical component of the position of the reference block by the scaling factor. (Item 6) Receiving a bitstream and The scaling constant is determined from the bitstream, wherein the scaling constant is transmitted within the bitstream. The decoder described in item 1, further configured to perform the following actions. (Item 7) The bitstream includes an index for a predetermined scaling constant, as described in item 6. (Item 8) The decoder according to item 1, further configured to receive the reference frame by accessing the reference frame from memory. (Item 9) An entropy decoder processor, wherein the entropy decoder processor is configured to receive a bitstream and decode the bitstream into quantized coefficients, An inverse quantization and inverse transform processor, wherein the inverse quantization and inverse transform processor is configured to process the quantized coefficients, including performing an inverse discrete cosine operation; Deblocking filter and Frame buffer and Intra Prediction Processor and The decoder described in item 1, further equipped with the features described in item 1. (Item 10) The aforementioned current block is a coding tree unit, the decoder described in item 1. (Item 11) The current block mentioned above is the decoder described in item 1, which is a coding unit. (Item 12) The current block is the decoder described in item 1, which is the prediction unit. (Item 13) It is a method, Receiving the reference frame, For the current block, determine the scaling constant, determining a scaled reference block using the reference frame and the scaling constant; determining a scaled prediction block using the scaled reference block; reconstructing pixel data of the current block using the rescaled prediction block; the method comprising. (Item 14) The method of Item 13, wherein determining the scaled prediction block comprises determining scaled motion information. (Item 15) The method of Item 14, wherein the scaled motion information is determined by multiplying a horizontal component of a motion vector by the scaling constant and multiplying a vertical component of the motion vector by the scaling constant. (Item 16) The method of Item 13, wherein determining the scaled reference block comprises determining a position of the scaled reference block. (Item 17) The method of Item 16, wherein the position is determined by multiplying a horizontal component of a reference block position by a scaling factor and multiplying a vertical component of the reference block position by the scaling factor. (Item 18) receiving a bitstream; determining the scaling constant from the bitstream, wherein the scaling constant is signaled in the bitstream; The method of Item 13, further comprising. (Item 19) The method of Item 18, wherein the bitstream comprises an index to a predetermined scaling constant. (Item 20) The method of Item 13, wherein receiving the reference frame comprises accessing the reference frame from a memory. (Item 21) At least one of receiving, determining, and reconstructing is performed by a decoder, and the decoder An entropy decoder processor, wherein the entropy decoder processor is configured to receive a bitstream and decode the bitstream into quantized coefficients, An inverse quantization and inverse transform processor, wherein the inverse quantization and inverse transform processor is configured to process the quantized coefficients, including performing an inverse discrete cosine operation; Deblocking filter and Frame buffer and Intra Prediction Processor and The method described in item 13, comprising: (Item 22) The current block is a coding tree unit, as described in item 13. (Item 23) The aforementioned current block is a coding unit, as described in item 13. (Item 24) The current block is a prediction unit, as described in item 13. [Brief explanation of the drawing]

[0010] For the purpose of illustrating the present invention, the drawings illustrate aspects of one or more embodiments of the invention. However, it should be understood that the invention is not limited to the precise arrangement and means shown in the drawings.

[0011] [Figure 1] Figure 1 shows exemplary reference frames and exemplary prediction frames at various resolution scales.

[0012] [Figure 2] Figure 2 illustrates the exemplary baseline frame, the exemplary rescaled baseline frame, and the exemplary subsequent block prediction process.

[0013] [Figure 3] Figure 3 is a diagram illustrating the exemplary reference block positions before and after the scaling process.

[0014] [Figure 4] Figure 4 is a diagram illustrating exemplary motion vector scaling, including a reference block, a predicted block, a scaled reference block, and a scaled predicted block.

[0015] [Figure 5] Figure 5 is a process flow diagram illustrating exemplary processes for several implementations of this subject.

[0016] [Figure 6] Figure 6 is a system block diagram illustrating an exemplary decoder capable of decoding a bitstream according to several implementations of this subject.

[0017] [Figure 7] Figure 7 is a process flow diagram illustrating an exemplary process for encoding video according to several implementations of this subject.

[0018] [Figure 8] Figure 8 is a system block diagram illustrating exemplary video encoders based on several implementations of this subject.

[0019] [Figure 9] Figure 9 is a block diagram of a computing system that may be used to implement any one or more of the methodologies disclosed herein or in any one or more portions thereof.

[0020] Drawings are not necessarily to scale and may be illustrated using imaginary lines, schematic representations, and partial drawings. In some cases, details that are not necessary for understanding the embodiment or that make other details difficult to perceive may be omitted. Similar reference symbols in various drawings indicate similar elements. [Modes for carrying out the invention]

[0021] Detailed explanation In many current state-of-the-art encoders, resolution is managed by recoding and retransmitting entire portions of video known as group of pictures (GOP). This requires transmitting intraframes (I-frames) because those frames involve a large portion of the bits within the GOP, which can incur additional costs.

[0022] Embodiments described herein relate to Adaptive Resolution Management (ARM), a technique that enables additional flexibility for video encoders / decoders, allowing for bitrate savings in various use cases. Generally, ARM involves performing a prediction using a reference frame with a different resolution than the current frame. In current coding standards, the reference frame has the same resolution as the predicted frame. In ARM, the reference frame may have a lower or higher resolution than the predicted frame. This approach can be used to downscale video resolution and thus reduce the bitrate, or to upscale video resolution and thus improve the display characteristics of video playback.

[0023] ARM may be referred to as reference picture resampling (RPR) for the purposes of this disclosure, either as an alternative or equivalent term, and RPR and ARM may be used synonymously.

[0024] Some implementations of this subject may involve using ARM for any number of frames at any position within the GOP, and thus eliminating the requirement for recoding I frames.

[0025] Figure 1 shows reference and predicted frames at various resolution scales. Frame 1 is smaller (lower resolution) than the reference frame, Frame 2 is the same size (same resolution), while Frame 3 is larger (higher resolution). As used in this disclosure, “resolution” is the number of pixels in a picture, frame, subframe, and / or other displayed area or portion thereof used in video playback, compression, or equivalent, where a higher number of pixels corresponds to a higher resolution, and a lower number of pixels corresponds to a lower resolution. Resolution may also be measured in terms of area, for example, by using one or more dimensions of length measured in pixels that define a certain area, for example, but not limited to. For example, a circular subframe or other area may have a resolution defined according to its radius. Alternatively, or in addition, resolution may be defined by the total number of pixels.

[0026] As an example, continuing with reference to Figure 1, if the reference frame and / or subframe have a geometric shape over which the area can be fully defined in terms of two length parameters, such as a triangle, parallelogram, and / or rectangle, then the reference frame and / or subframe may have a resolution W × H (where W and H may represent the number of pixels describing the width (or base) dimension and height dimension of the reference frame and / or subframe, respectively). Each prediction frame may also have a resolution, which can be determined similarly to the resolution of the reference frame. For example, frame 1 may have a lower resolution WS × HS, frame 2 may have the same resolution as the reference frame W × H, and frame 3 may have a higher resolution WL × HL. The width and height of smaller and larger frames can be obtained by multiplying the reference width and height by an arbitrary rescaling constant (Rc), also called a scaling factor and / or constant. For smaller frames, Rc may have a value between 0 and 1. For larger frames, Rc can have a value greater than 1; for example, Rc can have a value between 1 and 4. Other values ​​are also possible. The rescaling constant can differ with respect to one resolution dimension; for example, the rescaling constant Rch may be used to rescale the height, while another rescaling constant Rcw may be used to rescale the width.

[0027] Referring still to Figure 1, ARM may be implemented as a mode. In the case of ARM mode activation at some point during decoding, the decoder may have already received a reference frame at resolution W×H and may rescale the predicted frame using a rescaling constant. In some implementations, the encoder may signal the decoder a rescaling constant for use. This signaling may be done within a sequence parameter set (SPS) corresponding to the GOP containing the current picture, and / or a picture parameter set (PPS) corresponding to the current picture. For example, but not limited to, the encoder may signal the rescaled parameters using fields such as pps_pic_width_in_luma_samples, pps_pic_height_in_luma_samples, pps_scaling_win_left_offset, pps_scaling_win_right_offset, pps_scaling_win_top_offset, pps_scaling_win_bottom_offset, and / or sps_num_subpics_minus1.

[0028] Referring further to Figure 1, the W and H parameters described above can, but are not limited to, be expressed using the variables CurrPicScalWinWidthL and CurrPicScalWinHeightL, respectively, and these variables can be derived from the signal-transmitted parameters described above using one or more mathematical relationships between the signal-transmitted parameters and the variables. For example, but are not limited to, CurrPicScalWinWidthL can be derived according to the following equation. CurrPicScalWinWidthL = pps_pic_width_in_luma_samples - SubWidthC * (pps_scaling_win_right_offset + pps_scaling_win_left_offset) As a further non-limiting embodiment, CurrPicScalWinHeightL can be derived according to the following equation. CurrPicScalWinWidthL = pps_pic_width_in_luma_samples - SubWidthC * (pps_scaling_win_right_offset + pps_scaling_win_left_offset)

[0029] Those skilled in the art will recognize, upon examination of the entirety of this disclosure, various alternative calculations that may be used to derive the variables described above. The encoder may, as an alternative or in addition, directly signal one or more such variables Rc, Rch, and / or Rcw within the PPS and / or SPS, for example, but not limited to these.

[0030] Alternatively, or in addition, still referring to Figure 1, the rescaling constants and / or sets of rescaling constants described above may be signaled in the bitstream using stored scaling constants and / or multiple scaling constants, and / or references to indices of frames and / or blocks that were signaled using the scaling constants and / or multiple scaling constants that were signaled immediately before. The references to the indices of stored scaling constants may be explicitly signaled and / or determined from one or more additional parameters that are signaled in the bitstream. For example, but not limited to, the decoder may identify a group of pictures containing a reference frame and / or the current frame, and if the rescaling constants were previously signaled and / or used in such a group of pictures using the reference frame that is signaled so as to be applicable to the current frame and / or the current group of pictures, or equivalents, the decoder may identify those rescaling constants for use as rescaling constants using the current frame.

[0031] In some implementations, continuing with reference to Figure 1, the ARM operation may be performed depending on the block level of the encoded frame. For example, the reference frame may first be rescaled, and then the prediction may be performed as depicted in Figure 2. Figure 2 is a diagram depicting the reference frame, the rescaled reference frame, and the subsequent block prediction process. The block prediction process may be performed on a scaled reference frame (with scaled resolution) rather than the original reference frame. Rescaling the reference frame may include rescaling according to any parameter signaled by the encoder as described above, for example, if the reference frame to be used with the current picture is signaled via a reference to an index value associated with the reference frame or its equivalent, the signaled reference frame may be rescaled prior to the prediction according to any of the rescaling methods described above. The rescaled reference frame may be stored in memory and / or in a buffer, which may include, but not limited to, a buffer that identifies the frames contained therein by index, according to any frame reads that may be performed, and the buffer may include one or more additional buffers implemented by the decoded picture buffer (DCB) and / or decoder. The prediction process may include, for example, interpicture prediction, including motion compensation.

[0032] Some implementations of block-based ARM may allow for the flexibility of applying the optimal filter to each block instead of applying the same filter across the entire frame. In some implementations, a skip ARM mode may be considered, such that some blocks (based on, for example, uniformity of pixel and bitrate costs) may be in a skip ARM mode (where rescaling would not change the bitrate). The skip ARM mode may be signaled within the bitstream, and may also be signaled within the PPS parameters, for example, but not limited to these. Alternatively, or in addition, the decoder may determine that the skip ARM mode is active based on one or more parameters set by the decoder and / or signaled within the bitstream. Spatial filters used within block-based ARM may include, but are not limited to, bicubic spatial filters that apply bicubic interpolation, bilinear spatial filters that apply bilinear interpolation, Lanczos filters that use Lanczos filtering, and / or sinc filters, sinc function interpolation, and / or Lanczos resampling that use a combination of signal reconstruction techniques, and a person skilled in the art will recognize, upon a thorough examination of the entire disclosure, a variety of filters that may be used consistently in the disclosure for interpolation.

[0033] Continuing to refer to Figure 2, if the sequence of frames has already been encoded, post-encoding ARM may enable rapid rescaling and transmission of the expected coding units (blocks or frames). This can eliminate the need to re-encode video frames at different resolutions. Several implementations of this subject can reduce computational complexity at a relatively small cost in terms of rate distortion performance.

[0034] Referring further to Figure 2, depending on the use case, rescaling may be performed in the encoding and decoding process, which the encoder transmits to the decoder, either on the encoder side (to downscale the reduced bandwidth) or on the decoder side (to upscale and then reduce the bandwidth required for video transmission). In some implementations, scaling may be performed using interpolation filters, such as subpixel prediction filters. The interpolation filters may include, but are not limited to, any filters described above, such as low-pass filters, which may be used with an upsampling process, by which pixels between pixels in the block and / or frame immediately preceding the scaling may be initialized to zero, and then the output of the low-pass filter may be taken. Alternatively, or in addition, any luminance sample interpolation filtering process may be used. Luminance sample interpolation may include calculating an interpolated value in a half-sample interpolation filter index corresponding to two consecutive sample values ​​of an unscaled sample array. The calculation of interpolated values ​​may, but are not limited to, be performed by reading coefficients and / or weights from a lookup table, the selection of the lookup table may be performed as a function of the motion model of coding units and / or scaling ratios, for example, using scaling constants as described above. The calculation may, but are not limited to, perform a weighted sum of adjacent pixel values ​​if the weights are read from a lookup table. The calculated values ​​may, alternatively or in addition, be shifted, for example, but are not limited to, the values ​​being shifted by a minimum value (4, i.e., bit depth -8), a maximum value (2, i.e., 14-bit depth), or equivalents. Those skilled in the art will recognize various alternative or additional implementations that may be used for interpolation filters in accordance with a thorough examination of this disclosure.

[0035] In such an approach, still referring to Figure 2, the predicted elements may be rescaled by a rate Rc (scaling constant or rate), which may be signaled to the decoder, for example, as described above. For example, but not limited to, pps_scaling_window_explicit_signalling_flag equal to 1 may indicate that the scaling window offset parameter is in the PPS, while pps_scaling_window_explicit_signalling_flag equal to 0 may indicate that the scaling window offset parameter is not in the PPS. When sps_ref_pic_resampling_enabled_flag is equal to 0, pps_scaling_window_explicit_signalling_flag may be equal to 0.

[0036] The scaled elements may include intracoding units and intercoding units (e.g., blocks) that can be rescaled using the scaling rates depicted in Figure 3, etc. Figure 3 is a diagram illustrating the reference block positions before and after the scaling process.

[0037] Scaled elements may include motion vectors. Figure 4 illustrates motion vector scaling, including a reference block, a predicted block, a scaled reference block, and a scaled predicted block. Figure 4 illustrates motion-compensated predictions with respect to the original-size reference frame (top) and the scaled reference frame (bottom).

[0038] Figure 5 is a process flow diagram illustrating an exemplary embodiment of process 500 of adaptive resolution management, which can enable additional flexibility for video encoders / decoders, allowing for bitrate savings in various use cases.

[0039] In step 505, still referring to Figure 5, the reference frame is received. The reference frame may be received (and accessed, for example) by memory. The reference frame may have been decoded immediately before from the bitstream. The bitstream may have been received by the decoder. The bitstream may contain the current block. The current block may be contained within the bitstream received by the decoder. The bitstream may contain data found in a stream of bits, for example, the input to the decoder when using data compression. The bitstream may contain information necessary to decode the video. Receiving may involve extracting and / or parsing blocks and associated signaling information from the bitstream. In some implementations, the current block may contain coding tree units (CTUs), coding units (CUs), and / or prediction units (PUs).

[0040] In step 510, continuing to refer to Figure 5, the scaling constant is determined with respect to the current block. In some implementations, the scaling constant may be determined from the bitstream. For example, the scaling constant may be signaled within the bitstream, the bitstream may contain an index to one of several predetermined scaling constants, or similar.

[0041] In step 515, still referring to Figure 5, the scaled reference block may be determined using the reference frame and a scaling constant. Determining the scaled reference block may include determining the position of the scaled reference block. In some implementations, the position may be determined by multiplying at least one or more dimensions of the reference block by one or more scaling factors, for example, the horizontal component of the reference block's position may be multiplied by the scaling factor, and the vertical component of the reference block's position may be multiplied by the scaling factor.

[0042] In step 520, continuing to refer to Figure 5, the scaled prediction block may be determined using the scaled reference block. In some implementations, determining the scaled prediction block may include determining the scaled motion information. The scaled motion information may be determined by multiplying the horizontal component of the motion vector by a scaling constant and the vertical component of the motion vector by a scaling constant.

[0043] In step 525, still referring to Figure 5, the pixel data of the current block may be reconstructed using the rescaled predicted block.

[0044] Figure 6 is a system block diagram illustrating an exemplary decoder 600 capable of adaptive resolution-controlled predictive rescaling as described herein. The decoder 600 may include an entropy decoder processor 604, an inverse quantization and inverse transform processor 608, a deblocking filter 612, a frame buffer 616, a motion compensation processor 620, and / or an intra-predictive processor 624.

[0045] During operation, still referring to Figure 6, the bitstream 628 may be received by the decoder 600 and input to the entropy decoder processor 604, which may entropi-decode a portion of the bitstream into quantized coefficients. The quantized coefficients may be provided to the inverse quantization and inverse transform processor 608, which may perform inverse quantization and inverse transform and generate a residual signal, which may be added to the output of the motion compensation processor 620 or the intra-prediction processor 624, depending on the processing mode. The outputs of the motion compensation processor 620 and the intra-prediction processor 624 may include block predictions based on the block that was decoded immediately before. The sum of the predictions and residuals may be processed by the deblocking filter 612 and stored in the frame buffer 616.

[0046] Figure 7 is a process flow diagram illustrating an exemplary embodiment of a process 700 for encoding video using adaptive resolution management, which may enable additional flexibility for video encoders and / or decoders, allowing for bitrate savings in various use cases. In step 705, the video frames may undergo initial block segmentation using a tree-structured macroblock partitioning scheme, which may include, for example, partitioning picture frames into CTUs and CUs.

[0047] In step 710, still referring to Figure 7, block-based adaptive resolution management may be performed, including resolution scaling of a frame or part thereof.

[0048] In step 715, continuing to refer to Figure 7, the block may be encoded and included in the bitstream. Encoding may include, for example, utilizing inter-prediction mode and intra-prediction mode.

[0049] Figure 8 is a system block diagram illustrating an exemplary video encoder 800 capable of adaptive resolution-managed predictive rescaling as described herein. The exemplary video encoder 800 may receive an input video 804, which may first be segmented or divided according to a processing scheme such as a tree-structured macroblock partitioning scheme (e.g., quadtree + binary tree). An embodiment of the tree-structured macroblock partitioning scheme may include partitioning a picture frame into larger block elements called coding tree units (CTUs). In some implementations, each CTU may be further partitioned once or more times into several subblocks called coding units (CUs). The final result of this partitioning may include a group of subblocks, which may be called predictive units (PUs). Transformation units (TUs) may also be utilized.

[0050] Still referring to Figure 8, the exemplary video encoder 800 may include an intra-prediction processor 808, which may also be called an inter-prediction processor, capable of constructing a motion vector candidate list, including adding global motion vector candidates to the motion vector candidate list; a motion estimation / compensation processor 812; a transform / quantization processor 816; an inverse quantization / inverse transform processor 820; an in-loop filter 824; a decoded picture buffer 828; and / or an entropiccoding processor 832. Bitstream parameters may be input to the entropiccoding processor 832 for inclusion in the output bitstream 836.

[0051] During operation, continuing to refer to Figure 8, for each block of frames in the input video 804, it may be determined whether to process the block via intra-picture prediction or using motion estimation / compensation. The block may be provided to the intra-prediction processor 808 or the motion estimation / compensation processor 812. If the block should be processed via intra-prediction, the intra-prediction processor 808 may perform the processing and output predictors. If the block should be processed via motion estimation / compensation, the motion estimation / compensation processor 812 may perform the processing, including, if applicable, constructing a motion vector candidate list, including adding global motion vector candidates to the motion vector candidate list.

[0052] Referring further to Figure 8, the residual may be formed by subtracting the predictor from the input video. The residual may be received by a transform / quantization processor 816, which may perform a transform (e.g., a discrete cosine transform (DCT)) and produce coefficients that can be quantized. The quantized coefficients and any associated signal-transmission information may be provided to an entropic coding processor 832 for entropic encoding and inclusion in the output bitstream 836. The entropic coding processor 832 may assist in encoding the signal-transmission information related to encoding the current block. In addition, the quantized coefficients may be combined with the predictor and provided to an inverse quantization / inverse transform processor 820, which can reconstruct pixels that can be processed by an in-loop filter 824, the output of which may be stored in a decoded picture buffer 828 for use by a motion estimation / compensation processor 812, which can construct a motion vector candidate list, including adding global motion vector candidates to the motion vector candidate list.

[0053] Continuing to refer to Figure 8, several variations have been described in detail above, but other modifications or additions are also possible. For example, in some implementations, the current blocks may include any symmetric blocks (8×8, 16×16, 32×32, 64×64, 128×128, and equivalents) as well as any asymmetric blocks (8×4, 16×8, and equivalents).

[0054] In some implementations, still referring to Figure 8, a quadtree-binary decision tree (QTBT) may be implemented. In a QTBT, at the coding tree unit level, the partition parameters of the QTBT may be dynamically derived to fit local characteristics without transmitting any overhead. Subsequently, at the coding unit level, a joint classifier decision tree structure may be used to eliminate unnecessary iterations and control the risk of false predictions. In some implementations, an LTR frameblock update mode may be available as an additional option per leaf node of the QTBT.

[0055] In some implementations, still referring to Figure 8, additional syntactic elements may be signaled at different hierarchical levels of the bitstream. For example, a flag may be enabled throughout the sequence by including an enabled flag coded within the sequence parameter set (SPS). Furthermore, a CTU flag may be coded at the coding tree unit (CTU) level.

[0056] Some embodiments may include a non-transient computer program product (i.e., a physically embodied computer program product) that, when executed by one or more data processors of one or more computing systems, stores instructions causing at least one data processor to perform the operations described herein.

[0057] Embodiments disclosed herein may include a decoder having a network configured to receive a reference frame, determine a scaling constant with respect to the current block, determine a scaled reference block using the reference frame and the scaling constant, determine a scaled predicted block using the scaled reference block, and reconstruct the pixel data of the current block using the rescaled predicted block.

[0058] In some embodiments, the decoder may further be configured to determine a scaled prediction block by determining scaled motion information. The scaled motion information may be determined by multiplying at least the horizontal component of the motion vector by a scaling constant and the vertical component of the motion vector by a scaling constant. The decoder may further be configured to determine a scaled reference block by determining the position of a scaled reference block. The position may be determined by multiplying the horizontal component of the reference block's position by a scaling factor and the vertical component of the reference block's position by a scaling factor. The decoder may further be configured to receive a bitstream and determine a scaling constant from the bitstream, which is a scaling constant that is signaled within the bitstream. The bitstream may include an index for a given scaling constant. The decoder may further be configured to receive a reference frame by accessing a reference frame from memory. The decoder may include an entropy decoder processor configured to receive a bitstream and decode the bitstream into quantized coefficients, an inverse quantization and inverse transform processor configured to process the quantized coefficients, including performing inverse discrete cosine, a deblocking filter, a frame buffer, and an intra-prediction processor. The current block may include a coding tree unit. The current block may contain coding units. The current block may contain prediction units.

[0059] Embodiments disclosed herein may include a method which may include receiving a reference frame, determining a scaling constant with respect to the current block, determining a scaled reference block using the reference frame and the scaling constant, determining a scaled predicted block using the scaled reference block, and reconstructing the pixel data of the current block using the rescaled predicted block.

[0060] In some embodiments, determining a scaled prediction block may include determining scaled motion information. The scaled motion information may be determined by multiplying the horizontal component of the motion vector by a scaling constant and the vertical component of the motion vector by a scaling constant. Determining a scaled reference block may include determining the position of a scaled reference block. The position may be determined by multiplying the horizontal component of the reference block's position by a scaling factor and the vertical component of the reference block's position by a scaling factor. The method may include receiving a bitstream and determining a scaling constant from the bitstream, wherein the scaling constant is signaled within the bitstream. The bitstream may include an index for a given scaling constant. Receiving a reference frame may include accessing the reference frame from memory. At least one of receiving, determining, and reconstructing may be performed by a decoder comprising: an entropy decoder processor configured to receive a bitstream and decode the bitstream into quantized coefficients; an inverse quantization and inverse transform processor configured to process the quantized coefficients, including performing an inverse discrete cosine; a deblocking filter; a frame buffer; and an intra-prediction processor. The current block may include a coding tree unit. The current block may include a coding unit. The current block may include a prediction unit.

[0061] It should be noted that any one or more of the aspects and embodiments described herein may be conveniently implemented using digital electronic networks, integrated circuit networks, specially designed application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), computer hardware, firmware, software, and / or combinations thereof, so as will be obvious to those skilled in the computer art, and may be realized and / or implemented in one or more machines (e.g., one or more computing devices used as a user computing device for electronic documents, one or more server devices such as a document server, etc.) programmed in accordance with the teachings herein. These various aspects or features may include implementations in one or more computer programs and / or software that are executable and / or interpretable on a programmable system, including at least one programmable processor, which may be for special purpose or general purpose, that receives data and instructions from a storage system, at least one input device, and at least one output device, and is coupled to transmit data and instructions to them. Appropriate software coding can be readily prepared by a skilled programmer based on the teachings of this disclosure, so as to be obvious to those skilled in the software art. The aspects and implementations discussed above, employing software and / or software modules, may also include appropriate hardware to assist in the implementation of machine-executable instructions of the software and / or software modules.

[0062] Such software may be a computer program product employing a machine-readable storage medium. The machine-readable storage medium may be any medium capable of storing and / or encoding a sequence of instructions for execution by a machine (e.g., a computing device), causing the machine to implement any one of the methodologies and / or embodiments described herein. Examples of machine-readable storage media include, but are not limited to, magnetic disks, optical disks (e.g., CDs, CD-Rs, DVDs, DVD-Rs, etc.), magneto-optical disks, read-only memory "ROM" devices, random access memory "RAM" devices, magnetic cards, optical cards, solid-state memory devices, EPROMs, EEPROMs, programmable logic devices (PLDs), and / or any combination thereof. As used herein, the machine-readable storage medium is intended to include a single medium, as well as a collection of physically separate media, such as a compact disk or a collection of one or more hard disk drives in combination with computer memory. As used herein, the machine-readable storage medium does not include transient forms of signal transmission.

[0063] Such software may also include information (e.g., data) carried as data signals on a data carrier such as a carrier wave. For example, machine-executable information may include data-carry signals embodied in a data carrier, where the signals encode a sequence or part of instructions for execution by a machine (e.g., a computing device), and any related information (e.g., data structures and data) that causes the machine to implement any one of the methodologies and / or embodiments described herein.

[0064] Embodiments of computing devices include, but are not limited to, e-book reading devices, computer workstations, terminal computers, server computers, handheld devices (e.g., tablet computers, smartphones, etc.), web devices, network routers, network switches, network bridges, and any machine capable of executing a sequence of instructions that define the actions to be taken by such machines and any combination thereof. In one embodiment, the computing device includes and / or may be contained within a kiosk.

[0065] Figure 9 shows a schematic representation of one embodiment of a computing device in an exemplary form of computer system 900, in which a set of instructions can be executed to cause a control system to implement one or more aspects and / or methodologies of the present disclosure. It is also conceivable that multiple computing devices may be used to implement a specially configured set of instructions for causing one or more of the devices to implement one or more aspects and / or methodologies of the present disclosure. Computer system 900 includes a processor 904 and memory 908 that communicate with each other and other components via a bus 912. Bus 912 may include any of several types of bus structures, including, but not limited to, a memory bus, a memory controller, a peripheral bus, a local bus, and any combination thereof, using any of various bus architectures.

[0066] Memory 908 may include, but is not limited to, various components (e.g., machine-readable media), including random-access memory components, read-only components, and any combination thereof. In one embodiment, a basic input / output system 916 (BIOS), which includes basic routines that help transfer information between elements within the computer system 900 during startup, etc., may be stored in memory 908. Memory 908 may also include instructions (e.g., software) 920 (e.g., stored on one or more machine-readable media) that embody one or more aspects and / or methodologies of this disclosure. In another embodiment, memory 908 may further include, but is not limited to, any number of program modules, including an operating system, one or more application programs, other program modules, program data, and any combination thereof.

[0067] The computer system 900 may also include a storage device 924. Embodiments of the storage device (e.g., storage device 924) include, but are not limited to, hard disk drives, magnetic disk drives, optical disk drives in combination with optical media, solid-state memory devices, and any combination thereof. The storage device 924 may be connected to the bus 912 by a suitable interface (not shown). Exemplary interfaces include, but are not limited to, SCSI, Advanced Technology Attachment (ATA), Serial ATA, Universal Serial Bus (USB), IEEE 1394 (FIREWIRE®), and any combination thereof. In one embodiment, the storage device 924 (or one or more of its components) may be removably interfaced with the computer system 900 (e.g., via an external port connector (not shown)). In particular, the storage device 924 and associated machine-readable media 928 may provide non-volatile and / or volatile storage devices for machine-readable instructions, data structures, program modules, and / or other data for the computer system 900. In one embodiment, the software 920 may reside entirely or partially within a machine-readable medium 928. In another embodiment, the software 920 may reside entirely or partially within a processor 904.

[0068] The computer system 900 may also include an input device 932. In one embodiment, a user of the computer system 900 may input commands and / or other information into the computer system 900 via the input device 932. Embodiments of the input device 932 include, but are not limited to, alphanumeric input devices (e.g., keyboards), pointing devices, joysticks, gamepads, audio input devices (e.g., microphones, voice response systems, etc.), cursor control devices (e.g., mice), touchpads, optical scanners, video capture devices (e.g., still cameras, video cameras), touchscreens, and any combination thereof. The input device 932 may interface to the bus 912 via, but are not limited to, a variety of interfaces (not shown) including serial interfaces, parallel interfaces, game ports, USB interfaces, FIREWIRE® interfaces, direct interfaces to the bus 912, and any combination thereof. The input device 932 may also include a touchscreen interface, which may be part of or separate from the display 936, which will be discussed further below. The input device 932 may be used as a user selection device for selecting one or more graphic representations within a graphical interface as described above.

[0069] The user may also input commands and / or other information to the computer system 900 via a storage device 924 (e.g., a removable disk drive, a flash drive, etc.) and / or a network interface device 940. Network interface devices such as network interface device 940 may be used to connect the computer system 900 to one or more of various networks such as network 944, and one or more remote devices 948 connected thereto. Embodiments of network interface devices include, but are not limited to, network interface cards (e.g., mobile network interface cards, LAN cards), modems, and any combination thereof. Embodiments of networks include, but are not limited to, wide area networks (e.g., the Internet, corporate networks), local area networks (e.g., networks associated with offices, buildings, campuses, or other relatively small geographical spaces), telephone networks, data networks associated with telephone / voice providers (e.g., data and / or voice networks of mobile communication providers), direct connections between two computing devices, and any combination thereof. Networks such as network 944 may employ wired and / or wireless modes of communication. In general, any network topology may be used. Information (e.g., data, software 920, etc.) may be communicated to and / or from the computer system 900 via the network interface device 940.

[0070] The computer system 900 may further include a video display adapter 952 for communicating images that can be displayed on a display device such as a display device 936. Embodiments of the display device include, but are not limited to, liquid crystal displays (LCDs), cathode ray tubes (CRTs), plasma displays, light-emitting diode (LED) displays, and any combination thereof. The display adapter 952 and the display device 936 may be used in combination with a processor 904 to provide a graphic representation of aspects of this disclosure. In addition to the display devices, the computer system 900 may include, but are not limited to, one or more other peripheral output devices, including audio speakers, printers, and any combination thereof. Such peripheral output devices may be connected to the bus 912 via a peripheral interface 956. Embodiments of the peripheral interface include, but are not limited to, serial ports, USB connections, FIREWIRE®, parallel connections, and any combination thereof.

[0071] The foregoing is a detailed description of illustrative embodiments of the present invention. Various modifications and additions can be made without departing from the spirit and scope of the invention. Each feature of the various embodiments described above can be combined with features of other described embodiments as appropriate to provide a combination of multiple features in the associated new embodiments. Furthermore, the foregoing describes several distinct embodiments, but those described herein are merely illustrative of the application of the principles of the invention. In addition, certain methods described herein may be illustrated and / or described as being carried out in a specific order, but the order is highly variable among those skilled in the art to achieve embodiments such as those disclosed herein. Therefore, this description is intended to be taken only as examples and is not intended to otherwise limit the scope of the invention.

[0072] In the above description and claims, phrases such as “at least one of ~” or “one or more of ~” may occur and be followed by a conjunctive enumeration of elements or features. The term “and / or” may also occur within an enumeration of two or more elements or features. Unless otherwise implicitly or expressly refuted by the context in which such phrase is used, it is intended to mean either of the individually enumerated elements or features, or any of the elements or features cited in combination with any other cited elements or features. For example, the phrases “at least one of A and B,” “one or more of A and B,” and “A and / or B” are intended to mean “A alone, B alone, or A and B together,” respectively. A similar interpretation is also intended for enumerations containing three or more items. For example, the phrases “at least one of A, B, and C,” “one or more of A, B, and C,” and “A, B, and / or C” are intended to mean “A alone, B alone, C alone, A and B together, A and C together, B and C together, or A and B as well as C together,” respectively. In addition, the use of the term “based on” above and within the claims is intended to mean “at least, based on” any feature or element not cited, as is also acceptable.

[0073] The subject matter described herein can be embodied in systems, apparatus, methods, and / or articles, depending on the desired configuration. The implementations described herein do not represent all implementations consistent with the subject matter described herein. Rather, they are merely some examples consistent with aspects related to the subject matter described herein. While some variations are described in detail above, other modifications or additions are also possible. In particular, further features and / or variations may be provided in addition to those described herein. For example, the implementations described above may cover various combinations and subcombinations of the disclosed features and / or combinations and subcombinations of some further features disclosed above. In addition, the logical flows depicted in the accompanying diagrams and / or described herein do not necessarily require a specific order or sequential order shown to achieve the desired result. Other implementations may also be within the scope of the following claims.

Claims

1. An encoder, wherein the encoder is Receiving a video signal, By processing the aforementioned video signal, an encoded bitstream is generated. It is configured to do the following: The encoded bitstream comprises a reference picture, a current picture, and information useful for determining a scaling constant, wherein the current picture has a resolution different from that of the reference picture, and includes a first block and a second block. Processing the aforementioned video signal is Determining the scaling constant, Determining a scaled prediction block from the aforementioned reference picture, wherein determining the scaled prediction block from the aforementioned reference picture is The location of the scaled predicted block is determined by scaling the motion vector components using the scaling constant, The resolution of the blocks of the reference picture is scaled using the scaling constant, and a filter is applied that is determined for the scaled predicted blocks but not for the entire reference picture. The act of being done by, Reconstructing the first block using the scaled prediction block, Reconstructing the second block without using scaled prediction blocks, The information useful for determining the scaling constant is transmitted within the encoded bitstream. Encoders, including

2. The encoder according to claim 1, wherein the information in the bitstream that helps determine the scaling constant includes an index to a set of predetermined values, the index being used to determine the scaling constant.

3. An encoder, wherein the encoder is Receiving a video signal, By processing the aforementioned video signal, an encoded bitstream is generated. It is configured to do the following: The encoded bitstream includes a current coded picture comprising a first plurality of coded blocks and a second plurality of coded blocks, Processing the aforementioned video signal is Encoding each block of the aforementioned first plurality of coded blocks, wherein encoding each block of the aforementioned first plurality of coded blocks is Determining the scaling constant, Determining scaled prediction blocks from a reference picture having a resolution different from the resolution of the currently coded picture, wherein determining the scaled prediction blocks is done by scaling the motion vector components to determine the location of the scaled prediction blocks, scaling the blocks from the reference picture using the scaling constant, and applying individually determined filters to the scaled prediction blocks. The act of being done by, Encoding each block of the first plurality of coded blocks using the scaled prediction block, Encoding each block of the second plurality of coded blocks without using scaled prediction blocks of the reference picture and without applying a resolution change filter determined for the blocks of the reference picture. Encoders, including

4. The encoder according to claim 3, wherein the information in the bitstream includes an index for a set of predetermined values, the index being used to determine the scaling constant.

5. A video encoder having a network, wherein the network is Receiving a video signal, Encoding a bitstream and It is configured to do the following: The bitstream comprises a reference picture, a current picture, and information useful for determining a scaling constant, wherein the current picture has a different resolution than the reference picture and includes a first block and a second block, the second block being an intracoded block. The aforementioned network further, Enable block-based mode for decoding a current picture of one resolution using a reference picture of a different resolution, Determining the scaling constant, Determining a scaled prediction block from the reference picture using the enabled mode, The location of the scaled predicted block is determined by scaling the motion vector components using the scaling constant, The resolution of the blocks of the reference picture is scaled using the scaling constant, and a filter is applied that is determined for the predicted blocks but not for the reference picture as a whole. The act of being done by, Reconstructing the first block using the scaled prediction block, Reconstructing the intracoded second block without using the aforementioned mode, The information useful for determining the scaling constant is transmitted within the bitstream. A video encoder configured to perform the following actions.

6. A video encoder having a network, wherein the network is configured to encode a bitstream, The bitstream includes a reference picture, a current picture, signal transmission information for determining whether a block-based mode is enabled for decoding a current picture of a certain resolution using a reference picture of a different resolution, and information for determining a scaling constant, wherein the current picture has a different resolution than the reference picture and includes a first block and a second block, the second block being an intracoded block. The aforementioned network further, Enable block-based mode for decoding a current picture of one resolution using a reference picture of a different resolution, Determining the scaling constant, Determining a scaled prediction block from the reference picture using the enabled mode, The location of the scaled predicted block is determined by scaling the motion vector components using the scaling constant, The resolution of the blocks of the reference picture is scaled using the scaling constant, and a filter is applied that is determined for the predicted blocks but not for the reference picture as a whole. The act of being done by, Reconstructing the first block using the scaled prediction block, Reconstructing the intracoded second block without using the aforementioned mode, The signal transmission information and the information for determining the scaling constant are transmitted within the bitstream. A video encoder configured to perform the following actions.

7. A method, The present invention generates an encoded bitstream comprising a reference picture, a current picture, and information useful for determining a scaling constant, wherein the current picture has a different resolution than the reference picture and comprises a first block and a second block, the second block being an intra-coded block. Enable block-based mode for decoding a current picture of one resolution using a reference picture of a different resolution, Determining the scaling constant, Determining a scaled prediction block from the reference picture using the enabled mode, The location of the scaled predicted block is determined by scaling the motion vector components using the scaling constant, The resolution of the blocks of the reference picture is scaled using the scaling constant, and a filter is applied that is determined for the predicted blocks but not for the reference picture as a whole. The act of being done by, Reconstructing the first block using the scaled prediction block, Reconstructing the intracoded second block without using the aforementioned mode Methods that include...

8. A method, The method for generating an encoded bitstream comprising a reference picture, a current picture, signal transmission information for determining whether a block-based mode for decoding a current picture of a certain resolution using a reference picture of a different resolution is enabled, and information for determining a scaling constant, wherein the current picture has a different resolution than the reference picture and comprises a first block and a second block, the second block being an intra-coded block. Determining the scaling constant, Determining a scaled prediction block from the reference picture using the enabled mode, The location of the scaled predicted block is determined by scaling the motion vector components using the scaling constant, The resolution of the blocks of the reference picture is scaled using the scaling constant, and a filter is applied that is determined for the predicted blocks but not for the reference picture as a whole. The act of being done by, Reconstructing the first block using the scaled prediction block, Reconstructing the intracoded second block without using the aforementioned mode Methods that include...

Images