Adaptive resolution management predictive rescaling

Abstract

A method includes receiving a reference frame, determining a scaling constant for a current block, determining a scaled reference block using the reference frame and the scaling constant, determining a scaled prediction block using the scaled reference block, and reconstructing pixel data of the current block using the scaled prediction block. Related apparatus, systems, techniques, and articles are also described.

Description

[0001] This application is a continuation of application No. 2020800700426 filed on August 6, 2020 in China with the title “Adaptive Resolution Management Prediction Rescaling”. CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 883,454, filed on August 6, 2019, entitled “ADAPTIVE RESOLUTION MANAGEMENT PREDICTION RESCALING”, the entirety of which is incorporated by reference herein. TECHNICAL FIELD

[0003] The present invention relates generally to the field of video compression. In particular, the present invention relates to adaptive resolution management prediction rescaling. BACKGROUND

[0004] A video codec can include electronic circuitry or software that compresses or decompresses digital video. The electronic circuitry or software can convert uncompressed video into compressed format, and vice versa. In the context of video compression, a device that compresses (and / or performs some function of) video can be generally referred to as an encoder, and a device that decompresses (and / or performs some function of) video can be generally referred to as a decoder.

[0005] The format of the compressed data can conform to a standard video compression specification. Compression can be lossy, in that some information present in the original video can be missing from the compressed video. Consequences of this can include that the decompressed video can have lower quality than the original uncompressed video, because there is not enough information to accurately reconstruct the original video.

[0006] There is a complex relationship between video quality, amount of data used to represent the video (e.g., determined by bit rate), complexity of encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access, end-to-end delay (e.g., latency), and the like.

[0007] Motion compensation can include a method of predicting a video frame or a portion of a given reference frame (e.g., a previous frame and / or a future frame) by taking into account motion of a camera and / or objects in the video. The method can be used for encoding and decoding of video data for video compression, such as using the Moving Picture Experts Group (MPEG) Advanced Video Coding (AVC) standard, also known as H.264. Motion compensation can describe an image in terms of a transformation from a reference image to a current image. The reference image can be temporally previous or from the future when compared to the current image. Summary of the Invention

[0008] On one hand, a decoder includes circuitry configured to: receive a reference frame, determine a scaling constant for a current block, determine a scaled reference block using the reference frame and the scaling constant, determine a scaled prediction block using the scaled reference block, and reconstruct pixel data of the current block using the scaled prediction block.

[0009] On the other hand, a decoder includes: receiving a reference frame, determining a scaling constant for a current block, using the reference frame and the scaling constant to determine a scaled reference block, using the scaled reference block to determine a scaled prediction block, and reconstructing pixel data of the current block using the scaled prediction block.

[0010] Details of one or more variations of the subject matter described herein are set forth in the accompanying drawings and detailed descriptions. Further features and advantages of the subject matter described herein will become apparent from the detailed descriptions, the drawings, and the claims. Attached Figure Description

[0011] To illustrate the invention, the accompanying drawings show various aspects of one or more embodiments of the invention. However, it should be understood that the invention is not limited to the precise arrangements and means shown in the drawings, in which: Figure 1 These are illustrations of exemplary reference frames and exemplary prediction frames for various resolution levels; Figure 2 This is an illustration depicting an exemplary reference frame, an exemplary rescaling reference frame, and an exemplary subsequent block prediction process; Figure 3 This is an illustration depicting the positions of an exemplary reference block before and after the scaling process; Figure 4 This is an illustration depicting an exemplary motion vector scaling that includes a reference block, a prediction block, a scaled reference block, and a scaled prediction block; Figure 5 This is a process flowchart illustrating an exemplary process according to some implementations of the present topic; Figure 6 This is a system block diagram illustrating an exemplary decoder capable of decoding bitstreams according to some embodiments of the present topic. Figure 7 This is a flowchart illustrating an exemplary process for encoding video according to some implementations of the present topic; Figure 8 This is a system block diagram illustrating an exemplary video encoder according to some implementations of the present topic; and Figure 9It is a block diagram of a computational system that can be used to implement any one or more methods disclosed herein and any one or more parts thereof.

[0012] The accompanying drawings are not necessarily drawn to scale and may use dashed lines, schematic diagrams, and partial views for illustration. In some cases, details that are unnecessary for understanding the embodiments or that make other details difficult to understand may have been omitted. In the drawings, the same reference numerals refer to the same elements. Detailed Implementation

[0013] In many of today's state-of-the-art encoders, resolution is managed by re-encoding and retransmitting entire portions of video known as Groups of Pictures (GOPs). This requires sending internal frames (I-frames), which can result in additional cost because these frames are responsible for most of the bits in the GOP.

[0014] The embodiments described in this disclosure relate to Adaptive Resolution Management (ARM), a technique that provides additional flexibility to video encoders / decoders, thereby allowing bitrate savings in a variety of use cases. Typically, ARM involves performing prediction using a reference frame with a different resolution than the current frame. In current coding standards, the reference frame has the same resolution as the predicted frame. In ARM, the resolution of the reference frame may be less than or greater than the resolution of the frame being predicted. This approach can be used to reduce video resolution, thereby reducing the bitrate, or to increase video resolution, thereby improving the display characteristics of video playback.

[0015] For the purposes of this disclosure, ARM may be alternatively or equivalently referred to as Reference Image Resampling (RPR); RPR and ARM are used interchangeably.

[0016] Some implementations of the current topic may include using ARM for any number of frames at any location within a GOP, thereby eliminating the need for I-frame re-encoding.

[0017] Figure 1 This is an illustration of exemplary reference frames and exemplary predicted frames illustrating various resolution levels. Frame 1 is smaller than the reference frame (lower resolution), frame 2 is the same size (same resolution), and frame 3 is larger (higher resolution). As used in this disclosure, "resolution" refers to the number of pixels in an image, frame, subframe, and / or other display area or portion thereof used in video playback, compression, etc., with a higher number of pixels corresponding to a higher resolution and a lower number of pixels corresponding to a lower resolution. Resolution can be measured based on area, for example, but not limited to, by using one or more length dimensions to define the area in pixels. For example, a circular subframe or other region may have a resolution defined according to its radius.

[0018] Alternatively or additionally, resolution can be defined by the total number of pixels.

[0019] As an example, continue to refer to Figure 1 The reference frame and / or subframe have a geometry whose area can be defined entirely by two length parameters, such as, but not limited to, triangles, parallelograms, and / or rectangles. The reference frame and / or subframe can have a resolution W×H, where W and H can indicate the number of pixels describing the width (or bottom) and height dimensions of the reference frame and / or subframe, respectively. Each predicted frame can also have a resolution that can be determined similarly to the resolution of the reference frame; for example, frame 1 can have a smaller resolution WS×HS, frame 2 can have the same resolution W×H as the reference frame, and frame 3 can have a larger resolution WL×HL. The width and height of the smaller and larger frames can be obtained by multiplying the reference width and height by an arbitrary rescaling constant (Rc), also known as a scaling factor and / or constant. In the case of the smaller frame, Rc can have a value between 0 and 1. In the case of the larger frame, Rc can have a value greater than 1; for example, Rc can have a value between 1 and 4. Other values ​​are also possible. The rescaling constant for one resolution dimension may differ from that for another resolution dimension; for example, the rescaling constant Rch can be used to rescale the height, while another rescaling constant Rcw can be used to rescale the width.

[0020] Still referencing Figure 1 ARM can be implemented as a mode. If ARM mode is activated at some point during decoding, the decoder may have already received a reference frame with a resolution of W×H and can rescale the predicted frame using a rescaling constant. In some implementations, the encoder can signal to the decoder which rescaling constant to use. Signaling can be performed in the Sequence Parameter Set (SPS) corresponding to the GOP containing the current image and / or the Image Parameter Set (PPS) corresponding to the current image. For example, but not limited to, the encoder can use fields such as pps_pic_width_in_luma_samples, pps_pic_height_in_luma_samples, pps_scaling_win_left_offset, pps_scaling_win_right_offset, pps_scaling_win_top_offset, pps_scaling_win_bottom_offset, and / or sps_num_subpics_minus1 to signal the rescaled parameters.

[0021] Further reference Figure 1The W and H parameters mentioned above can be represented, but are not limited to, by the variables CurrPicScalWinWidthL and CurrPicScalWinHeightL, respectively; these variables can be derived from the signal parameters described above using one or more mathematical relationships between the signal parameters and the variables. For example, but not limited to, CurrPicScalWinWidthL can be derived from the following equation: CurrPicScalWinWidthL=pps_pic_width_in_luma_samples–SubWidthC*(pps_scaling_win_right_offset+pps_scaling_win_left_offset) As another non-restrictive example, CurrPicScalWinHeightL can be derived from the following equation: CurrPicScalWinWidthL=pps_pic_width_in_luma_samples–SubWidthC*(pps_scaling_win_right_offset+pps_scaling_win_left_offset) Those skilled in the art, upon reading the full contents of this disclosure, will understand the various alternative calculations that can be used to derive the aforementioned variables. The encoder may alternatively or additionally signal one or more such variables Rc, Rch, and / or Rcw directly, for example, but not limited to, in the PPS and / or SPS.

[0022] Alternatively or additionally, refer to Figure 1 The rescaling constant and / or a set of rescaling constants, as described above, can be signaled in the bitstream using references to the indices of one or more stored scaling constants and / or frames and / or blocks that were previously signaled and / or used with one or more scaling constants. The references to the indices of the stored scaling constants can be explicitly signaled and / or determined from one or more additional parameters signaled in the bitstream. For example, but not limited to, the decoder can identify a reference frame and / or image group containing the current frame; where a rescaling constant has been previously signaled and / or used in such an image group, signaling that the reference frame can be applied to the current frame and / or current image group, etc., the decoder can identify that rescaling constant to use as the rescaling constant for the current frame.

[0023] In some implementations, reference continues. Figure 1ARM operations can be performed at the block level of the encoded frame. For example, a reference frame can be rescaled first, followed by prediction, such as... Figure 2 As shown. Figure 2 This is an illustration depicting a reference frame, a rescaled reference frame, and the subsequent block prediction process. The block prediction process can be performed on the scaled reference frame (with scaled resolution) instead of the original reference frame. As described above, the rescaled reference frame can include rescaling based on any parameters signaled by the encoder; for example, but not limited to, in the case of a reference frame signaled to be used with the current frame, such as by referencing an index value associated with the reference frame, the signaled reference frame can be rescaled according to any of the rescaling methods described above before prediction. The rescaled reference frame can be stored in memory and / or a buffer, which can include, but is not limited to, a buffer that identifies the frames contained therein by an index, based on which frame retrieval can be performed; the buffer can include a decoded image buffer (DCB) and / or one or more additional buffers implemented by the decoder. The prediction process can include, for example, inter-picture prediction including motion compensation.

[0024] Some implementations of block-based ARM allow for the flexibility to apply optimal filters to each block, rather than applying the same filters to the entire frame. In some implementations, a skip-ARM mode is possible, allowing some blocks (e.g., based on pixel uniformity and bitrate cost) to be in skip-ARM mode (so that rescaling does not change the bitrate). The skip-ARM mode can be signaled in the bitstream; for example, but not limited to, the skip-ARM mode can be signaled in the PPS parameter. Alternatively or additionally, the decoder can determine whether the skip-ARM mode is active based on one or more parameters set by the decoder and / or signaled in the bitstream. Spatial filters used in block-based ARM can include, but are not limited to, bicubic spatial filters applying bicubic interpolation, bilinear spatial filters applying bilinear interpolation, resampling Lanczos filters using Lanczos filtering and / or combinations of sine filters, sinc function interpolation and / or signal reconstruction techniques; various filters consistent with this disclosure will be understood by those skilled in the art upon reading the full contents of this disclosure.

[0025] Continue to refer to Figure 2 With the frame sequence already encoded, post-coded ARM allows for rapid rescaling and transmission of predicted coding units (blocks or frames). This eliminates the need to re-encode video frames at different resolutions. Some implementations of the present topic can reduce computational complexity at a relatively negligible cost in terms of rate-distortion performance.

[0026] Further referenceFigure 2 Depending on the use case, rescaling can be performed on the encoder side (to reduce bandwidth for scaling) or on the decoder side (to reduce the bandwidth required for video transmission again for scaling) during the encoding and decoding process from the encoder to the decoder. In some implementations, scaling can be performed using interpolation filters such as subpixel prediction filters. As a non-limiting example, the interpolation filter can include any of the filters described above, a low-pass filter, which can be used by, but is not limited to, an upsampling process, thereby initializing pixels between pixels in the block and / or frame before scaling to zero and then filling them with the output of the low-pass filter. Alternatively or additionally, any luma sample interpolation filtering process can be used. Luma sample interpolation can include calculating an interpolated value at a half-sample interpolation filter index that falls between two consecutive sample values ​​in the unscaled sample array. The calculation of the interpolation value can be performed by retrieving coefficients and / or weights from a lookup table, but is not limited to this; the choice of lookup table can be performed based on the motion model of the coding unit and / or the scaling factor, for example, using the scaling constants described above. The calculation can include, but is not limited to, performing a weighted summation of adjacent pixel values, where the weights are retrieved from a lookup table. Alternatively or additionally, the calculated value can be shifted; for example, but not limited to, the value can be shifted to Min (4, bit depth - 8), 6, Max (2, 14 bit depth), etc. Those skilled in the art will recognize various alternative or additional implementations that can be used for interpolation filters after reading the full contents of this disclosure.

[0027] In this method, reference is still made. Figure 2 After prediction, elements can be rescaled using a coefficient Rc (scaling constant or coefficient), which can be signaled to the decoder, as described above. For example, but not limited to, a pps_scaling_window_explicit_signalling_flag of 1 indicates the presence of a scaling window offset parameter in the PPS, while a pps_scaling_window_explicit_signalling_flag of 0 indicates its absence. When sps_ref_pic_resampling_enabled_flag is 0, pps_scaling_window_explicit_signalling_flag can also be 0.

[0028] Scaling elements can contain intra-frame and inter-frame coding units (e.g., blocks), which can use, for example... Figure 3 The coefficients described in the figure are scaled to rescale. Figure 3 This is an illustration depicting the positions of an exemplary reference block before and after the scaling process.

[0029] Scaling elements can include motion vectors. Figure 4 This is an illustration depicting an exemplary motion vector scaling that includes a reference block, a prediction block, a scaled reference block, and a scaled prediction block. Figure 4 The motion compensation predictions for the original-size reference frame (top) and the scaled-down reference frame (bottom) are shown.

[0030] Figure 5 This is a process flowchart illustrating an exemplary embodiment of an adaptive resolution management process 500, which can enable additional activity in the video encoder / decoder, thereby allowing bitrate savings in a variety of use cases.

[0031] At step 505, still refer to Figure 5 The decoder receives a reference frame. The reference frame can be received from memory (e.g., accessed from memory). The reference frame may have been previously decoded from the bitstream. The decoder may have already received the bitstream. The bitstream may include the current block. The current block may be included in the bitstream received by the decoder. The bitstream may include, for example, data found in the bitstream that is the input to the decoder when data compression is used. The bitstream may include information required for decoding the video. Receiving may include extracting and / or parsing blocks from the bitstream and associated signaling information. In some implementations, the current block may include a coding tree unit (CTU), a coding unit (CU), and / or a prediction unit (PU).

[0032] At step 510, continue to refer to Figure 5 The scaling constant is determined for the current block. In some implementations, the scaling constant can be determined based on the bitstream. For example, the scaling constant can be signaled in the bitstream, which may include an index of one of a plurality of predetermined scaling constants, etc.

[0033] At step 515, still refer to Figure 5 A scaled reference block can be determined using a reference frame and a scaling constant. Determining the scaled reference block may include determining its position. In some implementations, the position may be determined by multiplying one or more dimensions of the reference block by one or more scaling factors; for example, the horizontal component of the reference block's position may be multiplied by a scaling factor and the vertical component of the reference block's position may be multiplied by a scaling factor.

[0034] At step 520, continue to refer to Figure 5 The scaled prediction block can be determined using a scaled reference block. In some implementations, determining the scaled prediction block may include determining scaled motion information. The scaled motion information may be determined at least by multiplying the horizontal component of the motion vector by a scaling constant and the vertical component of the motion vector by the scaling constant.

[0035] At step 525, still refer to Figure 5 The pixel data of the current block can be reconstructed using the rescaled predicted block.

[0036] Figure 6 This is a system block diagram illustrating an exemplary decoder 600 capable of implementing adaptive resolution-managed prediction rescaling as described in this disclosure. Decoder 600 may include an entropy decoding processor 604, an inverse quantization and inverse transform processor 608, a deblocking filter 612, a frame buffer 616, a motion compensation processor 620, and / or an intra-frame prediction processor 624.

[0037] During operation, still refer to Figure 6 The bitstream 628 can be received by the decoder 600 and input to the entropy decoding processor 604, which can entropy decode a portion of the bitstream into quantization coefficients. The quantization coefficients can be provided to the inverse quantization and inverse transform processor 608, which can perform inverse quantization and inverse transform to create a residual signal. This residual signal can be added to the output of the motion compensation processor 620 or the intra-frame prediction processor 624, depending on the processing mode. The outputs of the motion compensation processor 620 and the intra-frame prediction processor 624 can include block prediction based on previous decoded blocks. The sum of the prediction and the residual can be processed by the deblocking filter 612 and stored in the frame buffer 616.

[0038] Figure 7 This is a flowchart illustrating an exemplary embodiment of a process 700 for encoding video using adaptive resolution management. This process 700 can provide additional flexibility for the video encoder and / or decoder, thereby allowing bitrate savings in various use cases. At step 705, the video frame may undergo initial block segmentation, for example, using a tree-structured macroblock segmentation scheme, which may include dividing the image frame into CTUs and CUs.

[0039] At step 710, still refer to Figure 7 It can perform block-based adaptive resolution management, including resolution scaling of frames or portions thereof.

[0040] At step 715, continue to refer to Figure 7 Blocks can be encoded and included in the bitstream. For example, encoding can include using inter-frame prediction and intra-frame prediction modes.

[0041] Figure 8This is a system block diagram illustrating an exemplary video encoder 800 capable of implementing adaptive resolution-managed predictive rescaling as described in this disclosure. The exemplary video encoder 800 may receive input video 804, which may be initially segmented or partitioned according to a processing scheme such as a tree-structured macroblock segmentation scheme (e.g., quadtree plus binary tree). Examples of a tree-structured macroblock segmentation scheme may include dividing an image frame into large blocks called coding tree units (CTUs). In some implementations, each CTU may be further segmented once or multiple times into multiple sub-blocks called coding units (CUs). The result of such segmentation may include a set of sub-blocks called prediction units (PUs). Transform units (TUs) may also be used.

[0042] Still referencing Figure 8 An exemplary video encoder 800 may include an intra-frame prediction processor 808, a motion estimation / compensation processor 812 (also referred to as an inter-frame prediction processor) capable of constructing a list of motion vector candidates (including adding global motion vector candidates to the list of motion vector candidates), a transform / quantization processor 816, an inverse quantization / inverse transform processor 820, a loop filter 824, a decoded image buffer 828, and / or an entropy coding processor 832. Bitstream parameters may be input to the entropy coding processor 832 to be included in the output bitstream 836.

[0043] Continue to refer to the instructions during operation. Figure 8 For each block of a frame in the input video 804, it can be determined whether the block will be processed via intra-frame prediction or motion estimation / compensation. The block can be provided to either the intra-frame prediction processor 808 or the motion estimation / compensation processor 812. If the block is to be processed via intra-frame prediction, the intra-frame prediction processor 808 can perform processing to output a predictor. If the block is to be processed via motion estimation / compensation, the motion estimation / compensation processor 812 can perform processing, if applicable, including constructing a list of motion vector candidates, including adding global motion vector candidates to the list.

[0044] Further reference Figure 8The residual can be formed by subtracting the prediction factor from the input video. The residual is received by a transform / quantization processor 816, which can perform transform processing (e.g., Discrete Cosine Transform (DCT)) to produce coefficients that can be quantized. The quantization coefficients and any associated signaling information can be provided to an entropy coding processor 832 for entropy coding and included in the output bitstream 836. The entropy coding processor 832 can support encoding signaling information related to encoding the current block. Furthermore, the quantization coefficients can be provided to an inverse quantization / inverse transform processor 820, which can reproduce pixels that can be combined with the prediction factor and processed by a loop filter 824. The output of the loop filter 824 can be stored in a decoded image buffer 828 for use by a motion estimation / compensation processor 812, which is capable of constructing a motion vector candidate list, including adding global motion vector candidates to the motion vector candidate list.

[0045] Continue to refer to Figure 8 Although some variations have been described in detail above, other modifications or additions are possible. For example, in some implementations, the current block may include any symmetrical block (8×8, 16×16, 32×32, 64×64, 128×128, etc.) and any asymmetrical block (8×4, 16×8, etc.).

[0046] In some implementations, reference is still made to Figure 8 Quadtree plus binary decision tree (QTBT) can be implemented. For QTBT, at the encoding tree unit level, the splitting parameters of QTBT can be dynamically derived to adapt to local features without any transmission overhead. Subsequently, at the encoding unit level, the joint classifier decision tree structure can eliminate unnecessary iterations and control the risk of incorrect predictions. In some implementations, the LTR frame block update mode can be used as an additional option available at each leaf node of the QTBT.

[0047] In some implementations, reference is still made to Figure 8 Signals can be used to notify other syntax elements at different levels of the bitstream. For example, an enable flag can be included in the Sequence Parameter Set (SPS) to enable the entire sequence. Furthermore, CTU flags can be encoded at the Code Tree Unit (CTU) level.

[0048] Some embodiments may include a non-transitory computer program product (i.e., a physically implemented computer program product) containing instructions that, when executed by one or more data processors of one or more computing systems, cause at least one data processor to perform the operations described herein.

[0049] The embodiments disclosed herein may include a decoder having circuitry configured to: receive a reference frame, determine a scaling constant for a current block, determine a scaled reference block using the reference frame and the scaling constant, determine a scaled prediction block using the scaled reference block, and reconstruct pixel data of the current block using the scaled prediction block.

[0050] In some embodiments, the decoder may be further configured to determine the scaled prediction block by determining scaled motion information. The scaled motion information may be determined at least by multiplying the horizontal component of the motion vector by the scaling constant and the vertical component of the motion vector by the scaling constant. The decoder may be further configured to determine the scaled reference block by determining the position of the scaled reference block. The position may be determined by multiplying the horizontal component of the reference block's position by a scaling factor and the vertical component of the reference block's position by the scaling factor. The decoder may be further configured to: receive a bitstream and determine the scaling constant based on the bitstream, wherein the scaling constant is signaled in the bitstream. The bitstream may include an index of a predetermined scaling constant. The decoder may be further configured to receive the reference frame by accessing the reference frame from memory. The decoder may include an entropy decoding processor configured to receive the bitstream and decode the bitstream into quantization coefficients; an inverse quantization and inverse transform processor configured to process the quantization coefficients, the processing of the quantization coefficients including performing an inverse discrete cosine transform; a deblocking filter; a frame buffer; and an intra-frame prediction processor. The current block may include a coding tree unit. The current block may include a coding unit. The current block may include a prediction unit.

[0051] The embodiments disclosed herein may include a method. The method may include: receiving a reference frame; determining a scaling constant for a current block; using the reference frame and the scaling constant to determine a scaled reference block; using the scaled reference block to determine a scaled prediction block; and reconstructing pixel data of the current block using the scaled prediction block.

[0052] In some embodiments, determining the scaled prediction block may include determining scaled motion information. The scaled motion information may be determined at least by multiplying the horizontal component of the motion vector by the scaling constant and the vertical component of the motion vector by the scaling constant. Determining the scaled reference block may include determining the position of the scaled reference block. The position may be determined by multiplying the horizontal component of the reference block's position by a scaling factor and the vertical component of the reference block's position by the scaling factor. The method may include: receiving a bitstream and determining the scaling constant based on the bitstream, wherein the scaling constant is signaled in the bitstream. The bitstream may include an index of a predetermined scaling constant. Receiving the reference frame may include accessing the reference frame from memory. At least one of the receiving, determining, and reconstructing may be performed by a decoder, the decoder including an entropy decoding processor configured to receive the bitstream and decode the bitstream into quantization coefficients; an inverse quantization and inverse transform processor configured to process the quantization coefficients, the processing of the quantization coefficients including performing an inverse discrete cosine transform; a deblocking filter; a frame buffer; and an intra-frame prediction processor. The current block may include a coding tree unit. The current block may include a coding unit. The current block may include a prediction unit.

[0053] It should be noted that any one or more aspects and embodiments described herein can be readily implemented using digital electronic circuits, integrated circuits, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), computer hardware, firmware, software, and / or combinations thereof, as implemented and / or carried out in one or more machines programmed according to the teachings of this specification (e.g., one or more computing devices used as user computing devices for electronic documents, one or more server devices such as document servers, etc.), as will be apparent to those skilled in the art of computers. These various aspects or features may include implementations in one or more computer programs and / or software executable and / or interpretable on a programmable system, said programmable system including at least one programmable processor, which may be dedicated or general-purpose, coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to send data and instructions to the storage system, at least one input device, and at least one output device. It will be apparent to those skilled in the art of software that a skilled programmer can readily prepare appropriate software code based on the teachings of this disclosure. The aspects and implementation methods of employing software and / or software modules discussed above may also include appropriate hardware for assisting in the implementation of machine-executable instructions for the software and / or software modules.

[0054] Such software can be a computer program product employing a machine-readable storage medium. A machine-readable storage medium can be any medium capable of storing and / or encoding a sequence of instructions executable by a machine (e.g., a computing device) and causing the machine to perform any of the methods and / or embodiments described herein. Examples of machine-readable storage media include, but are not limited to, magnetic disks, optical disks (e.g., CDs, CD-Rs, DVDs, DVD-Rs, etc.), magneto-optical disks, read-only memory "ROM" devices, random access memory "RAM" devices, magnetic cards, optical cards, solid-state storage devices, EPROMs, EEPROMs, programmable logic devices (PLDs), and / or any combination thereof. The use of machine-readable media herein is intended to include both single media and collections of physically separate media, such as collections of optical disks, or one or more hard disk drives combined with computer memory. Machine-readable storage media as used herein do not include temporary forms of signal transmission.

[0055] Such software may also include information (e.g., data) carried as data signals on a data carrier such as a carrier wave. For example, machine-executable information may be included as data-bearing signals contained in a data carrier, wherein the signals encode a sequence of instructions or portions thereof executed by a machine (e.g., a computing device), and any relevant information (e.g., data structures and data) that causes the machine to perform any of the methods and / or embodiments described herein.

[0056] Examples of computing devices include, but are not limited to, e-book readers, computer workstations, terminal computers, server computers, handheld devices (e.g., tablets, smartphones, etc.), network devices, network routers, network switches, bridges, any machine capable of executing a sequence of instructions specifying the action it is to take, and any combination of the foregoing. In one example, a computing device may include and / or be included in a kiosk.

[0057] Figure 9 The illustration shows an embodiment of a computing device in an exemplary form of a computer system 900, wherein a set of instructions can be executed to cause a control system to perform any one or more aspects and / or methods of this disclosure. It is also contemplated that a specially configured set of instructions can be implemented using multiple computing devices to cause one or more of the devices to perform any one or more aspects and / or methods of this disclosure. The computer system 900 includes a processor 904 and a memory 908, which communicate with each other and with other components via a bus 912. The bus 912 can include any of a variety of bus structures using any of a variety of bus architectures, including but not limited to a memory bus, a memory controller, a peripheral bus, a local bus, and any combination thereof.

[0058] Memory 908 may include various components (e.g., machine-readable media), including but not limited to random access memory components, read-only components, and any combination thereof. In one example, a basic input / output system 916 (BIOS) may be stored in memory 908, the BIOS including basic routines such as those that facilitate the transfer of information between elements within computer system 900 during startup. Memory 908 may also include (e.g., stored on one or more machine-readable media) instructions (e.g., software) 920 embodying any one or more aspects and / or methods of this disclosure. In another example, memory 908 may further include any number of program modules, including but not limited to an operating system, one or more application programs, other program modules, program data, and any combination thereof.

[0059] Computer system 900 may also include storage device 924. Examples of storage devices (e.g., storage device 924) include, but are not limited to, hard disk drives, disk drives, optical disc drives combined with optical media, solid-state storage devices, and any combination thereof. Storage device 924 may be connected to bus 912 via a suitable interface (not shown). Exemplary interfaces include, but are not limited to, SCSI, Advanced Technology Attachment (ATA), Serial ATA, Universal Serial Bus (USB), IEEE 1394 (FireWire), and any combination thereof. In one example, storage device 924 (or one or more components thereof) may be removably connected to computer system 900 (e.g., via an external port connector (not shown)). In particular, storage device 924 and associated machine-readable medium 928 may provide non-volatile and / or volatile storage of machine-readable instructions, data structures, program modules, and / or other data for computer system 900. In one example, software 920 may be stored wholly or partially within machine-readable medium 928. In one example, software 920 may be stored wholly or partially within processor 904.

[0060] Computer system 900 may also include input device 932. In one example, a user of computer system 900 may input commands and / or other information into computer system 900 via input device 932. Examples of input device 932 include, but are not limited to: alphanumeric input devices (e.g., keyboard), pointing devices, joysticks, game controllers, audio input devices (e.g., microphones, voice response systems, etc.), cursor control devices (e.g., mice), touchpads, optical scanners, video capture devices (e.g., still cameras, camcorders), touchscreens, and any combination thereof. Input device 932 may be connected to bus 912 via any of a variety of interfaces (not shown), including but not limited to serial interfaces, parallel interfaces, game ports, USB interfaces, FireWire interfaces, direct interfaces to bus 912, and any combination thereof. Input device 932 may include a touchscreen interface, which may be part of or separate from display 936, as will be discussed further below. Input device 932 may be used as a user selection device for selecting one or more graphical representations in a graphical interface as described above.

[0061] Users can also input commands and / or other information to computer system 900 via storage device 924 (e.g., removable disk drive, flash drive, etc.) and / or network interface device 940. Network interface devices such as network interface device 940 can be used to connect computer system 900 to one or more of various networks such as network 944, and one or more remote devices 948 connected to network 944. Examples of network interface devices include, but are not limited to, network interface cards (e.g., mobile network interface cards, LAN cards), modems, and any combination thereof. Examples of networks include, but are not limited to, wide area networks (e.g., the Internet, corporate networks), local area networks (e.g., networks associated with offices, buildings, campuses, or other relatively small geographical areas), telephone networks, data networks associated with telephone / voice providers (e.g., mobile communication provider data and / or voice networks), direct connections between two computing devices, and any combination thereof. Networks such as network 944 can employ wired and / or wireless communication modes. Typically, any network topology can be used. Information (e.g., data, software 920, etc.) can be transmitted to and / or from computer system 900 via network interface device 940.

[0062] Computer system 900 may further include a video display adapter 952 for transmitting displayable images to a display device such as display device 936. Examples of display devices include, but are not limited to, liquid crystal displays (LCDs), cathode ray tube (CRTs), plasma displays, light-emitting diode (LED) displays, and any combination thereof. Display adapter 952 and display device 936 may be used in conjunction with processor 904 to provide graphical representations of various aspects of this disclosure. In addition to display devices, computer system 900 may include one or more other peripheral output devices, including, but not limited to, audio speakers, printers, and any combination thereof. Such peripheral output devices may be connected to bus 912 via peripheral interface 956. Examples of peripheral interfaces include, but are not limited to, serial ports, USB connections, FireWire connections, parallel connections, and any combination thereof.

[0063] Exemplary embodiments of the present invention have been described in detail above. Various modifications and additions may be made without departing from the spirit and scope of the invention. To provide multiple combinations of features in relevant new embodiments, features of each of the various embodiments described above may be appropriately combined with features of other described embodiments. Furthermore, while many individual embodiments have been described above, the content described herein is merely illustrative of the application of the principles of the invention. Moreover, although specific methods herein may be shown and / or described as being performed in a particular order, the order is highly variable in common techniques for implementing the embodiments disclosed herein. Therefore, this description is intended only as an example and not as limiting the scope of the invention.

[0064] In the foregoing description and claims, phrases such as "at least one" or "one or more" may be followed by a list of connected elements or features. The term "and / or" may also appear in a list of two or more elements or features. Unless implicitly or explicitly contradicting the context in which the phrase is used, such phrases are intended to mean any element or feature listed individually, or a combination of any listed element or feature with any other listed element or feature. For example, the phrases "at least one of A and B," "one or more of A and B," and "A and / or B" are intended to mean "A alone, B alone, or both A and B," respectively. A similar interpretation applies to lists containing three or more items. For example, the phrases "at least one of A, B, and C," "one or more of A, B, and C," and "A, B, and / or C" are intended to mean "A alone, B alone, C alone, both A and B, both A and C, both B and C, or A and B and C," respectively. Furthermore, the use of the term "based on" in the foregoing and claims is intended to mean "at least partially based on," thus allowing for the inclusion of uncited features or elements.

[0065] The subject matter described herein can be embodied in systems, apparatus, methods, and / or articles according to desired configurations. The embodiments set forth in the foregoing description do not represent all embodiments consistent with the subject matter described herein. Rather, they are merely examples of aspects consistent with the described subject matter. Although some variations have been described in detail above, other modifications or additions are possible. In particular, further features and / or variations may be provided in addition to the features and / or variations set forth herein. For example, the above embodiments may be applicable to various combinations and sub-combinations of the disclosed features and / or combinations and sub-combinations of several other features disclosed above. Furthermore, the logical flows described in the drawings and / or described herein do not necessarily require a specific order or sequence to achieve the desired results. Other embodiments are within the scope of the claims.

Claims

1. An encoder configured as follows: Receive video signals; The video signal is processed to generate an encoded bitstream, the encoded bitstream including a reference frame, a current image having a resolution different from that of the reference frame and including a first block and a second block, and information for determining a scaling constant. The encoder also configures the bitstream to be decoded using a method including the following steps: The scaling constant is determined using the information in the bitstream; The scaled prediction block is determined from the reference frame using the following steps: The position of the scaled prediction block is determined by scaling the motion vector components using the scaling constant; and The resolution of the blocks of the reference frame is scaled using the scaling constant, and a filter determined for the scaled prediction blocks rather than for the entire reference frame is applied. The first block is reconstructed using the scaled prediction block; as well as The second block is reconstructed without using scaled prediction blocks.

2. The encoder according to claim 1, wherein, The information in the bitstream used to determine the scaling constant includes an index of a set of predetermined values, and the index is used to determine the scaling constant.

3. An encoder configured as follows: Receive video signals; The video signal is processed to generate an encoded bitstream, the encoded bitstream including a current encoded image, the current encoded image including a first plurality of coded blocks and a second plurality of coded blocks, and the encoder further configures the encoded bitstream to be decoded by a method including the following steps: Decoding each of the first plurality of coded blocks is performed using the following steps: The scaling constant is determined using the information in the bitstream; as well as The location of the scaled prediction block is determined by scaling the motion vector components, the block in the reference frame is scaled using the scaling constant, and a filter specifically determined for the scaled prediction block is applied to determine the scaled prediction block from the reference frame having a resolution different from that of the currently encoded image. Each of the first plurality of coding blocks is decoded using a scaled prediction block of each of the first plurality of coding blocks; as well as Decoding is performed on each of the second plurality of coded blocks without using the predicted blocks scaled by the reference frame and without applying the resolution-changing filter determined for the blocks of the reference frame.

4. The encoder according to claim 3, wherein, The information in the bitstream includes an index of a set of predetermined values, and the index is used to determine the scaling constant.

5. A video encoder having circuitry configured to receive a video signal and encode a bitstream, the bitstream including a reference frame, a current image having a different resolution from the reference frame and including a first block and a second block, and information for determining a scaling constant, the second block being an intra-frame coded block, the encoded bitstream being configured to be decoded by a method including the following steps: Determine whether block-based mode for decoding a current image at a different resolution using reference frames is enabled; The scaling constant is determined using the information in the bitstream; Using the enabled mode, the scaled prediction block is determined from the reference frame through the following steps: The position of the scaled prediction block is determined by scaling the motion vector components using the scaling constant; and The resolution of the blocks of the reference frame is scaled using the scaling constant, and a filter determined individually for the predicted block rather than for the entire reference frame is applied. The first block is reconstructed using the scaled prediction block; as well as The second block of intra-frame coding is reconstructed without using the aforementioned mode.

6. A video encoder having circuitry configured to encode a bitstream comprising a reference frame, a current image having a different resolution than the reference frame and comprising a first block and a second block, signaling information for determining whether to enable a block-based mode for decoding the current image at a resolution using the reference frame at a different resolution, and information for determining a scaling constant, the second block being an intra-frame coded block, the encoder further configuring the bitstream to be decoded by a method comprising the following steps: Receive the encoded bit stream; The scaling constant is determined using the information in the bitstream; Using the enabled mode, determine the scaled prediction block from the reference frame through the following steps: The position of the scaled prediction block is determined by scaling the motion vector components using the scaling constant; and The resolution of the blocks of the reference frame is scaled using the scaling constant, and a filter determined individually for the predicted block rather than for the entire reference frame is applied. The first block is reconstructed using the scaled prediction block; as well as The second block of intra-frame coding is reconstructed without using the aforementioned mode.

7. A non-transient computer-readable medium storing an encoded bitstream, the encoded bitstream comprising a reference frame, a current image having a different resolution from the reference frame and including a first block and a second block, and information for determining a scaling constant, the second block being an intra-frame coded block, the encoded bitstream being decodeable by a video decoder configured to: Receive the encoded bit stream; Determine whether block-based mode for decoding a current image at a different resolution using reference frames of different resolutions is enabled; The scaling constant is determined using the information in the bitstream; Using the enabled mode, the scaled prediction block is determined from the reference frame through the following steps: The position of the scaled prediction block is determined by scaling the motion vector components using the scaling constant; and The resolution of the blocks of the reference frame is scaled using the scaling constant, and a filter determined individually for the predicted block rather than for the entire reference frame is applied. The first block is reconstructed using the scaled prediction block; as well as The second block of intra-frame coding is reconstructed without using the aforementioned mode.

8. A non-transient computer-readable medium storing an encoded bitstream, the encoded bitstream comprising a reference frame, a current image having a different resolution than the reference frame and including a first block and a second block, signaling information for determining whether to enable a block-based mode for decoding the current image at a resolution using the reference frame at a different resolution, and information for determining a scaling constant, the second block being an intra-frame coded block, the encoded bitstream being decodeable by a video decoder configured to: Receive the encoded bit stream; The scaling constant is determined using the information in the bitstream; Using the enabled mode, the scaled prediction block is determined from the reference frame through the following steps: The position of the scaled prediction block is determined by scaling the motion vector components using the scaling constant; and The resolution of the blocks of the reference frame is scaled using the scaling constant, and a filter determined individually for the predicted block rather than for the entire reference frame is applied. The first block is reconstructed using the scaled prediction block; as well as The second block of intra-frame coding is reconstructed without using the aforementioned mode.

Images