Implicit identification of frame type based adaptive resolution management

Abstract

A method includes receiving a bitstream, using the bitstream and determining, for a current frame, whether the current frame is usable as a reference frame, responsive to determining that the current frame is usable as a reference frame, setting a variable characterizing that an adaptive resolution management mode is not allowed, and reconstructing pixel data of the current frame, wherein the adaptive resolution management mode is not allowed. Related apparatuses, systems, techniques, and articles of manufacture are also described.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 883,484, filed August 6, 2019, entitled "Implicit SIGNALING OF ADAPTIVE RESOLUTION MANAGEMENT BASED ON FRAMETYPE", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This invention generally relates to the field of video compression. Specifically, this invention relates to implicit identifiers for adaptive resolution management based on frame type. Background Technology

[0004] Video codecs can include electronic circuitry or software that compresses or decompresses digital video. They can convert uncompressed video to a compressed format and vice versa. In the context of video compression, devices that compress video (and / or perform some of its functions) are generally referred to as encoders, and devices that decompress video (and / or perform some of its functions) are referred to as decoders.

[0005] The compressed data can conform to standard video compression specifications. Compression may be lossy because the compressed video may lack some information present in the original video. This can result in the decompressed video having lower quality than the original uncompressed video because there isn't enough information to accurately reconstruct the original video.

[0006] There can be complex relationships between video quality, the amount of data used to represent the video (e.g., determined by bit rate), the complexity of encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access, end-to-end latency (e.g., delay time), and so on.

[0007] Motion compensation can include methods for predicting a portion of a video frame or a given reference frame (e.g., a previous frame and / or a future frame) by taking into account the motion of objects in the camera and / or video. The methods can encode and decode video data for video compression, for example, using the Moving Picture Experts Group (MPEG) High-Level Video Coding (AVC) standard (also known as H.264).

[0008] Motion compensation can describe an image based on the transformation from a reference image to the current image. When compared with the current image, the reference image can be temporally previous, and / or, when compared with the current image, the reference image can be from the future. Summary of the Invention

[0009] On one hand, a decoder includes circuitry configured to: receive a bitstream; use the bitstream and for a current frame to determine whether the current frame is available as a reference frame; in response to determining that the current frame is available as a reference frame, set a variable characterizing that an adaptive resolution management mode is not allowed; and reconstruct pixel data of the current frame, wherein the adaptive resolution management mode is not allowed.

[0010] On the other hand, one method includes: receiving a bitstream, using the bitstream and for a current frame, determining whether the current frame is usable as a reference frame, in response to determining that the current frame is usable as a reference frame, setting a variable characterizing that an adaptive resolution management mode is not allowed, and reconstructing pixel data of the current frame, wherein the adaptive resolution management mode is not allowed.

[0011] Details of one or more variations of the subject matter described herein are set forth in the accompanying drawings and detailed descriptions. Further features and advantages of the subject matter described herein will become apparent from the detailed descriptions, the drawings, and the claims. Attached Figure Description

[0012] To illustrate the invention, the accompanying drawings show various aspects of one or more embodiments of the invention. However, it should be understood that the invention is not limited to the precise arrangements and means shown in the drawings, in which:

[0013] Figure 1 These are illustrations of example reference frames and example predicted frames for various resolution levels;

[0014] Figure 2 It is a diagram depicting the prediction process of the example reference frame, the example rescaled reference frame, and the example subsequent block;

[0015] Figure 3 This is a process flowchart illustrating example processes according to some implementations of the current topic;

[0016] Figure 4 This is a system block diagram illustrating an example decoder capable of decoding bitstreams according to some implementations of the present topic;

[0017] Figure 5 This is a flowchart illustrating an example process for encoding video according to some implementations of the current topic;

[0018] Figure 6 This is a system block diagram illustrating an example video encoder according to some implementations of the current topic; and

[0019] Figure 7It is a block diagram of a computational system that can be used to implement any one or more methods disclosed herein and any one or more parts thereof.

[0020] The accompanying drawings are not necessarily drawn to scale and may use dashed lines, schematic diagrams, and partial views for illustration. In some cases, details that are unnecessary for understanding the embodiments or that make other details difficult to understand may have been omitted. In the drawings, the same reference numerals refer to the same elements. Detailed Implementation

[0021] In many of today's state-of-the-art encoders, resolution is managed by re-encoding and retransmitting entire portions of video known as Groups of Pictures (GOPs). This requires sending internal frames (I-frames), which can result in additional cost because these frames are responsible for most of the bits in the GOP.

[0022] The embodiments described in this disclosure relate to Adaptive Resolution Management (ARM), a technique that provides additional flexibility to video encoders / decoders, thereby allowing bitrate savings in a variety of use cases. Typically, ARM involves performing prediction using a reference frame with a different resolution than the current frame. In current coding standards, the reference frame has the same resolution as the predicted frame. In ARM, the resolution of the reference frame may be less than or greater than the resolution of the frame being predicted. This approach can be used to reduce video resolution, thereby reducing the bitrate, or to increase video resolution, thereby improving the display characteristics of video playback.

[0023] For the purposes of this disclosure, ARM may be alternatively or equivalently referred to as Reference Image Resampling (RPR); RPR and ARM are used interchangeably.

[0024] Some implementations of the current topic may include using ARM for any number of frames at any location within a GOP, thereby eliminating the need for I-frame re-encoding.

[0025] Figure 1 This is a diagram illustrating reference and predicted frames at various resolution levels. Frame 1 is smaller than the reference frame (lower resolution), frame 2 is the same size as the reference frame (same resolution), and frame 3 is larger than the reference frame (higher resolution). As used in this disclosure, "resolution" refers to the number of pixels in an image, frame, subframe, and / or other display area or portion thereof used in video playback, compression, etc., with a higher number of pixels corresponding to a higher resolution and a lower number of pixels corresponding to a lower resolution. Resolution can be measured by area, for example, but not limited to, by using one or more length dimensions to define the area in pixels. For example, a circular subframe or other area may have a resolution defined by its radius. Alternatively or additionally, resolution may be defined by the total number of pixels.

[0026] As an example, continue to refer to Figure 1 The reference frame and / or subframe have a geometry whose area can be defined entirely by two length parameters, such as, but not limited to, triangles, parallelograms, and / or rectangles. The reference frame and / or subframe can have a resolution W×H, where W and H can indicate the number of pixels describing the width (or bottom) and height dimensions of the reference frame and / or subframe, respectively. Each predicted frame can also have a resolution that can be determined similarly to the resolution of the reference frame; for example, frame 1 can have a smaller resolution WS×HS, frame 2 can have the same resolution W×H as the reference frame, and frame 3 can have a larger resolution WL×HL. The width and height of the smaller and larger frames can be obtained by multiplying the reference width and height by an arbitrary rescaling constant (Rc), also known as a scaling factor and / or constant. In the case of smaller frames, Rc can have values ​​between 0 and 1. In the case of larger frames, Rc can have values ​​greater than 1; for example, Rc can have values ​​between 1 and 4. Other values ​​are also possible. The rescaling constant for one resolution dimension may differ from that for another resolution dimension; for example, the rescaling constant Rch can be used to rescale the height, while another rescaling constant Rcw can be used to rescale the width.

[0027] Still referencing Figure 1 ARM can be implemented as a mode. If ARM mode is activated at some point during decoding, the decoder may have already received a reference frame with a resolution of W×H and can rescale the predicted frame using a rescaling constant. In some implementations, the encoder can identify to the decoder which rescaling constant to use. This identification can be performed in the Sequence Parameter Set (SPS) corresponding to the GOP containing the current image and / or in the Image Parameter Set (PPS) corresponding to the current image. For example, but not limited to, the encoder can use fields such as pps_pic_width_in_luma_samples, pps_pic_height_in_luma_samples, pps_scaling_win_left_offset, pps_scaling_win_right_offset, pps_scaling_win_top_offset, pps_scaling_win_bottom_offset, and / or sps_num_subpics_minus1 to identify the rescaled parameters.

[0028] Further reference Figure 1The W and H parameters mentioned above can, but are not limited to, be represented by the variables CurrPicScalWinWidthL and CurrPicScalWinHeightL, respectively; these variables can be derived from the identified parameters as described above using one or more mathematical relationships representing the relationship between the parameters and the variables. For example, but not limited to, CurrPicScalWinWidthL can be derived from the following equation:

[0029] CurrPicScalWinWidthL=pps_pic_width_in_luma_samples–

[0030] SubWidthC*(pps_scaling_win_right_offset+pps_scaling_win_left_offset)

[0031] As another non-restrictive example, CurrPicScalWinHeightL can be derived from the following equation:

[0032] CurrPicScalWinWidthL=pps_pic_width_in_luma_samples–

[0033] SubWidthC*(pps_scaling_win_right_offset+pps_scaling_win_left_offset)

[0034] Those skilled in the art will understand, upon reading the full contents of this disclosure, that various alternative calculations can be used to derive the aforementioned variables. The encoder may, alternatively or additionally, directly, for example, but not limited to, identifying one or more such variables Rc, Rch, and / or Rcw in the PPS and / or SPS.

[0035] Alternatively or additionally, refer to Figure 1The rescaling constants and / or a set of rescaling constants described above can be identified in the bitstream using references to the indices of one or more stored scaling constants and / or frames and / or blocks, which are identified using one or more previously identified and / or used scaling constants. The references to the indices of the stored scaling constants can be explicitly identified and / or determined from one or more additional parameters identified in the bitstream. For example, but not limited to, the decoder can identify a reference frame and / or image group containing the current frame; where rescaling constants have been previously identified and / or used in such image groups, the identified reference frame can be applied to the current frame and / or current image group, etc., and the decoder can identify the rescaling constant to use as the rescaling constant for the current frame.

[0036] In some implementations, reference continues. Figure 1 ARM operations can be performed at the block level of the encoded frame. For example, a reference frame can be rescaled first, followed by prediction, such as... Figure 2 As shown. Figure 2 This is an illustration depicting a reference frame, a rescaled reference frame, and the subsequent block prediction process. The block prediction process can be performed on the scaled reference frame (with scaled resolution) instead of the original reference frame. As described above, rescaling the reference frame can include rescaling according to any parameters identified by the encoder; for example, but not limited to, if a reference frame to be used with the current frame is identified, such as by referencing an index value associated with the reference frame, the identified reference frame can be rescaled according to any of the rescaling methods described above before prediction. The rescaled reference frame can be stored in memory and / or a buffer, which can include, but is not limited to, a buffer that identifies the frame contained therein by an index, according to which frame retrieval can be performed; the buffer can include a decoded picture buffer (DCB) and / or one or more additional buffers implemented by the decoder. The prediction process can include, for example, inter-picture prediction including motion compensation.

[0037] Some implementations of block-based ARM allow for the flexibility to apply optimal filters to each block, rather than applying the same filters to the entire frame. In some implementations, skip-ARM modes are possible, allowing some blocks (e.g., based on pixel uniformity and bitrate cost) to be in skip-ARM mode (so that rescaling does not change the bitrate). Skip-ARM modes can be identified in the bitstream; for example, but not limited to, skip-ARM modes can be identified in the PPS parameter. Alternatively or additionally, the decoder can determine whether the skip-ARM mode is active based on one or more parameters set by the decoder and / or identified in the bitstream. Spatial filters used in block-based ARM can include, but are not limited to, bicubic spatial filters applying bicubic interpolation, bilinear spatial filters applying bilinear interpolation, resampling Lanczos filters using Lanczos filtering and / or using sinc filters, sinc function interpolation and / or signal reconstruction techniques, etc.; various filters consistent with this disclosure will be understood by those skilled in the art after reading the entire disclosure. As a non-limiting example, the interpolation filter may include any filter as described above, a low-pass filter, which may be used through, but is not limited to, an upsampling process, wherein pixels between pixels of a block and / or frame before scaling may be initialized to zero and then filled with the output of the low-pass filter. Alternatively or additionally, any luminance sample interpolation filtering process may be used. Luminance sample interpolation may include calculating an interpolated value at a half-sample interpolation filter index that falls between two consecutive sample values ​​in an unscaled sample array. The calculation of the interpolated value may be performed by retrieving coefficients and / or weights from a lookup table, but is not limited to this; the selection of the lookup table may be performed as a function of the motion model and / or scaling factor of the coding unit, for example, using the scaling constants as described above. The calculation may include, but is not limited to, performing a weighted summation of adjacent pixel values, wherein the weights are retrieved from a lookup table. Alternatively or additionally, the calculated value may be shifted; for example, but not limited to, the value may be shifted to Min(4, bit depth - 8), 6, Max(2, 14 - bit depth), etc. Various alternative or additional implementations that may be used for the interpolation filter will be apparent to those skilled in the art after reading the full contents of this disclosure.

[0038] Still referencing Figure 2Adaptive resolution management allows for the rescaling of frames and / or regions of frames to improve compression efficiency and / or quality. A key challenge is the reproducibility of the rescaling method in the decoder. Depending on the rescaling method used, the decoder and encoder may not be able to reproduce the same scaled subsequent frames. In this case, using a rescaled frame as a reference frame can lead to prediction mismatch and / or encoder and decoder drift. Drift can become visible as coding artifacts (quality issues). To avoid these drift problems, implicit and explicit identifiers of ARM, as described in this disclosure, can be utilized.

[0039] For example, further reference Figure 2 The implicit identification of ARM can be based on frame type. For example, in some implementations, the decoder and / or encoder can be configured such that frames that can be used as reference frames cannot be encoded using ARM; in other words, ARM may be disallowed for frames that can be used as reference frames. For example, if the frame type or slice type identified in the image header indicates that a frame can be used as a reference frame, such as a frame identified as type I or P in the image header, then the decoder and / or encoder can determine that the frame cannot be encoded using ARM, and all ARM-related parameters may not be present in the image header of the frame. These parameters can be identified in the SPS and / or PPS, but are not limited to this. For example, the SPS can identify a parameter represented as `sps_res_change_in_clvs_allowed_flag`, which, when equal to 1, can indicate that the spatial resolution of the image may change. One aspect of resolution management can be that the resolution may change during the sequence. As another non-restrictive example, a `gci_no_ref_pic_resampling_constraint_flag` equal to 1 can specify that `sps_ref_pic_resampling_enabled_flag` will be equal to 0 for all images in OlsInScope. When `gci_no_ref_pic_resampling_constraint_flag` is equal to 0, such a constraint is not imposed. In the slice header, `sh_slice_type` can specify the encoding type for slices between I, P, and B types. A `ph_non_ref_pic_flag` equal to 1 can specify that the current image is never used as a reference image. A `ph_non_ref_pic_flag` equal to 0 can specify whether the current image can or cannot be used as a reference image.

[0040] Figure 3 This is a process flowchart illustrating an exemplary embodiment of an adaptive resolution management process 300, which enables additional flexibility for video encoders / decoders, thereby allowing bitrate savings in a variety of use cases.

[0041] At step 305, continue to refer to Figure 3 The decoder receives a bitstream. The current block may be included in the bitstream received by the decoder. The bitstream may, for example, include data found in the bitstream input to the decoder when data compression is used. The bitstream may include information required for decoding the video. Receiving may include extracting and / or parsing blocks from the bitstream and associated identification information. In some implementations, the current block may include a coding tree unit (CTU), a coding unit (CU), and / or a prediction unit (PU).

[0042] In step 310, still refer to Figure 3 The bitstream can be used to determine whether the current frame is usable as a reference frame. In some implementations, determining whether the current frame is usable as a reference frame includes determining whether the type of the current frame is type I or type P as described above. As a non-limiting example, in the slice header, sh_slice_type can specify the encoding type of the slice between types I, P, and B. The type of the current frame can be identified in the bitstream. In some implementations, as a non-limiting example, the current frame may include slices, defined as a set of CTUs and / or a set of an integer number of consecutive blocks within a block.

[0043] Still referencing Figure 3 At step 315, in response to determining that the current frame can be used as a reference frame, a variable characterizing that the adaptive resolution management mode is not allowed can be set. In some implementations, the parameters associated with the adaptive resolution management mode are not present in the image header of the current frame. As a non-limiting example, SPS can indicate that sps_res_change_in_clvs_allowed_flag is equal to 1, specifying that the spatial resolution of the image may change. Disabling based on slice type can be indicated at the encoder based on the image type.

[0044] In step 340, further refer to Figure 3 This allows for the reconstruction of the pixel data of the current frame. During reconstruction, adaptive resolution management mode may not be permitted. In one embodiment, determining that the current frame cannot be used as a reference frame can lead to determining that ARM is available. For example, SPS can indicate that sps_res_change_in_clvs_allowed_flag is equal to 1, specifying that the spatial resolution of the image may change, as described above.

[0045] Figure 4This is a system block diagram illustrating an example decoder 400 capable of implementing frame type-based adaptive resolution management as described in this disclosure. Decoder 400 may include an entropy decoding processor 404, an inverse quantization and inverse transform processor 408, a deblocking filter 412, a frame buffer 416, a motion compensation processor, and / or an intra-frame prediction processor 424.

[0046] During operation, still refer to Figure 4 The bitstream 428 can be received by the decoder 400 and input to the entropy decoding processor 404, which can entropy decode a portion of the bitstream into quantization coefficients. The quantization coefficients can be provided to the inverse quantization and inverse transform processor 408, which can perform inverse quantization and inverse transform to create a residual signal. This residual signal can be added to the output of the motion compensation processor 420 or the intra-prediction processor 424, depending on the processing mode. The outputs of the motion compensation processor 420 and the intra-prediction processor 424 can include block prediction based on previous decoded blocks. The sum of the prediction and the residual can be processed by the deblocking filter 412 and stored in the frame buffer 416.

[0047] Figure 5 This is a flowchart illustrating an exemplary process 500 for encoding video using adaptive resolution management. Process 500 can provide additional flexibility for the video encoder and / or decoder, thereby allowing bitrate savings in a variety of use cases. At step 505, the video frame may undergo initial block segmentation, for example, using a tree-structured macroblock segmentation scheme, which may include dividing the image frame into CTUs and CUs.

[0048] At step 510, block-based adaptive resolution management can be performed, including resolution scaling of frames or portions thereof.

[0049] At position 515, the block can be encoded and included in the bitstream. For example, encoding can include using inter-frame prediction and intra-frame prediction modes.

[0050] Figure 6 This is a system block diagram illustrating an example video encoder 600 capable of implementing frame type-based adaptive resolution management as described in this disclosure. The example video encoder 600 can receive input video 604, which can be initially segmented or partitioned according to a processing scheme such as a tree-structured macroblock segmentation scheme (e.g., quadtree plus binary tree). Examples of tree-structured macroblock segmentation schemes may include dividing image frames into large blocks called coding tree units (CTUs). In some implementations, each CTU may be further segmented once or multiple times into multiple sub-blocks called coding units (CUs). The result of such segmentation may include a set of sub-blocks called prediction units (PUs). Transform units (TUs) may also be used.

[0051] Still referencing Figure 6 An exemplary video encoder 600 may include an intra-frame prediction processor 608, a motion estimation / compensation processor 612 (also referred to as an inter-frame prediction processor) capable of constructing a motion vector candidate list, including adding global motion vector candidates to the motion vector candidate list, a transform / quantization processor 616, an inverse quantization / inverse transform processor 620, a loop filter 624, a decoded image buffer 628, and / or an entropy coding processor 632. Bitstream parameters may be input to the entropy coding processor 632 to be included in the output bitstream 636.

[0052] Continue to refer to the instructions during operation. Figure 6 For each block of a frame in the input video 604, it can be determined whether the block will be processed via intra-frame prediction or motion estimation / compensation. The block can be provided to either the intra-frame prediction processor 608 or the motion estimation / compensation processor 612. If the block is to be processed via intra-frame prediction, the intra-frame prediction processor 608 can perform processing to output a predictor. If the block is to be processed via motion estimation / compensation, the motion estimation / compensation processor 612 can perform processing, if applicable, including constructing a list of motion vector candidates, including adding global motion vector candidates to the list.

[0053] Further reference Figure 6 The residual can be formed by subtracting the prediction factor from the input video. The residual can be received by a transform / quantization processor 616, which can perform transform processing (e.g., Discrete Cosine Transform (DCT)) to produce coefficients that can be quantized. The quantized coefficients and any associated identification information can be provided to an entropy coding processor 632 for entropy coding and included in the output bitstream 636. The entropy coding processor 632 can support encoding identification information about the current block. Furthermore, the quantization coefficients can be provided to an inverse quantization / inverse transform processor 620, which can reproduce pixels that can be combined with the prediction factor and processed by a loop filter 624. The output of the loop filter 624 can be stored in a decoded image buffer 628 for use by a motion estimation / compensation processor 612, which is capable of constructing a motion vector candidate list, including adding global motion vector candidates to the motion vector candidate list.

[0054] Continue to refer to Figure 6Although some variations have been described in detail above, other modifications or additions are possible. For example, in some implementations, the current block may include any symmetrical block (8×8, 16×16, 32×32, 64×64, 128×128, etc.) and any asymmetrical block (8×4, 16×8, etc.).

[0055] In some implementations, reference is still made to Figure 6 Quadtree plus binary decision tree (QTBT) can be implemented. For QTBT, at the encoding tree unit level, the splitting parameters of QTBT can be dynamically derived to adapt to local features without any transmission overhead. Subsequently, at the encoding unit level, the joint classifier decision tree structure can eliminate unnecessary iterations and control the risk of mispredictions. In some implementations, the LTR frame block update mode can be used as an additional option available at each leaf node of the QTBT.

[0056] In some implementations, reference is still made to Figure 6 Other syntax elements can be identified at different levels of the bitstream. For example, an enable flag can be included for the entire sequence by including an encoded enable flag in the Sequence Parameter Set (SPS). Furthermore, CTU flags can be encoded at the Code Tree Unit (CTU) level.

[0057] Some embodiments may include a non-transitory computer program product (i.e., a physically implemented computer program product) containing instructions that, when executed by one or more data processors of one or more computing systems, cause at least one data processor to perform the operations described herein.

[0058] In some embodiments disclosed herein, the decoder may include circuitry configured to: receive a bitstream, use the bitstream and determine whether the current frame is available as a reference frame, in response to determining that the current frame is available as a reference frame, set a variable characterizing that an adaptive resolution management mode is not allowed, and reconstruct pixel data of the current frame, wherein the adaptive resolution management mode is not allowed.

[0059] In some embodiments, determining whether the current frame is usable as a reference frame may include determining whether the type of the current frame is type I. Determining whether the current frame is usable as a reference frame may include determining whether the type of the current frame is type P. The type of the current frame may be identified in the bitstream. The current frame may be a fragment. Parameters associated with the adaptive resolution management mode may not exist in the image header of the current frame. The decoder may be further configured to decode the current block from the bitstream. The current block may include a coding tree unit. The current block may include a prediction unit. The decoder may include an entropy decoding processor configured to receive the bitstream and decode the bitstream into quantization coefficients; an inverse quantization and inverse transform processor configured to process the quantization coefficients, the processing of the quantization coefficients including performing an inverse discrete cosine transform; a deblocking filter; a frame buffer; and an intra-frame prediction processor.

[0060] Some embodiments disclosed herein include a method comprising: receiving a bitstream, using the bitstream and determining, for a current frame, whether the current frame is usable as a reference frame, in response to determining that the current frame is usable as a reference frame, setting a variable characterizing that an adaptive resolution management mode is not allowed, and reconstructing pixel data of the current frame, wherein the adaptive resolution management mode is not allowed.

[0061] In some embodiments, determining whether the current frame is usable as a reference frame may include determining whether the type of the current frame is type I. Determining whether the current frame is usable as a reference frame includes determining whether the type of the current frame is type P. The type of the current frame may be identified in the bitstream. The current frame may be a fragment. Parameters associated with the adaptive resolution management mode may not exist in the image header of the current frame. The method may further include decoding the current block from the bitstream. The current block may include a coding tree unit. The current block may include a prediction unit. At least one of the receiving, determining, setting, and reconstructing may be performed by a decoder, the decoder including: an entropy decoding processor configured to receive the bitstream and decode the bitstream into quantization coefficients; an inverse quantization and inverse transform processor configured to process the quantization coefficients, the processing of the quantization coefficients including performing an inverse discrete cosine transform; a deblocking filter; a frame buffer; and an intra-frame prediction processor.

[0062] It should be noted that any one or more aspects and embodiments described herein can be readily implemented using digital electronic circuits, integrated circuits, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), computer hardware, firmware, software, and / or combinations thereof, as implemented and / or carried out in one or more machines programmed according to the teachings of this specification (e.g., one or more computing devices used as user computing devices for electronic documents, one or more server devices such as document servers, etc.), as will be apparent to those skilled in the art of computers. These various aspects or features may include implementations in one or more computer programs and / or software executable and / or interpretable on a programmable system, said programmable system including at least one programmable processor, which may be dedicated or general-purpose, coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to send data and instructions to the storage system, at least one input device, and at least one output device. It will be apparent to those skilled in the art of software that a skilled programmer can readily prepare appropriate software code based on the teachings of this disclosure. The aspects and implementation methods of employing software and / or software modules discussed above may also include appropriate hardware for assisting in the implementation of machine-executable instructions for the software and / or software modules.

[0063] Such software can be a computer program product employing a machine-readable storage medium. A machine-readable storage medium can be any medium capable of storing and / or encoding a sequence of instructions executable by a machine (e.g., a computing device) and causing the machine to perform any of the methods and / or embodiments described herein. Examples of machine-readable storage media include, but are not limited to, magnetic disks, optical disks (e.g., CDs, CD-Rs, DVDs, DVD-Rs, etc.), magneto-optical disks, read-only memory "ROM" devices, random access memory "RAM" devices, magnetic cards, optical cards, solid-state storage devices, EPROMs, EEPROMs, programmable logic devices (PLDs), and / or any combination thereof. The use of machine-readable media herein is intended to include both single media and collections of physically separate media, such as collections of optical disks, or one or more hard disk drives combined with computer memory. The use of machine-readable storage media herein does not include temporary forms of signal transmission.

[0064] Such software may also include information (e.g., data) carried as data signals on a data carrier such as a carrier wave. For example, machine-executable information may be included as data-bearing signals contained in a data carrier, wherein the signals encode: a sequence of instructions or portions thereof executed by a machine (e.g., a computing device), and any relevant information (e.g., data structures and data) that causes the machine to perform any of the methods and / or embodiments described herein.

[0065] Examples of computing devices include, but are not limited to, e-book readers, computer workstations, terminal computers, server computers, handheld devices (e.g., tablets, smartphones, etc.), network devices, network routers, network switches, bridges, any machine capable of executing a sequence of instructions specifying the action it is to take, and any combination of the foregoing. In one example, a computing device may include and / or be included in a kiosk (information station).

[0066] Figure 7 The illustration shows an embodiment of a computing device in an example form of a computer system 700, in which an instruction set may be executed to cause the control system to perform any or more aspects and / or methods of this disclosure. It is also contemplated that multiple computing devices may be used to implement a specially configured set of instructions to cause one or more devices to perform any or more aspects and / or methods of this disclosure. The computer system 700 includes a processor 704 and a memory 708, which communicate with each other and with other components via a bus 712. The bus 712 may include any of several types of bus architectures (using any of a variety of bus architectures), including but not limited to: a memory bus, a memory controller, a peripheral bus, a local bus, and any combination thereof.

[0067] Memory 708 may include various components (e.g., machine-readable media), including but not limited to random access memory components, read-only components, and any combination thereof. In one example, the basic input / output system 716 (BIOS) includes basic routines that facilitate the transfer of information between elements within the computer system 700, such as during startup; these basic routines may be stored in memory 708. Memory 708 may also include instructions (e.g., software) 720 stored in one or more machine-readable media; these instructions implement any or more aspects and / or methods of this disclosure. In another example, memory 708 may also include any number of program modules, including but not limited to: an operating system, one or more application programs, other program modules, program data, and any combination thereof.

[0068] Computer system 700 may also include storage device 724. Examples of storage devices (e.g., storage device 724) include, but are not limited to, hard disk drives, disk drives, optical disk drives combined with optical media, solid-state storage devices, and any combination thereof. Storage device 724 may be connected to bus 712 via a suitable interface (not shown). Example interfaces include, but are not limited to, SCSI, advanced technology attachment (ATA), serial ATA, universal serial bus (USB), IEEE 1394 (FIREWIRE), and any combination thereof. In one example, storage device 724 (or one or more components thereof) may removably interact with computer system 700 (e.g., via an external port connector (not shown)). In particular, storage device 724 and associated machine-readable medium 728 may provide non-volatile and / or volatile storage of machine-readable instructions, data structures, program modules, and / or other data for computer system 700. In one example, software 720 may reside wholly or partially within machine-readable medium 728. In another example, software 720 may reside entirely or partially within processor 704.

[0069] Computer system 700 may also include input device 732. In one example, a user of computer system 700 may input commands and / or other information into computer system 700 via input device 732. Examples of input device 732 include, but are not limited to: alphanumeric input devices (e.g., keyboard), pointing devices, joysticks, game controllers, audio input devices (e.g., microphones, voice response systems, etc.), cursor control devices (e.g., mice), touchpads, optical scanners, video capture devices (e.g., still cameras, camcorders), touchscreens, and any combination thereof. Input device 732 may be connected to bus 712 via any of a variety of interfaces (not shown), including but not limited to: serial interfaces, parallel interfaces, game ports, USB interfaces, firewire interfaces, direct interfaces connected to bus 712, and any combination thereof. Input device 732 may include a touchscreen interface, which may be part of or separate from display 736, as will be discussed further below. Input device 732 may be used as a user selection device for selecting one or more graphical representations within the graphical interface described above.

[0070] Users can also input commands and / or other information to computer system 700 via storage device 724 (e.g., removable disk drive, flash drive, etc.) and / or network interface device 740. Network interface devices, such as network interface device 740, can be used to connect computer system 700 to one or more networks (e.g., network 744) and one or more remote devices 748 connected to that network. Examples of network interface devices include, but are not limited to: network interface cards (e.g., mobile network interface cards, LAN cards), modems, and any combination thereof. Examples of networks include, but are not limited to: wide area networks (e.g., the Internet, corporate networks), local area networks (e.g., networks associated with offices, buildings, campuses, or other relatively small geographical areas), telephone networks, data networks associated with telephone / voice providers (e.g., mobile communication provider data and / or voice networks), direct connections between two computing devices, and any combination thereof. Networks, such as network 744, may employ wired and / or wireless communication modes. Typically, any network topology can be used. Information (e.g., data, software 720, etc.) can be transmitted to computer system 200 and / or transmitted out of computer system 700 via network interface device 740.

[0071] Computer system 700 may further include a video display adapter 752 for transmitting displayable images to a display device, such as display device 736. Examples of display devices include, but are not limited to, liquid crystal displays (LCDs), cathode ray tube displays (CRTs), plasma displays, light-emitting diode displays (LEDs), and any combination thereof. Display adapter 752 and display device 736 may be used in conjunction with processor 704 to provide a graphical representation of aspects of this disclosure. In addition to display devices, computer system 700 may include one or more other peripheral output devices, including, but not limited to, audio speakers, printers, and any combination thereof. These peripheral output devices may be connected to bus 712 via peripheral interface 756. Examples of peripheral interfaces include, but are not limited to, serial ports, USB connections, firewire connections, parallel connections, and any combination thereof.

[0072] The foregoing is a detailed description of illustrative embodiments of the present invention. Various modifications and additions can be made without departing from the spirit and scope of the invention. Features of each of the various embodiments described above can be suitably combined with features of other described embodiments to provide multiple feature combinations in associated new embodiments. Furthermore, while several individual embodiments have been described above, the description herein is merely an illustration of the application of the principles of the invention. Moreover, although specific methods herein may be illustrated and / or described as being performed in a particular order, the order is highly variable within the ordinary technical scope of implementing the embodiments disclosed herein. Therefore, this description is intended only as an example and does not otherwise limit the scope of the invention.

[0073] In the foregoing description and claims, phrases such as “at least one” or “one or more” may appear after a list of combinations of elements or features. The term “and / or” may also appear in a list of two or more elements or features. Unless implied or explicitly contradicted by the context in which it is used, such phrases are intended to mean any element or feature listed individually, or any listed element or feature combined with any other listed element or feature. For example, the phrases “at least one of A and B;”, “one or more of A and B;”, and “A and / or B” respectively mean “A alone, B alone, or A and B together.” A similar interpretation applies to lists containing three or more items. For example, the phrases “at least one of A, B, and C;”, “one or more of A, B, and C;”, and “A, B, and / or C” respectively mean “A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B, and C together.” Furthermore, the term “based on” as used in the foregoing and claims is intended to mean “at least partially based on,” allowing for the inclusion of features or elements not listed.

[0074] Depending on the desired configuration, the subject matter described herein can be embodied in systems, devices, methods, and / or articles. The embodiments set forth in the foregoing description do not represent all embodiments consistent with the subject matter described herein. Rather, they are merely some examples consistent with aspects related to the described subject matter. Although some variations have been described in detail above, other modifications or additions are possible. In particular, further features and / or variations may be provided in addition to those set forth herein. For example, the above embodiments may involve various combinations and sub-combinations of the disclosed features and / or combinations and sub-combinations of several further features described above. Furthermore, the logical flows depicted in the drawings and / or described herein do not necessarily need to be in the specific order or sequence shown to achieve the desired results. Other implementations may be within the scope of the following claims.

Claims

1. A decoder with an inter-frame prediction mode for generating a predictor from a reference image, the predictor being used for subsequent images having a different spatial resolution than the reference image, the decoder being configured to: Receive an encoded bitstream, the encoded bitstream including a first image, subsequent images and an image header associated with the first image but not associated with the subsequent images, the subsequent images having a different spatial resolution than the first image, and the image header associated with the first image not including parameters for utilizing the mode; In response to the image header and without utilizing the mode, the first image is decoded; The first image decoded is retained as a reference image for the subsequent images; A predictor is formed by using a scaling factor signaled in the encoded bitstream and using filters selected for specific blocks of the reference image rather than the entire reference image to scale the spatial resolution of the pixel blocks of the reference image. as well as The pixel blocks of the subsequent image are decoded by adding the residual pixel values ​​to the predictor.

2. A video encoder, configured as follows: The bitstream is encoded for decoding by a decoder, the decoder having an inter-frame prediction mode for generating a predictor from a reference image, the predictor being used for subsequent images having a different spatial resolution than the reference image, the decoder being configured as follows: Receive an encoded bitstream, the encoded bitstream including a first image, subsequent images and an image header associated with the first image but not associated with the subsequent images, the subsequent images having a different spatial resolution than the first image, and the image header associated with the first image not including parameters for utilizing the mode; In response to the image header and without utilizing the mode, the first image is decoded; The first image decoded is retained as a reference image for the subsequent images; A predictor is formed by using a scaling factor signaled in the encoded bitstream and using filters selected for specific blocks of the reference image rather than the entire reference image to scale the spatial resolution of the pixel blocks of the reference image. as well as The pixel blocks of the subsequent image are decoded by adding the residual pixel values ​​to the predictor.

3. A decoder having an inter-frame prediction mode for generating a predictor from a reference image, the predictor being used for subsequent images having a different spatial resolution than the reference image, the decoder being configured to: Receive an encoded bitstream, the encoded bitstream including a first image, subsequent images and an image header associated with the first image but not associated with the subsequent images, the subsequent images having a different spatial resolution than the first image, and the image header associated with the first image not including parameters for utilizing the mode; In response to the image header and without utilizing the mode, the first image is decoded, and the information in the bitstream is used to determine that the first image can be used as a reference image for the subsequent images; A predictor is formed by using a scaling factor signaled in the encoded bitstream and using filters selected for specific blocks of the reference image rather than the entire reference image to scale the spatial resolution of the pixel blocks of the reference image. as well as The predictor is used to decode the pixel blocks of the subsequent image.

Images