Signaling for reference picture resampling

Abstract

To provide a method, device, and non-transitory computer-readable medium for decoding an encoded video bitstream using at least one processor.SOLUTION: A method comprises the steps of: obtaining a coded picture; decoding the coded picture to generate a decoded picture; obtaining a first flag indicating whether reference picture resampling is enabled; obtaining a second flag indicating whether reference pictures have a constant reference picture size; obtaining a third flag indicating whether output pictures have a constant output picture size; generating a reference picture by resampling the decoded picture to have the constant reference picture size, and storing the reference picture in a decoded picture buffer; and generating an output picture by resampling the decoded picture to have the constant output picture size, and outputting the output picture.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority under 35 U.S.C. §119 to U.S. Provisional Application No. 62 / 865,955, filed June 24, 2019, and U.S. Patent Application No. 16 / 899,202, filed June 11, 2020, the disclosures of which are incorporated herein by reference in their entireties.

[0002] The disclosed subject matter relates to video encoding and decoding, and more particularly, to signaling information for reference image resampling and adaptive resolution change. [Background technology]

[0003] Video encoding and decoding using inter-picture prediction with motion compensation is known. Uncompressed digital video can consist of a sequence of images, each image having spatial dimensions of, for example, 1920x1080 luma samples and associated chroma samples. The sequence of images can have a fixed or variable image rate (also informally known as frame rate), for example, 60 images per second or 60 Hz. Uncompressed video has significant bitrate requirements. For example, 1080p60 4:2:0 video (1920x1080 luma sample resolution at a frame rate of 60 Hz) with 8 bits per sample requires a bandwidth approaching 1.5 Gbit / s. One hour of such video requires more than 600 GByte of storage space.

[0004] One of the goals of video encoding and video decoding may be the reduction of redundancy in the input video signal through compression. Compression can help reduce the aforementioned bandwidth or storage space requirements, in some cases by more than two orders of magnitude. Both lossless and lossy compression, as well as combinations thereof, may be used. Lossless compression refers to techniques where an exact replica of the original signal can be reconstructed from the compressed original signal. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signals is small enough to make the reconstructed signal useful for the intended application. For video, lossy compression is widely adopted. The amount of distortion tolerated depends on the application, e.g., a user of a particular consumer streaming application may tolerate higher distortion than a user of a television distribution application. The achievable compression ratio may reflect that a higher tolerable / acceptable distortion can result in a higher compression ratio.

[0005] Video encoders and decoders can utilize techniques from a number of broad categories, including, for example, motion compensation, transform, quantization, and entropy coding, some of which are introduced below.

[0006] Historically, video encoders and decoders have tended to work with a given picture size that was defined and remained constant for a coded video sequence (CVS), group of pictures (GOP), or similar multi-picture time frame in most cases. For example, in MPEG-2, system designs are known to change the horizontal resolution (and thereby the picture size) depending on factors such as scene activity, but only in I-pictures, and thus typically for GOPs. Resampling of reference pictures to use different resolutions within a CVS is known, for example, from ITU-T Rec. H.263 Annex P. However, here the picture size does not change, only the reference picture is resampled, and it is possible that (in the case of downsampling) only a part of the picture canvas is used, or (in the case of upsampling) only a part of the scene is captured. Furthermore, H.263 Annex Q allows for resampling of individual macroblocks by a factor of two upwards or downwards (in each dimension). Again, the picture size remains the same. The size of the macroblocks is fixed in H.263, so it does not need to be signaled.

[0007] Image resizing of predicted images has become more mainstream in modern video coding. For example, VP9 allows for reference image resampling and changing the resolution of the entire image. Similarly, certain proposals made for VVC (including, for example, Hendry et al., "On adaptive resolution change (ARC) for VVC," Joint Video Team document JVET-M0135-v1, Jan 9-19, 2019, incorporated herein in its entirety) allow for resampling of the entire reference image to different higher or lower resolutions. In that document, different candidate resolutions are proposed that are coded in the sequence parameter set and referenced by per-image syntax elements in the image parameter set. Summary of the Invention [Means for solving the problem]

[0008] In one embodiment, a method for decoding an encoded video bitstream using at least one processor includes the steps of obtaining an encoded image from the encoded video bitstream; decoding the encoded image to generate a decoded image; obtaining from the encoded video bitstream a first flag indicating whether reference image resampling is enabled; obtaining from the encoded video bitstream a second flag indicating whether a reference image has a certain reference image size indicated in the encoded video bitstream based on the first flag indicating reference image resampling is enabled; and determining whether reference image resampling is enabled based on the first flag indicating reference image resampling is enabled. In one embodiment, a method is provided, comprising the steps of: obtaining from the encoded video bitstream a third flag indicating whether the output image has a constant output image size indicated in the encoded video bitstream based on a first flag indicating whether the output image has a constant output image size; generating a reference image by resampling the decoded image to have the constant reference image size based on a second flag indicating the reference image has the constant reference image size and storing the reference image in a decoded image buffer; generating an output image by resampling the decoded image to have the constant output image size based on a third flag indicating the output image has the constant output image size, and outputting the output image.

[0009] In one embodiment, a device for decoding an encoded video bitstream, the device comprising at least one memory configured to store program code; and at least one processor configured to read the program code and operate as instructed by the program code, the program code comprising: a first acquisition code configured to cause the at least one processor to acquire an encoded image from the encoded video bitstream; a decoding code configured to cause the at least one processor to decode the encoded image to generate a decoded image; a second acquisition code configured to cause the at least one processor to acquire, from the encoded video bitstream, a first flag indicating whether reference image resampling is enabled; and based on the first flag indicating that reference image resampling is enabled, cause the at least one processor to acquire, from the encoded video bitstream, a first flag indicating whether a reference image has a constant reference image size indicated in the encoded video bitstream. a third acquisition code configured to cause the at least one processor to obtain, from the encoded video bitstream, a third flag indicating whether reference image resampling is enabled; a fourth acquisition code configured to cause the at least one processor to obtain, based on the first flag indicating reference image resampling is enabled, a third flag indicating whether an output image has a constant output image size indicated in the encoded video bitstream; a first generation code configured to cause the at least one processor to generate a reference image by resampling a decoded image to have a constant reference image size and store the reference image in a decoded image buffer based on the second flag indicating the reference image has a constant reference image size; and a second generation code configured to cause the at least one processor to generate an output image by resampling a decoded image to have a constant output image size and output the output image based on the third flag indicating the output image has a constant output image size.

[0010] In one embodiment, a non-transitory computer readable medium storing instructions, which when executed by one or more processors of a device for decoding an encoded video bitstream, causes the one or more processors to obtain an encoded image from the encoded video bitstream, decode the encoded image to generate a decoded image, obtain from the encoded video bitstream a first flag indicating whether reference image resampling is enabled, obtain from the encoded video bitstream a second flag indicating whether a reference image has a constant reference image size indicated in the encoded video bitstream based on the first flag indicating reference image resampling is enabled, and a third flag indicating whether an output image has a constant output image size indicated in the encoded video bitstream based on a first flag indicating pulling is enabled; generating a reference image by resampling a decoded image to have the constant reference image size based on a second flag indicating the reference image has the constant reference image size, storing the reference image in a decoded image buffer; generating an output image by resampling the decoded image to have the constant output image size based on a third flag indicating the output image has the constant output image size; and outputting the output image.

[0011] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram of a simplified block diagram of a communication system according to one embodiment. [Diagram 2] FIG. 1 is a schematic diagram of a simplified block diagram of a communication system according to one embodiment. [Diagram 3] FIG. 2 is a schematic diagram of a simplified block diagram of a decoder according to one embodiment. [Figure 4] FIG. 2 is a schematic diagram of a simplified block diagram of an encoder according to one embodiment. [Diagram 5] FIG. 2 is a schematic diagram of a simplified block diagram of an encoder according to one embodiment. [Figure 6] FIG. 2 is a schematic diagram of a simplified block diagram of a decoder according to one embodiment. [Figure 7] 1 is a flowchart of an exemplary process for decoding an encoded video bitstream, according to one embodiment. [Figure 8] FIG. 1 is a schematic diagram of a computer system according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The embodiments disclosed herein may be used separately or combined in any order. Furthermore, each of the methods (or embodiments), the encoder, and the decoder may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium.

[0014] FIG. 1 shows a simplified block diagram of a communication system (100) according to one embodiment of the present disclosure. The system (100) may include at least two terminals (110-120) interconnected via a network (150). In the case of a one-way transmission of data, a first terminal (110) may encode video data at a local location for transmission to another terminal (120) via the network (150). The second terminal (120) may receive the encoded video data of the other terminal from the network (150), decode the encoded data, and display the restored video data. One-way data transmission may be common in media serving applications, etc.

[0015] 1 illustrates a second pair of terminals (130, 140) provided to support two-way transmission of encoded video, such as might occur during a video conference. For two-way transmission of data, each terminal (130, 140) may encode video data captured at a local location for transmission over a network (150) to the other terminal. Each terminal (130, 140) may also receive encoded video data transmitted by the other terminal, decode the encoded data, and display the recovered video data on a local display device.

[0016] In FIG. 1, the terminals (110-140) may be depicted as servers, personal computers, and smartphones, although the principles of the present disclosure may not be so limited. Embodiments of the present disclosure find application with laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. The network (150) represents any number of networks that convey encoded video data between the terminals (110-140), including, for example, wired and / or wireless communication networks. The communication network (150) may exchange data over circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this discussion, the architecture and topology of the network (150) may not be important to the operation of the present disclosure unless otherwise described herein below.

[0017] 2 shows the arrangement of video encoders and decoders in a streaming environment as an example of an application of the disclosed subject matter. The disclosed subject matter is equally applicable to other video-enabled applications including, for example, video conferencing, digital television, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0018] The streaming system may include a video source (201) and a capture subsystem (213) that may include, for example, a digital camera that creates an uncompressed video sample stream (202). The sample stream (202), shown as a thick line to emphasize the larger amount of data when compared to an encoded video bitstream, may be processed by an encoder (203) coupled to the camera (201). The encoder (203) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video bitstream (204), shown as a thin line to emphasize the smaller amount of data compared to the sample stream, may be stored on a streaming server (205) for future use. One or more streaming clients (206, 208) may access the streaming server (205) to obtain copies (207, 209) of the encoded video bitstream (204). The client (206) may include a video decoder (210) that decodes an incoming copy of an encoded video bitstream (207) and creates an outgoing video sample stream (211) that can be rendered on a display (212) or other rendering device (not shown). In some streaming systems, the video bitstreams (204, 207, 209) may be encoded according to a particular video encoding / compression standard. For example, these standards include ITU-T Recommendation H.265. A video encoding standard informally known as Versatile Video Coding, or VVC, is under development. The disclosed subject matter may be used in the context of VVC.

[0019] FIG. 3 may be a functional block diagram of a video decoder (210) according to one embodiment of the disclosure.

[0020] The receiver (310) may receive one or more codec video sequences to be decoded by the decoder (210), in the same or another embodiment, one coded video sequence at a time, with the decoding of each coded video sequence being independent of the other coded video sequences. The coded video sequences may be received from a channel (312), which may be a hardware / software link to a storage device that stores the coded video data. The receiver (310) may receive the coded video data together with other data, e.g., coded audio data and / or auxiliary data streams, that may be forwarded to a respective using entity (not shown). The receiver (310) may separate the coded video sequences from the other data. To combat network jitter, a buffer memory (315) may be coupled between the receiver (310) and the entropy decoder / parser (320) (hereinafter, the "parser"). When the receiver (310) is receiving data from a store / forward device of sufficient bandwidth and controllability, or from an isosynchronous network, the buffer (315) may not be needed or may be small. For use with best-effort packet networks such as the Internet, the buffer (315) may be needed and may be relatively large, and may advantageously be of adaptive size.

[0021] The video decoder (210) may include a parser (320) for reconstructing symbols (321) from the entropy-encoded video sequence. These categories of symbols may include information used to manage the operation of the decoder (210) and information for controlling a rendering device, such as a display (212), that is not an integral part of the decoder but may be coupled to the decoder, as shown in FIG. 3. The control information for the rendering device(s) may be in the form of a Supplementary Enhancement Information (SEI message) or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (320) may parse / entropy decode the received encoded video sequence. The encoding of the encoded video sequence may follow any video coding technique or standard and may follow principles well known to those skilled in the art, including variable length coding, Huffman coding, arithmetic coding with or without context-dependent coding, etc. The parser (320) may extract from the encoded video sequence at least one set of subgroup parameters for a subgroup of pixels in the video decoder based on at least one parameter corresponding to the group. A subgroup may include a Group of Pictures (GOP), a picture, a subpicture, a tile, a slice, a brick, a macroblock, a Coding Tree Unit (CTU), a block, a transform unit (TU), a prediction unit (PU), etc. A tile may indicate a rectangular region of a CU / CTU in a particular tile column and row in a picture. A brick may indicate a rectangular region of a CU / CTU row in a particular tile. A slice may indicate one or more bricks of a picture contained in a NAL unit. A subpicture may indicate a rectangular region of one or more slices in a picture. The entropy decoder / parser may also extract from the coded video sequence information such as transform coefficients, quantization parameter values, motion vectors, etc.

[0022] The parser (320) may perform entropy decoding / parsing operations on the video sequence received from the buffer (315) to produce symbols (321).

[0023] The reconstruction of the symbols (321) can involve a number of different units, depending on the type of encoded video image or part thereof (inter-image and intra-image, inter-block and intra-block, etc.), and other factors. Which units are involved and how can be controlled by subgroup control information parsed from the encoded video sequence by the parser (320). The flow of such subgroup control information between the parser (320) and the following units is not depicted for clarity.

[0024] Beyond the functional blocks already mentioned, the decoder (210) may be conceptually subdivided into a number of functional units, as described below. In an actual implementation operating under commercial constraints, many of these units will interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, the following conceptual subdivision into functional units is appropriate:

[0025] The first unit is a scalar / inverse transform unit (351), which receives quantized transform coefficients as well as control information from the parser (320) including the transform used, block size, quantization factor, quantization scaling matrix, etc. as symbols (321). The scalar / inverse transform unit can output blocks containing sample values ​​that can be input to an aggregator (355).

[0026] In some cases, the output samples of the scalar / inverse transform (351) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed image, but may use prediction information from a previously reconstructed portion of the current image. Such prediction information may be provided by an intra-image prediction unit (352). In some cases, the intra-image prediction unit (352) uses surrounding already reconstructed information fetched from the current (partially reconstructed) image (358) to generate a block of the same size and shape as the block being reconstructed. The aggregator (355) may add, on a sample-by-sample basis, the prediction information generated by the intra-prediction unit (352) to the output sample information provided by the scalar / inverse transform unit (351).

[0027] In other cases, the output samples of the scalar / inverse transform unit (351) may relate to an inter-coded, potentially motion-compensated block. In such cases, the motion compensation prediction unit (353) may access the reference picture memory (357) to fetch samples used for prediction. After motion compensating the fetched samples according to the symbols (321) associated with the block, these samples may be added to the output of the scalar / inverse transform unit by the aggregator (355) to generate output sample information (in this case referred to as residual samples or residual signals). The addresses in the reference picture memory form from which the motion compensation unit fetches the prediction samples may be controlled by a motion vector and are available to the motion compensation unit in the form of symbols (321), which may have, for example, X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​fetched from the reference picture memory when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, etc.

[0028] The output samples of the aggregator (355) may be subject to various loop filtering techniques in a loop filter unit (356). Video compression techniques may include in-loop filter techniques controlled by parameters contained in the encoded video bitstream and made available to the loop filter unit (356) as symbols (321) from the parser (320), but may also be responsive to meta-information obtained during decoding of previous (in decoding order) portions of the encoded image or encoded video sequence, and may also be responsive to previously reconstructed and loop filtered sample values.

[0029] The output of the loop filter unit (356) may be a sample stream that may be output to the rendering device (212) as well as stored in a reference picture memory for use in future inter-picture prediction.

[0030] Once a particular coded image is fully reconstructed, it can be used as a reference image for future predictions. Once a coded image is fully reconstructed and the coded image is identified as a reference image (e.g., by the parser (320)), the current reference image (358) can become part of the reference image buffer (357) and a new current image memory can be reallocated before starting the reconstruction of the next coded image.

[0031] The video decoder 210 may perform decoding operations according to a given video compression technique, which may be documented in a standard, such as ITU-T Rec. H.265. The encoded video sequence may be compliant with the syntax specified by the video compression technique or standard being used, in the sense that the encoded video sequence conforms to the syntax of the video compression technique or standard as specified in the video compression technique's document or standard, and in particular in the profile documents therein. Also required for compliance is that the complexity of the encoded video sequence is within a range defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level may in some cases be further limited by a hypothetical reference decoder (HRD) specification and HRD buffer management metadata conveyed in the encoded video sequence.

[0032] In one embodiment, the receiver (310) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the encoded video sequence. The additional data may be used by the video decoder (210) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or SNR enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0033] FIG. 4 may be a functional block diagram of a video encoder (203) according to one embodiment of the present disclosure.

[0034] The encoder (203) may receive video samples from a video source (201) (not part of the encoder) that may capture video images to be encoded by the encoder (203).

[0035] The video source (201) may provide a source video sequence to be encoded by the encoder (203) in the form of a digital video sample stream that may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source (201) may be a storage device that stores previously prepared video. In a video conferencing system, the video source (203) may be a camera that captures local image information as a video sequence. The video data may be provided as multiple individual images that give motion when viewed in sequence. The image itself may be organized as a spatial array of pixels, each of which may contain one or more samples, depending on the sampling structure, color space, etc. in use. Those skilled in the art can easily understand the relationship between pixels and samples. The following description focuses on samples.

[0036] According to one embodiment, the encoder (203) may encode and compress images of a source video sequence into an encoded video sequence (443) in real time or under any other time constraint required by the application. Enforcing an appropriate encoding rate is one function of the controller (450). The controller controls and is operatively coupled to other functional units as described below. For clarity, couplings are not depicted. Parameters set by the controller may include rate control related parameters (picture skip, quantizer, lambda value for rate distortion optimization techniques, etc.), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. Those skilled in the art may readily identify other functions of the controller (450) as they may relate to a video encoder (203) optimized for a particular system design.

[0037] Some video encoders operate in what those skilled in the art will readily recognize as a "coding loop." As an oversimplified explanation, the coding loop can consist of a coding part of the encoder (430) (hereafter the "source coder") (responsible for creating symbols based on the input image to be coded and the reference image), and a (local) decoder (433) embedded in the encoder (203) that reconstructs the symbols to create sample data that the (remote) decoder also creates (since any compression between the symbols and the coded video bitstream is lossless in the video compression techniques considered in the disclosed subject matter). That reconstructed sample stream is input to a reference image memory (434). Since the decoding of the symbol stream results in bit-exact results regardless of the location of the decoder (local or remote), the contents of the reference image buffer are also bit-exact between the local and remote encoders. In other words, the predictive part of the encoder "sees" exactly the same sample values ​​as the decoder "sees" when using prediction during decoding as reference image samples. This basic principle of reference image synchrony (and the drift that occurs when synchrony cannot be maintained, for example due to channel errors) is well known to those skilled in the art.

[0038] The operation of the "local" decoder (433) may be the same as that of the "remote" decoder (210), which is described in detail above in connection with Figure 3. However, with brief reference also to Figure 4, symbols may be available, and the encoding / decoding of the symbols into an encoded video sequence by the entropy coder (445) and parser (320) may be lossless, and the entropy decoding portion of the decoder (210), including the channel (312), receiver (310), buffer (315) and parser (320), may not be fully implemented in the local decoder (433).

[0039] An observation that can be made at this point is that any decoder techniques other than analysis / entropy decoding present in a decoder must necessarily be present in a corresponding encoder in substantially identical functional form. For this reason, the disclosed subject matter focuses on the decoder operation. A description of the encoder techniques can be omitted, since they are the inverse of the decoder techniques described generically. Only in certain areas is more detailed description necessary and is provided below.

[0040] As part of its operation, the source coder (430) may perform motion-compensated predictive coding, which predictively codes an input frame with reference to one or more previously coded frames from the video sequence designated as “reference frames.” In this manner, the coding engine (432) codes differences between pixel blocks of the input frame and pixel blocks of reference frames that may be selected as predictive references to the input frame.

[0041] The local video decoder (433) may decode the encoded video data of a frame that may be designated as a reference frame based on the symbols created by the source coder (430). The operation of the encoding engine (432) may advantageously be a lossy process. If the encoded video data can be decoded in a video decoder (not shown in FIG. 4), the reconstructed video sequence may be a replica of the source video sequence, usually with some errors. The local video decoder (433) may replicate the decoding process that may be performed by the video decoder on the reference frame and cause the reconstructed reference frame to be stored in a reference image cache (434). In this way, the encoder (203) may locally store a replica of the reconstructed reference frame that has common content as the reconstructed reference frame obtained by the far-end video decoder (without transmission errors).

[0042] The predictor (435) may perform a prediction search for the coding engine (432). That is, for a new frame to be coded, the predictor (435) may search the reference picture memory (434) for sample data (as candidate reference pixel blocks) or specific metadata such as reference picture motion vectors, block shapes, etc. that may serve as suitable prediction references for the new picture. The predictor (435) may operate on one sample block per pixel block to find a suitable prediction reference. In some cases, as determined by the search results obtained by the predictor (435), the input picture may have prediction references drawn from multiple reference pictures stored in the reference picture memory (434).

[0043] The controller (450) may manage the encoding operations of the video coder (430), including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0044] The output of all the aforementioned functional units may undergo entropy coding in an entropy coder (445), which converts the symbols produced by the various functional units into an encoded video sequence by losslessly compressing the symbols according to techniques known to those skilled in the art, such as Huffman coding, variable length coding, arithmetic coding, etc.

[0045] The transmitter (440) may buffer the encoded video sequence created by the entropy coder (445) and prepare it for transmission over a communication channel (460), which may be a hardware / software link to a storage device that stores the encoded video data. The transmitter (440) may merge the encoded video data from the video coder (430) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (sources not shown).

[0046] The controller (450) may manage the operation of the encoder (203). During encoding, the controller (450) may assign a particular encoded picture type to each encoded picture, which may affect the encoding technique that may be applied to each picture. For example, pictures may often be assigned as any of the following frame types:

[0047] An intra picture (I-picture) may be one that can be coded and decoded without using other frames in a sequence as a source of prediction. Some video codecs can use different types of intra pictures, such as the Independent Decoder Refresh Picture. Those skilled in the art are aware of these variants of I-pictures and their respective uses and characteristics.

[0048] A predicted image (P-image) may be one that can be encoded and decoded using intra- or inter-prediction, which uses at most one motion vector and reference index to predict sample values ​​for each block.

[0049] A bidirectionally predicted image (B-image) may be one that can be encoded and decoded using intra- or inter-prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predicted images may use more than two reference images and associated metadata for the reconstruction of a single block.

[0050] A source image may generally be spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples, respectively) and coded block by block. A block may be predictively coded with reference to other (already coded) blocks, as determined by a coding assignment applied to the respective image of the block. For example, a block of an I image may be non-predictively coded or may be predictively coded with reference to already coded blocks of the same image (spatial or intra prediction). A pixel block of a P image may be non-predictively coded via spatial prediction or via temporal prediction with reference to one previously coded reference image. A block of a B image may be non-predictively coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference images.

[0051] The video coder (203) may perform encoding operations according to a given video encoding technique or standard, such as ITU-T Rec. H.265. In its operations, the video coder (203) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the encoded video data may conform to a syntax specified by the video encoding technique or standard being used.

[0052] In one embodiment, the transmitter (440) may transmit additional data along with the encoded video. The video coder (430) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, supplemental enhancement information (SEI) messages, visual usability information (VUI) parameter set fragments, etc.

[0053] FIG. 5 illustrates an example of an encoder 500 according to an embodiment, and FIG. 6 illustrates an example of a decoder 600 according to an embodiment. With reference to FIG. 5, the encoder 500 may include a downsampler 501, an image partitioner 502, an inverse quantizer 503, an entropy coder 504, an in-loop filter 505, an intra predictor 506, a decoded picture buffer (DPB) 507, a resampler 508, and an inter predictor 509. With reference to FIG. 6, the decoder 600 may include an encoded picture buffer 601, a video syntax parser 602, an inverse quantizer 603, an in-loop filter 604, a decoded picture buffer 605, a resampler 606, an inter predictor 607, and an intra predictor 608.

[0054] In an embodiment, one or more elements shown in FIG. 5 and / or FIG. 6 may correspond to or perform similar functions as one or more elements shown in FIG. 3 and / or FIG.

[0055] In embodiments, such as those shown in Figures 5 and 6, it is possible to change the image width and height for each image granularity, regardless of the image type. In the encoder 500, the input image data may be downsampled to a selected image size for the current image encoding, for example using a downsampler 501. After the first input image is encoded as an intra image, the decoded image is stored in the DPB 507. When the resulting image is downsampled with a different sampling ratio and encoded as an inter image, the reference images in the DPB may be upscaled or downscaled according to the spatial ratio between the reference image size and the current image size, for example using a resampler 508.

[0056] In the decoder 600, decoded images may be stored in the DPB 605 without resampling. However, reference images in the DPB 605, when used for motion compensation, may be upscaled or downscaled relative to the spatial ratio between the currently decoded image and the reference, for example using the resampler 606. When decoded images are pushed out for display, they may be upsampled to the original image size or the desired output image size, for example using the upsampler 609. In the motion estimation / compensation process, motion vectors may be scaled relative to the image size ratio as well as the image order count difference.

[0057] In an embodiment, a reference image resampling (RPR) scheme, for example as used in the embodiments disclosed in this specification, may include support for adaptive (decoded) image resolution changes within an encoded video sequence, support for a constant reference image resolution to simplify the motion compensation process, support for a constant output image resolution for a derived display resolution, as well as support for adaptive resampling modes with or without additional filtering.

[0058] In an embodiment, a set of high-level syntax modifications may be used to support the desired features for RPR and adaptive resolution change (ARC).

[0059] For example, in an embodiment, a minimum / maximum image resolution may be signaled in a decoder parameter set (DPS) to facilitate cap exchange / negotiation.

[0060] In an embodiment, a flag indicating that RPR is disabled in the encoded video sequence may be signaled in a sequence parameter set (SPS). The decoded image resolution may be signaled in a table in the SPS. The table may include a list of decoded image sizes that may be used by one or more images in the encoded video sequence.

[0061] In an embodiment, a flag may be signaled in the SPS to indicate that any reference image has the same spatial resolution and a constant reference image size. If the flag value is 1, any decoded image in the encoded video sequence may be upscaled by a resampling process, so that any reference image stored in the DPB may have the same image size as the reference image size signaled in the SPS.

[0062] In an embodiment, a flag may be signaled in the SPS to indicate that any output image has the same spatial resolution and has a constant output image size. If the flag value is 1, any output image in the encoded video sequence may be upscaled by a resampling process, so that any output image may have the same image size as the output image size signaled in the SPS.

[0063] In an embodiment, an index indicating the decoded picture size among the candidates signaled in the SPS may be signaled in a Picture Parameter Set (PPS), which may be used to facilitate cap exchange / negotiation.

[0064] In an embodiment, a flag may be signaled in the PPS to indicate that motion vector scaling of temporal motion vector prediction is disabled. If the flag value is 1, any temporal motion vector prediction may be processed without motion vector scaling.

[0065] In an embodiment, the filter mode selection may be signaled in the PPS.

[0066] An example of DPS syntax for notifying the above embodiment is shown in Table 1 below.

[0067] [Table 1]

[0068] In an embodiment, max_pic_width_in_luma_samples may specify the maximum width of the decoded image in units of luma samples in the bitstream. max_pic_width_in_luma_samples may not be equal to 0 and may be an integer multiple of MinCbSizeY. The value of max_pic_width_in_luma_samples[i] may not be greater than the value of max_pic_width_in_luma_samples.

[0069] In an embodiment, max_pic_height_in_luma_samples may specify the maximum height of a decoded image in units of luma samples. max_pic_height_in_luma_samples may not be equal to 0 and may be an integer multiple of MinCbSizeY. The value of max_pic_height_in_luma_samples[i] may not be greater than the value of max_pic_height_in_luma_samples.

[0070] An example of SPS syntax for notifying the above embodiment is shown in Table 2 below.

[0071] [Table 2]

[0072] In an embodiment, reference_pic_resampling_flag equal to 1 may specify that the decoded image size of the coded image associated with the SPS may or may not vary within the coded video sequence. reference_pic_resampling_flag equal to 0 specifies that the decoded image size of the coded image associated with the SPS may not vary within the coded video sequence. When the value of reference_pic_resampling_flag is equal to 1, there may be one or more decoded image sizes (dec_pic_width_in_luma_samples[i], dec_pic_height_in_luma_samples[i]) that may be indicated and used by coded images in the coded video sequence, and there are constant reference image sizes (reference_pic_width_in_luma_samples, reference_pic_height_in_luma_samples) and constant output image sizes output_pic_width_in_luma_samples, output_pic_height_in_luma_samples), which are adjusted by the values ​​of constant_ref_pic_size_present_flag and constant_output_pic_size_present_flag, respectively.

[0073] In an embodiment, constant_ref_pic_size_flag equal to 1 may specify that reference_pic_width_in_luma_samples and reference_pic_height_in_luma_samples are present.

[0074] In an embodiment, reference_pic_width_in_luma_samples may specify the width of the reference picture in units of luma samples. reference_pic_width_in_luma_samples may not be equal to 0. If not present, the value of reference_pic_width_in_luma_samples may be inferred to be equal to dec_pic_width_in_luma_samples[i].

[0075] In an embodiment, reference_pic_height_in_luma_samples may specify the height of the reference picture in units of luma samples. reference_pic_height_in_luma_samples may not be equal to 0. If not present, the value of reference_pic_height_in_luma_samples may be inferred to be equal to dec_pic_height_in_luma_samples[i]. The size of the reference picture stored in the DPB may be equal to the values ​​of reference_pic_width_in_luma_samples and reference_pic_height_in_luma_samples if the value of constant_pic_size_present_flag is 1. In this case, no additional resampling process may be performed for motion compensation.

[0076] In an embodiment, constant_output_pic_size_flag equal to 1 may specify that output_pic_width_in_luma_samples and output_pic_height_in_luma_samples are present.

[0077] In an embodiment, output_pic_width_in_luma_samples may specify the width of the output image in units of luma samples. output_pic_width_in_luma_samples shall not be equal to 0. If not present, the value of output_pic_width_in_luma_samples may be inferred to be equal to dec_pic_width_in_luma_samples[i].

[0078] In an embodiment, output_pic_height_in_luma_samples may specify the height of the output image in units of luma samples. height_pic_height_in_luma_samples may not be equal to 0. If not present, the value of output_pic_height_in_luma_samples may be inferred to be equal to dec_pic_height_in_luma_samples[i]. For a value of constant_output_pic, the size of the output image may be equal to the values ​​of output_pic_width_in_luma_samples and output_pic_height_in_luma_samples.

[0079] In an embodiment, num_dec_pic_size_in_luma_samples_minus 1 plus 1 may specify the number of decoded image sizes (dec_pic_width_in_luma_samples[i], dec_pic_height_in_luma_samples[i]) in units of luma samples in the encoded video sequence.

[0080] In an embodiment, dec_pic_width_in_luma_samples[i] may specify the i-th width of the decoded image size in units of luma samples in the encoded video sequence. dec_pic_width_in_luma_samples[i] may not be equal to 0 and may be an integer multiple of MinCbSizeY.

[0081] In an embodiment, dec_pic_height_in_luma_samples[i] may specify the i-th height of the decoded image size in units of luma samples in the encoded video sequence. dec_pic_height_in_luma_samples[i] may not be equal to 0 and may be an integer multiple of MinCbSizeY. The i-th decoded image size (dec_pic_width_in_luma_samples[i], dec_pic_height_in_luma_samples[i]) may be equal to the decoded image size of the decoded image in the encoded video sequence.

[0082] An example of PPS syntax for signaling the above embodiment is shown in Table 3 below.

[0083] [Table 3]

[0084] In an embodiment, dec_pic_size_idx may specify that the width of the decoded image should be equal to pic_width_in_luma_samples[dec_pic_size_idx] and the height of the decoded image should be equal to pic_height_in_luma_samples[dec_pic_size_idx].

[0085] In an embodiment, disabling_motion_vector_scaling_flag being 1 may specify that a reference motion vector is used without a scaling process that depends on the POC value or spatial resolution for temporal motion vector prediction. disabling_motion_vector_scaling_flag being 0 may specify that a reference motion vector is used with or without a scaling process that depends on the POC value or spatial resolution for temporal motion vector prediction.

[0086] In an embodiment, rpr_resampling_mode equal to 0 may indicate that the interpolated pixels in the reference image are not additionally filtered for motion compensation when the resolution of the current image differs from that of the reference image. rpr_resampling_mode equal to 1 may indicate that the interpolated pixels in the reference image are additionally filtered for motion compensation when the resolution of the current image differs from that of the reference image. rpr_resampling_mode equal to 2 may indicate that the pixels in the reference image are both filtered and interpolated for motion compensation when the resolution of the current image differs from that of the reference image. Other values ​​may be reserved.

[0087] ARCs may be included in the "Baseline / Main" profile. Sub-profiling may be used to remove them if not required for a particular application scenario. Certain limitations may be tolerated. In that regard, certain H.263+ profiles and "Recommended Modes" (previous profiles) contained the limitation that Annex P was only used as an "implicit factor of 4", i.e. binary downsampling in both dimensions. This was sufficient to support fast start (getting I-frames quickly) in videoconferencing.

[0088] In embodiments, all filtering can be done "on the fly" and there may be no or negligible memory bandwidth increase. As a result, there may be no need to put ARC into heterogeneous profiles.

[0089] As discussed in Marrakech in conjunction with JVET-M0135, complex tables etc. may not make sense for capability exchange. Assuming offer-answer and similar limited depth handshakes, the number of options may simply be too large to allow meaningful inter-vendor interoperability. To support ARC in a meaningful way in capability exchange scenarios, a small number of interop points may be used; for example, no ARC, ARC with implicit factor 4, full ARC. Alternatively, one could specify the required support for all ARC and leave the bitstream complexity limit to the higher level SDO.

[0090] Regarding the level, a condition for bitstream compatibility in some embodiments is that the sample count of the upsampled image must fit the level of the bitstream no matter how much upsampling is signaled in the bitstream, and all samples must fit into the upsampled coded image. Note that this was not the case in H263+, where certain samples could not exist.

[0091] 7 is a flowchart of an example process 700 for decoding an encoded video bitstream according to the above embodiment. In some implementations, one or more process blocks of FIG. 7 may be performed by the decoder 210 or the decoder 600. In some implementations, one or more process blocks of FIG. 7 may be performed by another device or group of devices separate from or including the decoder 210 or the decoder 600, such as the encoder 203 or the encoder 500.

[0092] As shown in FIG. 7, process 700 may include obtaining an encoded image from an encoded video bitstream (block 701).

[0093] As further shown in FIG. 7, the process 700 may include decoding the encoded image to generate a decoded image (block 702).

[0094] 7, the process 700 may include obtaining, from the encoded video bitstream, a first flag indicating whether reference picture resampling is enabled (block 703). In an embodiment, the first flag may correspond to the reference_pic_resampling_flag described above.

[0095] 7, process 700 may include determining from the first flag whether reference image resampling is enabled (block 704). If reference image resampling is enabled ("Yes" at block 704), process 700 may proceed to block 705. In an embodiment, if reference image resampling is not enabled, process 700 may decode a video bitstream that was encoded according to a different process.

[0096] 7, the process 700 may include obtaining, from the encoded video bitstream, a second flag indicating whether the reference picture has a constant reference picture size indicated in the encoded video bitstream and a third flag indicating whether the output picture has a constant output picture size indicated in the encoded video bitstream (block 705). In an embodiment, the second flag may correspond to the constant_ref_pic_size_flag described above, and the third flag may correspond to the constant_output_pic_size_flag described above.

[0097] 7, process 700 may include determining whether a second flag indicates that the reference image has a constant reference image size (block 706). If the reference image has a constant reference image size ("YES" at block 706), process 700 may proceed to block 707 and then to block 708. If the reference image does not have a constant reference image size ("NO" at block 706), process 700 may proceed directly to block 708.

[0098] As further shown in FIG. 7, the process 700 may include generating a reference image by resampling the decoded image to have a constant reference image size (block 707).

[0099] 7, the process 700 may include storing the reference image in a decoded image buffer (block 708). If block 707 is not performed, the decoded image may be stored as the reference image without resampling.

[0100] 7, process 700 may include determining whether a third flag indicates that the output image has a constant output image size (block 709). If the output image has a constant output image size ("YES" at block 709), process 700 may proceed to block 710 and then to block 711. If the output image does not have a constant output image size ("NO" at block 709), process 700 may proceed directly to block 711.

[0101] As further shown in FIG. 7, the process 700 may include generating an output image by resampling the decoded image to have a constant output image size (Block 710).

[0102] 7, the process 700 may include outputting an output image (block 711). If block 710 is not performed, the decoded image may be output as the output image without being resampled.

[0103] In an embodiment, the first flag, the second flag, and the third flag may be signaled in a sequence parameter set included in the encoded video bitstream.

[0104] In an embodiment, the process 700 further includes obtaining image resolution information from the encoded video bitstream, the image resolution information indicating at least one of a maximum image resolution and a minimum image resolution.

[0105] In an embodiment, the image resolution information may be signaled in a decoder parameter set included in the encoded video bitstream.

[0106] In an embodiment, the process 700 may further include obtaining the list of image sizes from the encoded video bitstream.

[0107] In an embodiment, the process 700 may further include obtaining an index indicating the image size of the decoded image in a list of image sizes.

[0108] In an embodiment, the list of picture sizes may be signaled in a sequence parameter set included in the encoded video bitstream, and the index may be signaled in a picture parameter set included in the encoded video bitstream.

[0109] In an embodiment, the process 700 may further include obtaining a fourth flag indicating whether motion vector scaling is enabled. In an embodiment, the fourth flag may correspond to the disabling_motion_vector_scaling_flag described above.

[0110] In an embodiment, the fourth flag may be signaled in a picture parameter set included in the encoded video bitstream.

[0111] 7 illustrates example blocks of process 700, in some implementations, process 700 may include additional, fewer, different, or differently arranged blocks than those illustrated in FIG 7. Additionally, or instead, two or more blocks of process 700 may be performed in parallel.

[0112] Additionally, the proposed methods may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium to perform one or more of the proposed methods.

[0113] The techniques described above may be implemented as computer software using computer readable instructions and physically stored on one or more computer readable media. For example, FIG. 8 illustrates a computer system 800 suitable for implementing certain embodiments of the disclosed subject matter.

[0114] Computer software can be encoded using any suitable machine or computer language that may be subject to assembly, compilation, linking, or similar mechanisms to create code containing instructions that can be executed directly or via translation, microcode execution, etc. by a computer central processing unit (CPU), graphics processing unit (GPU), etc.

[0115] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, and the like.

[0116] 8 for computer system 800 are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing the embodiments of the present disclosure, nor should the arrangement of components be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of computer system 800.

[0117] The computer system 800 may include certain human interface input devices. Such human interface input devices may respond to input by one or more human users, for example, via tactile input (keystrokes, swipes, data glove movements, etc.), audio input (voice, clapping, etc.), visual input (gestures, etc.), olfactory input (not shown). Human interface devices may also be used to capture certain media not necessarily directly associated with conscious human input, such as sound (speech, music, ambient sounds, etc.), images (scanned images obtained from a still camera, photographic images, etc.), video (two-dimensional video, three-dimensional video including stereoscopic video, etc.).

[0118] The input human interface devices may include one or more of a keyboard 801 (only one of each shown), a mouse 802, a trackpad 803, a touch screen 810 and associated graphics adapter 850, a data glove 1204, a joystick 805, a microphone 806, a scanner 807, a camera 808.

[0119] The computer system 800 may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses, for example, through haptic output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touch screen 810, data gloves 1204, or joystick 805 may be included, although there may also be haptic feedback devices that do not function as input devices), audio output devices (such as speakers 809, headphones (not shown)), visual output devices (such as screens 810 including cathode ray tube (CRT) screens, liquid crystal display (LCD) screens, plasma screens, organic light emitting diode (OLED) screens, each with or without touch screen input capability, each with or without haptic feedback capability, some of which may be capable of two-dimensional visual output or three or more dimensional output through means such as stereographic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0120] The computer system 800 may also include human accessible storage devices and their associated media, such as optical media including a CD / DVD ROM / RW 820 with CD / DVD or similar media 821, a thumb drive 822, a removable hard drive or solid state drive 823, legacy magnetic media such as tapes and floppy disks (not shown), specialized ROM / ASIC / PLD based devices (not shown) such as security dongles, etc.

[0121] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transitory signals.

[0122] The computer system 800 may also include interfaces to one or more communication networks (955). The networks may be, for example, wireless, wired, optical. The networks may further be local, wide area, metropolitan, vehicular and industrial, real-time, delay tolerant, etc. Examples of networks include local area networks such as Ethernet, wireless LAN, cellular networks including Global System for Mobile Communications (GSM), third generation (3G), fourth generation (4G), fifth generation (5G), long term evolution (LTE), etc., television wired or wireless wide area digital networks including cable television, satellite television, and terrestrial broadcast television, vehicular and industrial including CANBus, etc. A particular network typically requires an external network interface adapter (954) to be attached to a particular general purpose data port or peripheral bus (949) such as the Universal Serial Bus (USB) port of the computer system 800, while others are typically integrated into the core of the computer system (800) by connecting to a system bus as described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system, etc.). As an example, a network 855 may be connected to the peripheral bus 849 using a network interface 854. Using any of these networks, the computer system 800 may communicate with other entities. Such communication may be one-way, receive only (e.g., a television broadcast), one-way transmit only (e.g., a CANbus to a particular CANbus device), or two-way, e.g., to other computer systems using local or wide area digital networks. As noted above, specific protocols and protocol stacks may be used with each of these networks and network interfaces (954).

[0123] The aforementioned human interface devices, human accessible storage devices, and network interfaces may be connected to core 840 of computer system 800 .

[0124] The core 840 may include one or more central processing units (CPUs) 841, graphics processing units (GPUs) 842, specialized programmable processing units in the form of field programmable gate areas (FPGAs) 843, hardware accelerators for specific tasks 844, etc. These devices may connect through a system bus 848 along with read only memory (ROM) 845, random access memory (RAM) 846, internal mass storage such as an internal hard drive that is not accessible to the user, solid state drives (SSDs), etc. 847. In some computer systems, the system bus 848 may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be connected directly to the core's system bus 848 or through a peripheral bus 849. Peripheral bus architectures include peripheral component interconnect (PCI), USB, etc.

[0125] The CPU 841, GPU 842, FPGA 843, and accelerator 844 can execute certain instructions that, in combination, can constitute the aforementioned computer code. That computer code can be stored in ROM 845 or RAM 846. Transient data can also be stored in RAM 846, while permanent data can be stored in, for example, internal mass storage device 847. Rapid storage and retrieval of any memory device can be made possible through the use of cache memory, which can be closely associated with one or more of the CPU 841, GPU 842, mass storage device 847, ROM 845, RAM 846, etc.

[0126] The computer-readable medium can bear computer code for performing various computer-implemented operations. The medium and computer code can be those specially designed and constructed for the purposes of the present disclosure, or they can be of the kind well known and available to those skilled in the computer software arts.

[0127] As an example, and not by way of limitation, a computer system having the architecture 800, and in particular the core 840, can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be the user-accessible mass storage devices introduced above, as well as media associated with specific storage of the core 840 of a non-transitory nature, such as the core internal mass storage device 847 and the ROM 845. Software implementing various embodiments of the present disclosure can be stored in such devices and executed by the core 840. The computer-readable media can include one or more memory devices or chips, depending on the particular needs. The software can cause the core 840, and in particular the processor therein (including a CPU, GPU, FPGA, etc.), to perform certain processes or certain portions of certain processes described herein, including defining data structures stored in the RAM 846 and modifying such data structures according to processes defined by the software. Additionally or alternatively, a computer system may provide functionality as a result of being logically hardwired or embodied in circuitry (e.g., accelerator 844) that can operate in place of or together with software to perform certain processes or certain portions of certain processes described herein. References to software may include logic, and vice versa, as appropriate. References to computer-readable media may encompass circuitry (such as integrated circuits (ICs)) that stores software for execution, circuitry that embodies logic for execution, or both, as appropriate. The present disclosure encompasses any suitable combination of hardware and software.

[0128] While this disclosure describes several exemplary embodiments, there are modifications, permutations, and various substitute equivalents that are within the scope of the disclosure. It will thus be appreciated that those skilled in the art will be able to devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the disclosure and are therefore within its spirit and scope. [Explanation of symbols]

[0129] 100 Communication Systems 110 Terminal 120 terminals 130 terminals 140 terminals 150 Network 201 Source 202 Sample Stream 203 Encoder 204 Video Bitstream 205 Streaming Server 206 Streaming Client 207 Video Bitstream 208 Streaming Client 209 Video Bitstream 210 Decoder 211 outgoing video sample stream 212 Display 213 Capture Subsystem 310 Receiver 312 Channels 315 Buffer Memory 320 Parser 321 Symbols 351 Scaler / Descaler Unit 352 Intra Image Prediction Unit 353 Motion Compensation Prediction Unit 355 Aggregation Device 356 Loop Filter Unit 357 Reference Image Buffer 358 Current Image 430 Video Coder / Source Coder 432 Encoding Engine 433 Decoder 434 Reference Image Memory 435 Predictor 440 Transmitter 443 coded video sequence 445 Entropy Coder 450 Controller 460 Communication Channels 500 Encoder 501 Down Sampler 502 Image Partitioner 503 Inverse quantizer 504 Entropy Coder 505 In-Loop Filter 506 Intra Predictor 507 Decoded Image Buffer 508 Resampler 509 Inter Predictor 600 Decoder 601 Encoded Image Buffer 602 Video Syntax Parser 603 Inverse quantizer 604 In-Loop Filter 605 Decoded Image Buffer 606 Resampler 607 Inter Predictor 608 Intra Predictor 609 Upsampler 800 Computer Systems 801 Keyboard 802 Mouse 803 Trackpad 805 Joystick 806 Mike 807 Scanner 808 Camera 809 Speaker 810 Touch Screen 821 Medium 822 Thumb Drive 823 Solid State Drive 840 cores 841 Central Processing Unit (CPU) 842 Graphics Processing Unit (GPU) 843 Field Programmable Gate Area (FPGA) 844 Hardware accelerators for specific tasks 845 Read-Only Memory (ROM) 846 Random Access Memory (RAM) 847 Internal mass storage 848 System Bus 849 Surrounding Bus 850 Graphics Adapter 854 Network Interface 855 Network 949 Surrounding Bus 954 Network Interface 955 Communication Network 1204 Data Gloves

Claims

1. 1. A method for generating a video bitstream using at least one processor, the method comprising: determining a first flag indicating whether reference image resampling is enabled; determining a syntax element indicating a reference image resampling mode; determining whether the resolution of the image to be encoded differs from the resolution of the reference image; based on a determination that the resolution of the image to be encoded differs from the resolution of the reference image, encoding the image according to the reference image resampling mode; A method comprising:

2. A method for generating a video bitstream using at least one processor, the method comprising: determining a first flag indicating whether reference image resampling is enabled; if the reference image resampling is enabled, determining a syntax element indicating a reference image resampling mode; encoding the image to be encoded according to the reference image resampling mode when the resolution of the image to be encoded is different from the resolution of the reference image; A method comprising:

3. A method as described in claim 1 or 2, wherein the first flag is included in a sequence parameter set included in the encoded video bitstream.

4. A method as described in claim 1 or 2, wherein the syntax element is included in a picture parameter set included in the encoded video bitstream.

5. A method as described in claim 1 or 2, wherein, based on the reference image resampling mode being a first mode, interpolated pixels in the reference image are not additionally determined for motion compensation during decoding.

6. A method as described in claim 1 or 2, wherein, based on the reference image resampling mode being the second mode, interpolated pixels in the reference image are additionally filtered for motion compensation during decoding.

7. A method as described in claim 1 or 2, wherein, based on the reference image resampling mode being the third mode, pixels in the reference image are filtered and interpolated for motion compensation during decoding.

8. The method of claim 1, wherein the video bitstream further includes a list of image sizes.

9. The method of claim 8, further comprising an index indicating an image size of the image to be decoded within the list of image sizes.

10. The list of image sizes is included in a sequence parameter set included in the video bitstream, The method of claim 9 , wherein the index is included in a picture parameter set included in the video bitstream.

Images