Signaling reference picture resampling with constant window size indication in video bitstreams

Abstract

To provide a method, a device, and a non-transitory computer-readable medium for decoding an encoded video bitstream.SOLUTION: A decoding method includes the steps of obtaining a first flag indicating whether a constant picture size is used in a coded video sequence including a current picture, obtaining a second flag indicating whether a compatibility window size is signaled on the basis of the first flag indicating that a constant picture size is not used, obtaining a compatibility window size on the basis of the second flag indicating that the compatibility window size is signaled, determining a resampling ratio between the current picture and a reference picture on the basis of the compatibility window size, and performing reference picture resampling on the current picture using the resampling ratio.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Application No. 62 / 903,601, filed September 20, 2019, and U.S. Patent Application No. 17 / 009,979, filed September 2, 2020, the entire contents of which are incorporated herein by reference.

[0002] The disclosed subject matter relates to video coding and decoding, and more particularly to signaling picture or picture portion sizes that can vary from picture to picture or picture portion to picture portion. [Background technology]

[0003] Video coding and decoding using inter-picture prediction with motion compensation is known. Uncompressed digital video can consist of a series of pictures, each having spatial dimensions of, for example, 1920 x 1080 luminance samples and associated chrominance samples. The series of pictures can have a predetermined or variable picture rate (also informally known as a frame rate), for example, 60 pictures per second or 60 Hz. Uncompressed video has significant bitrate requirements. For example, 1080p60 4:2:0 video (1920 x 1080 luminance sample resolution at a 60 Hz frame rate) with 8 bits per sample requires a bandwidth approaching 1.5 Gbit / s. One hour of such video requires more than 600 GBytes of storage space.

[0004] One goal of video coding and decoding can be the reduction of redundancy in an input video signal through compression. Compression can help reduce the aforementioned bandwidth or storage space requirements by two or more orders of magnitude, in some cases. Both lossless and lossy compression, as well as combinations thereof, can be used. Lossless compression refers to a technique that allows an exact copy of the original signal to be reconstructed from a compressed version of the original signal. When lossy compression is used, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signal is small enough to make the reconstructed signal useful for its intended application. For video, lossy compression is widely adopted. The amount of acceptable distortion depends on the application; for example, users of certain consumer streaming applications may tolerate higher distortion than users of television contribution applications. The achievable compression ratio can reflect that higher acceptable / tolerable distortion can result in a higher compression ratio.

[0005] Video encoders and decoders can utilize techniques from several broad categories, including, for example, motion compensation, transforms, quantization, and entropy coding, some of which are introduced below.

[0006] Historically, video encoders and decoders have mostly tended to operate with a predefined picture size that remains constant and is defined for a coded video sequence (CVS), group of pictures (GOP), or similar multi-picture time frame. For example, in MPEG-2, system designs are known to change the horizontal resolution (and thereby the picture size) depending on factors such as scene activity, but only for I-pictures and therefore generally for GOPs. Resampling of reference pictures to use different resolutions within a CVS is known, for example, from ITU-T Rec. H.263 Annex P. However, here, the picture size remains the same; only the reference pictures are resampled, potentially resulting in only a portion of the picture canvas being used (in the case of downsampling) or only a portion of the scene being captured (in the case of upsampling). Furthermore, H.263 Annex Q allows for resampling of individual macroblocks upward or downward by a factor of two (in each dimension). Again, the picture size remains the same. The size of a macroblock is fixed in H.263 and therefore does not need to be signaled.

[0007] Picture size changes in predicted images have become more mainstream in modern video coding. For example, VP9 allows for reference picture resampling and resolution changes for the entire picture. Similarly, certain proposals made for VVC (including, for example, Hendry et al., "On adaptive resolution change (ARC) for VVC," Joint Video Team document JVET-M 0135-v 1, January 9-19, 2019, which is incorporated herein in its entirety) allow for resampling of the entire reference picture to different—higher or lower—resolutions. In that document, different candidate resolutions are proposed to be coded in the sequence parameter set and referenced by per-picture syntax elements in the picture parameter set. Summary of the Invention [Means for solving the problem]

[0008] In one embodiment, a method for decoding an encoded video bitstream using at least one processor is provided, the method including the steps of: obtaining a first flag indicating whether a constant picture size is used in a coded video sequence including a current picture; decoding the current picture without performing reference picture resampling based on the first flag indicating that a constant picture size is used; obtaining a second flag indicating whether a compatibility window size is signaled based on the first flag indicating that a constant picture size is not used; obtaining a compatibility window size based on the second flag indicating that a compatibility window size is signaled; determining a resampling ratio between the current picture and the reference picture based on the compatibility window size; and performing reference picture resampling on the current picture using the resampling ratio.

[0009] In one embodiment, a device for decoding an encoded video bitstream comprises: at least one memory configured to store program code; and at least one processor configured to read the program code and operate as instructed by the program code, the program code including: first obtaining code configured to cause the at least one processor to obtain a first flag indicating whether a constant picture size is used in a coded video sequence including a current picture; decoding code configured to cause the at least one processor to decode the current picture without performing reference picture resampling based on the first flag indicating that a constant picture size is used; second obtaining code configured to cause the at least one processor to obtain a second flag indicating whether a compatibility window size is signaled based on the first flag indicating that a constant picture size is not used; and execution code configured to cause the at least one processor to obtain a compatibility window size based on the second flag indicating that a compatibility window size is signaled, determine a resampling ratio between the current picture and the reference picture based on the compatibility window size, and perform reference picture resampling on the current picture using the resampling ratio.

[0010] In one embodiment, a non-transitory computer-readable medium is provided that stores instructions that, when executed by one or more processors of a device for decoding an encoded video bitstream, cause the one or more processors to: obtain a first flag indicating whether a constant picture size is used in a coded video sequence that includes a current picture; decode the current picture without performing reference picture resampling based on the first flag indicating that a constant picture size is used; obtain a second flag indicating whether a compatibility window size is signaled based on the first flag indicating that a constant picture size is not used; obtain a compatibility window size based on the second flag indicating that a compatibility window size is signaled; determine a resampling ratio between the current picture and the reference picture based on the compatibility window size; and perform reference picture resampling on the current picture using the resampling ratio.

[0011] Further features, nature and various advantages of the disclosed subject matter will be more apparent from the following detailed description and accompanying drawings. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram of a simplified block diagram of a communication system according to one embodiment. [Figure 2] FIG. 1 is a schematic diagram of a simplified block diagram of a communication system according to one embodiment. [Figure 3] FIG. 2 is a schematic diagram of a simplified block diagram of a decoder according to one embodiment. [Figure 4] FIG. 2 is a schematic diagram of a simplified block diagram of an encoder according to one embodiment. [Figure 5] FIG. 10 is a schematic diagram of options for signaling ARC / RPR parameters according to an embodiment. [Figure 6A] FIG. 2 is a schematic diagram of an example syntax table according to one embodiment. [Figure 6B] FIG. 2 is a schematic diagram of an example syntax table according to one embodiment. [Figure 7] 1 is a schematic diagram of signaling picture size and compatibility window in SPS according to one embodiment; [Figure 8] 1 is a schematic diagram of signaling picture size and compatibility window in PPS according to an embodiment; [Figure 9] 1 is a flowchart of an example process for decoding an encoded video bitstream according to one embodiment. [Figure 10] FIG. 1 is a schematic diagram of a computer system according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] 1 shows a simplified block diagram of a communication system (100) according to one embodiment of the present disclosure. The system (100) may include at least two terminals (110-120) interconnected via a network (150). For one-way data transmission, a first terminal (110) may locally code video data for transmission to another terminal (120) via the network (150). A second terminal (120) may receive the coded video data of the other terminal from the network (150), decode the coded data, and display the recovered video data. One-way data transmission may be common in media serving applications, for example.

[0014] 1 shows a second pair of terminals (130, 140) provided to support bidirectional transmission of coded video, such as may occur during a video conference. With respect to the bidirectional transmission of data, each terminal (130, 140) may code video data captured at a local location for transmission to the other terminal over a network (150). Each terminal (130, 140) may also receive coded video data transmitted by the other terminal, decode the coded data, and display the recovered video data on a local display device.

[0015] In FIG. 1 , the terminals (110-140) may be illustrated as servers, personal computers, and smartphones, although the principles of the present disclosure need not be so limited. Embodiments of the present disclosure find application with laptop computers, tablet computers, media players, and / or dedicated videoconferencing equipment. The network (150) represents any number of networks that convey coded video data between the terminals (110-140), including, for example, wired and / or wireless communication networks. The communication network (150) may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this description, the architecture and topology of the network (150) may not be important to the operation of the present disclosure unless otherwise described herein.

[0016] 2 illustrates the placement of a video encoder and decoder in a streaming environment as an example of an application for the disclosed subject matter, which may be equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0017] The streaming system may include a video source (201), e.g., a capture subsystem (213), which may include a digital camera, that generates an uncompressed video sample stream (202). The sample stream (202), depicted as a thick line to emphasize its high data content compared to the encoded video bitstream, may be processed by an encoder (203) coupled to the camera (201). The encoder (203) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video bitstream (204), depicted as a thin line to emphasize its lower data content compared to the sample stream, may be stored on a streaming server (205) for future use. One or more streaming clients (206, 208) may access the streaming server (205) to retrieve copies (207, 209) of the encoded video bitstream (204). The client (206) can include a video decoder (210) that decodes an incoming copy of the encoded video bitstream (207) and forms an outgoing video sample stream (211) that can be rendered on a display (212) or other rendering device (not shown). In some streaming systems, the video bitstreams (204, 207, 209) can be encoded according to a particular video coding / compression standard. Examples of these standards include ITU-T Recommendation H.265. A video coding standard informally known as Versatile Video Coding, or VVC, is under development. The disclosed subject matter can be used in conjunction with VVC.

[0018] FIG. 3 may be a functional block diagram of a video decoder (210) according to one embodiment of the present disclosure.

[0019] The receiver (310) may receive one or more codec video sequences to be decoded by the decoder (210), or in the same or other embodiments, may receive one coded video sequence at a time, where decoding of each coded video sequence is independent of other coded video sequences. The coded video sequences may be received from a channel (312), which may be a hardware / software link to a storage device that stores the encoded video data. The receiver (310) may also receive encoded video data along with other data, such as coded audio data and / or auxiliary data streams, which may be forwarded to their respective consuming entities (not shown). The receiver (310) may separate the coded video sequences from other data. To combat network jitter, a buffer memory (315) may be coupled between the receiver (310) and the entropy decoder / parser (320) (hereinafter "parser"). When the receiver 310 is receiving data from a store-and-forward device of sufficient bandwidth and controllability or from an isosynchronous network, the buffer 315 may not be needed or may be small. For use with best-effort packet networks such as the Internet, the buffer 315 may be needed, may be relatively large, and preferably may have an adaptive size.

[0020] The video decoder (210) may include a parser (320) for reconstructing symbols (321) from the entropy-coded video sequence. These symbol categories include information used to manage the operation of the decoder (210) and, potentially, information for controlling a rendering device, such as a display (212), which is not an integral part of the decoder but may be coupled to it, as shown in FIG. 3. Control information for the rendering device may be in the form of a supplemental enhancement information (SEI) message or a video usability information (VUI) parameter set fragment (not shown). The parser (320) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may follow a video coding technique or standard and may follow principles well known to those skilled in the art, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (320) may extract a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. A subgroup may include a group of pictures (GOP), a picture, a subpicture, a tile, a slice, a brick, a macroblock, a coding tree unit (CTU), a coding unit (CU), a block, a transform unit (TU), a prediction unit (PU), etc. A tile may refer to a rectangular region of a CU / CTU within a particular tile column and row within a picture. A brick may refer to a rectangular region of a CU / CTU row within a particular tile. A slice may refer to one or more bricks of a picture contained in a NAL unit. A subpicture may refer to a rectangular region of one or more slices within a picture. The entropy decoder / parser may also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from a coded video sequence.

[0021] The parser (320) may perform entropy decoding / parsing operations on the video sequence received from the buffer (315) to form symbols (321).

[0022] The reconstruction of the symbols (321) may involve several different units depending on the type of coded video picture or portion thereof (e.g., inter-picture and intra-picture, inter-block and intra-block, etc.) and other factors. Which units participate and how may be controlled by subgroup control information parsed by the parser (320) from the coded video sequence. The flow of such subgroup control information between the parser (320) and the following units is not depicted for clarity.

[0023] In addition to the functional blocks already mentioned, decoder 210 may be conceptually subdivided into several functional units, as described below. In actual implementations subject to commercial constraints, many of these units may interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, the following conceptual subdivision into functional units is appropriate.

[0024] The first unit is a scalar / inverse transform unit (351), which receives quantized transform coefficients as symbols (321) from the parser (320), including which transform to use, block size, quantization coefficients, quantization scaling matrices, etc., as well as control information. The scalar / inverse transform unit can output blocks containing sample values ​​that can be input to an aggregator (355).

[0025] In some cases, the output samples of the scaler / inverse transform (351) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from already reconstructed pictures but can use prediction information from already reconstructed portions of the current picture. Such prediction information may be provided by an intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) generates blocks of the same size and shape as the block under reconstruction using surrounding already reconstructed information fetched from the current (partially reconstructed) picture (358). The aggregator (355) adds the prediction information generated by the intra-prediction unit (352) to the output sample information provided by the scaler / inverse transform unit (351), possibly on a sample-by-sample basis.

[0026] In other cases, the output samples of the scaler / inverse transform unit (351) may relate to an inter-coded, potentially motion-compensated block. In such cases, the motion-compensated prediction unit (353) may access the reference picture memory (357) to fetch samples used for prediction. After motion-compensating the fetched samples according to the symbols (321) associated with the block, these samples may be added by the aggregator (355) to the output of the scaler / inverse transform unit (in this case, referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory from which the motion compensation unit fetches prediction samples may be controlled by motion vectors available to the motion compensation unit in the form of symbols (321), which may have, for example, an X component, a Y component, and a reference picture component. Motion compensation may also include interpolation of sample values ​​fetched from the reference picture memory when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, etc.

[0027] The output samples of the aggregator (355) may be subjected to various loop filtering techniques in a loop filter unit (356). Video compression techniques may include in-loop filter techniques controlled by parameters contained in the coded video bitstream and made available to the loop filter unit (356) as symbols (321) from the parser (320), but may also respond to meta-information obtained during decoding of a coded picture or previous portion (in decoding order) of a coded video sequence, as well as respond to sample values ​​that have already been reconstructed and loop filtered.

[0028] The output of the loop filter unit (356) may be a sample stream that may be output to a rendering device (212) and stored in a reference picture memory for use in future inter-picture prediction.

[0029] Once a particular coded picture is fully reconstructed, it can be used as a reference picture for future prediction. Once a coded picture is fully reconstructed and the coded picture has been identified as a reference picture (e.g., by the parser (320)), the current reference picture (358) can become part of the reference picture buffer (357), and a new current picture memory can be reallocated before beginning reconstruction of a subsequent coded picture.

[0030] Video decoder 210 may perform decoding operations according to a predetermined video compression technique, which may be documented in a standard such as ITU-T Rec. H.265. The coded video sequence may conform to the syntax defined by the video compression technique or standard being used, in the sense that it adheres to the syntax of the video compression technique or standard as defined in the video compression technique document or standard, particularly as defined in a profile document therein. Compliance may also require that the complexity of the coded video sequence be within boundaries defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level may, in some cases, be further constrained by a hypothetical reference decoder (HRD) specification and metadata for HRD buffer management signaled in the coded video sequence.

[0031] In one embodiment, the receiver (310) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the coded video sequence. The additional data may be used by the video decoder (210) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or SNR enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0032] FIG. 4 may be a functional block diagram of a video encoder (203) according to one embodiment of the present disclosure.

[0033] The encoder (203) may receive video samples from a video source (201) (not part of the encoder) that may capture video images to be coded by the encoder (203).

[0034] The video source (201) may provide a source video sequence to be coded by the encoder (203) in the form of a digital video sample stream, which may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source (201) may be a storage device that stores already preprocessed video. In a video conferencing system, the video source (203) may be a camera that captures local image information as a video sequence. The video data may be provided as multiple individual pictures that, when viewed sequentially, provide motion. The pictures themselves may be organized as a spatial array of pixels, where each pixel may contain one or more samples, depending on the sampling structure, color space, etc., in use. Those skilled in the art will readily understand the relationship between pixels and samples. The following description focuses on samples.

[0035] According to one embodiment, the encoder (203) may code and compress pictures of a source video sequence into a coded video sequence (443) in real time or under any other time constraint required by the application. Enforcing an appropriate coding rate is one function of the controller (450). The controller controls and is operatively coupled to other functional units, as described below. For clarity, coupling is not depicted. Parameters set by the controller may include rate control-related parameters (picture skip, quantizer, lambda value for rate-distortion optimization techniques, ...), picture size, group-of-picture (GOP) layout, maximum motion vector search range, etc. Those skilled in the art can readily identify other functions of the controller (450) that may be relevant to an optimized video encoder (203) for a given system design.

[0036] Some video encoders operate in what those skilled in the art readily recognize as a "coding loop." As an overly simplified explanation, the coding loop can consist of an encoding portion of the encoder (430) (hereinafter, the "source coder") (responsible for forming symbols based on the input picture to be coded and reference pictures) and a (local) decoder (433) embedded in the encoder (203) that reconstructs the symbols to form sample data that also forms the (remote) decoder (because, in the video compression techniques contemplated by the disclosed subject matter, any compression between the symbols and the coded video bitstream is lossless). That reconstructed sample stream is input to a reference picture memory (434). Because decoding the symbol stream yields bit-exact results independent of the decoder location (local or remote), the reference picture buffer contents are also bit-exact between the local and remote encoders. In other words, the predictive portion of the encoder "sees" the exact same sample values ​​as the decoder "sees" when using prediction during decoding. This basic principle of reference picture synchronization (and the resulting drift if synchronization cannot be maintained, eg, due to channel errors) is well known to those skilled in the art.

[0037] The operation of the "local" decoder (433) may be the same as the operation of the "remote" decoder (210) already described in detail above in connection with Figure 3. However, and referring briefly to Figure 4 as well, because symbols are available and the encoding / decoding of symbols for the coded video sequence by the entropy coder (445) and parser (320) may be lossless, the entropy decoding portion of the decoder (210), including the channel (312), receiver (310), buffer (315), and parser (320), may not be implemented entirely in the local decoder (433).

[0038] An observation that can be made at this point is that any decoder technology, with the exception of parsing / entropy decoding, that is present in a decoder must necessarily be present in substantially identical functional form in the corresponding encoder. For this reason, the disclosed subject matter focuses on decoder operation. Descriptions of encoder technology may be omitted, as they are the inverse of the decoder technology that has been comprehensively described. Only in certain areas is a more detailed description required and is provided below.

[0039] As part of its operation, the source coder (430) may perform motion-compensated predictive coding, which predictively codes an input frame relative to one or more already-coded frames from the video sequence designated as “reference frames.” In this manner, the coding engine (432) codes differences between pixel blocks of the input frame and pixel blocks of reference frames that may be selected as prediction references for the input frame.

[0040] The local video decoder (433) may decode the coded video data of frames that may be designated as reference frames based on symbols formed by the source coder (430). The operation of the coding engine (432) may preferably be a lossy process. If the coded video data can be decoded by a video decoder (not shown in FIG. 4), the reconstructed video sequence may generally be a replica of the source video sequence with some errors. The local video decoder (433) may replicate the decoding process that may be performed by the video decoder with respect to the reference frames and may cause the reconstructed reference frames to be stored in a reference picture cache (434). In this way, the encoder (203) may locally store copies of reconstructed reference frames that have common content as reconstructed reference frames obtained by a far-end video decoder (without transmission errors).

[0041] The predictor (435) may perform the prediction search for the coding engine (432). That is, for a new frame to be coded, the predictor (435) may search the reference picture memory (434) for sample data (as candidate reference pixel blocks) or specific metadata, such as reference picture motion vectors, block shapes, etc., that can serve as suitable prediction references for the new picture. The predictor (435) may operate on a sample block by pixel block basis to find an appropriate prediction reference. In some cases, as determined by the search results obtained by the predictor (435), the input picture may have prediction references drawn from multiple reference pictures stored in the reference picture memory (434).

[0042] The controller (450) may manage the coding operations of the video coder (430), including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0043] The outputs of all the aforementioned functional units may be subjected to entropy coding in an entropy coder (445), which converts the symbols produced by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques known to those skilled in the art, such as Huffman coding, variable length coding, arithmetic coding, etc.

[0044] The transmitter (440) may buffer the coded video sequence formed by the entropy coder (445) and preprocess it for transmission over a communication channel (460), which may be a hardware / software link to a storage device that stores the encoded video data. The transmitter (440) may merge the coded video data from the video coder (430) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0045] The controller (450) may manage the operation of the encoder (203). During decoding, the controller (450) may assign a particular coded picture type to each coded picture, which may affect the coding technique that may be applied to each picture. For example, pictures may often be assigned as one of the following frame types:

[0046] An intra picture (I-picture) may be one that can be coded and decoded without using any other frame in the sequence as a source of prediction. Some video codecs allow different types of intra pictures, including, for example, independent decoder refresh pictures. Those skilled in the art are aware of these variations of I-pictures and their respective uses and characteristics.

[0047] Predictive pictures (P pictures) may be coded and decoded using intra- or inter-prediction, which uses at most one motion vector and reference index to predict the sample values ​​of each block.

[0048] Bidirectionally predicted pictures (B-pictures) may be coded and decoded using intra- or inter-prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multi-predicted pictures may use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0049] A source picture may generally be spatially subdivided into multiple sample blocks (e.g., 4x4, 8x8, 4x8, or 16x16 blocks) and coded block by block. Blocks may be predictively coded relative to other (already coded) blocks as determined by the coding assignment applied to the block's respective picture. For example, blocks of an I-picture may be non-predictively coded or predictively coded relative to already coded blocks of the same picture (spatial prediction or intra-prediction). Pixel blocks of a P-picture may be non-predictively coded by spatial prediction relative to one already coded reference picture or by temporal prediction. Blocks of a B-picture may be non-predictively coded by spatial prediction relative to one or two already coded reference pictures or by temporal prediction.

[0050] The video coder (203) may perform coding operations according to a predetermined video coding technique or standard, such as ITU-T Rec. H.265. In its operation, the video coder (203) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the coded video data may conform to a syntax defined by the video coding technique or standard being used.

[0051] In one embodiment, the transmitter (440) may transmit additional data along with the encoded video. The video coder (430) may include such data as part of the coded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, Supplemental Enhancement Information (SEI) messages, Visual Usability Information (VUI) parameter set fragments, etc.

[0052] Recently, the compressed domain aggregation or extraction of multiple semantically independent picture portions into a single video picture has attracted some attention. In particular, for example, in the context of 360 coding or certain surveillance applications, multiple semantically independent source pictures (e.g., six cubic surfaces of a cubic-projected 360 scene or individual camera inputs in the case of a multi-camera surveillance setup) may require separate adaptive resolution settings to address different scene-specific activity at a given time. In other words, an encoder can choose to use different resampling factors for different semantically independent pictures that form the entire 360 ​​scene or surveillance scene at a given time. When combined into a single picture, it requires that resampling of reference pictures be performed and adaptive resolution coding signaling be available for the coded picture portions.

[0053] Below are some terms that will be referenced in the remainder of this description.

[0054] A subpicture may refer to a rectangular arrangement of samples, blocks, macroblocks, coding units, or similar entities that may be semantically grouped and coded independently at varying resolutions. One or more subpictures may form a picture. One or more coded subpictures may form a coded picture. One or more subpictures may be assembled into a picture, or one or more subpictures may be extracted from a picture. In certain circumstances, one or more coded subpictures may be assembled in the compressed domain without transcoding down to the sample level into a coded picture, or in the same or other cases, one or more coded subpictures may be extracted from a picture coded in the compressed domain.

[0055] Reference picture resampling (RPR) or adaptive resolution change (ARC) may refer to mechanisms that allow changing the resolution of pictures or sub-pictures in a coded video sequence, for example by reference picture resampling. Hereinafter, RPR / ARC parameters refer to the control information needed to perform adaptive resolution change, which may include, for example, filter parameters, scaling factors, resolution of output and / or reference pictures, various control flags, etc.

[0056] In an embodiment, coding and decoding may be performed with respect to a single semantically independent coded video picture. Before describing the implications of coding / decoding multiple sub-pictures with independent RPR / ARC parameters and the additional complexities that this implies, options for signaling the RPR / ARC parameters shall be described.

[0057] Referring to Figure 5, several embodiments for signaling RPR / ARC parameters are shown. As described by each of the embodiments, these embodiments may have certain advantages and disadvantages in terms of coding efficiency, complexity, and architecture. A video coding standard or technology may select one or more of these embodiments or options known from related art for signaling RPR / ARC parameters. It is contemplated that the embodiments may not be mutually exclusive and may be interchanged based on application needs, related standard technology, or encoder preference.

[0058] The classes of RPR / ARC parameters may include:

[0059] - separate or combined up / down sample factors in the X and Y dimensions

[0060] - Up / down sample factor with the addition of a time dimension, indicating a constant speed of zooming in / out for a given number of pictures

[0061] - Either of the above two may be accompanied by the coding of one or more possibly short syntax elements that can point to a table containing the factors.

[0062] -Resolution of the X or Y dimension, combined or separately, of an input picture, an output picture, a reference picture, a coded picture sample, a block, a macroblock, a coding unit (CU), or any other suitable unit of granularity. If there are two or more resolutions (e.g., a resolution in the input picture, a resolution in the reference picture, etc.), in certain cases one set of values ​​can be inferred from the other set of values. Such can be gated, for example, by the use of flags. See below for more detailed examples.

[0063] "Warping" coordinates similar to those used in H.263 Annex P, again at an appropriate granularity as described above. H.263 Annex P specifies one efficient way to code such warping coordinates, although other, potentially more efficient ways can conceivably be devised. For example, the variable-length reversible "Huffman"-type coding of Annex P's warping coordinates could be replaced by an appropriate-length binary coding, where the length of the binary code could be derived from the maximum picture size, and possibly multiplied by a particular factor and offset by a particular value, e.g., to allow "warping" outside the bounds of the maximum picture size.

[0064] -Upsample filter parameters or downsample filter parameters. In embodiments, there may be only a single filter for upsampling and / or downsampling. However, in embodiments, it may be desirable to allow more flexibility in filter design, which may require signaling of filter parameters. Such parameters may be selected by an index in a list of possible filter designs, the filter may be fully defined (e.g., by a list of filter coefficients using an appropriate entropy coding technique), the filter may be implicitly selected by the up / downsample ratio, and the up / downsample ratio may be signaled accordingly based on any of the aforementioned mechanisms, etc.

[0065] In the following, the description assumes the coding of a finite set of up / downsample coefficients (the same coefficients to be used in both the X and Y dimensions) indicated by a codeword. The codeword may be variable-length coded, for example, using extended Golomb codes common to certain syntax elements in video coding specifications such as H.264 and H.265. One suitable mapping of values ​​to up / downsample coefficients may, for example, follow Table 1.

[0066] [Table 1]

[0067] Many similar mappings can be devised according to the needs of the application and the capabilities of the upscaling and downscaling mechanisms available in the video compression technology or standard. The table can be extended to more values. The values ​​may also be represented by entropy coding mechanisms other than the extended Golomb code, for example using binary coding. This may have particular advantages when the resampling factor is targeted outside the video processing engine (primarily the encoder and decoder) itself, for example by MANE. It should be noted that in situations where no resolution change is required, the extended Golomb code, which is short and only one bit in the above table, can be selected. This may have coding efficiency advantages over using binary code in the most general cases.

[0068] The number of entries in a table, as well as their semantics, may be fully or partially configurable. For example, the basic outline of the table may be conveyed in a "high" parameter set, such as a sequence or decoder parameter set. In an embodiment, one or more such tables may be defined in a video coding technology or standard, and may be selected, for example, by a decoder or sequence parameter set.

[0069] The following describes how the upsample / downsample coefficients (ARC information) coded as described above can be included in a video coding technique or standard syntax. Similar considerations may apply to one or several codewords controlling an up / downsample filter. See below for a description of when a relatively large amount of data is required for a filter or other data structure.

[0070] As shown in Figure 5, H.263 Annex P includes ARC information (502) in the form of four warping coordinates into the picture header (501), specifically in the H.263 PLUSPTYPE (503) header extension. This can be a wise design choice when a) there is a picture header available and b) frequent changes to the ARC information are expected. However, the overhead when using H.263-type signaling can be very high, and because of the temporal nature of picture headers, scaling factors may not be relevant across picture boundaries.

[0071] In the same or other embodiments, signaling of ARC parameters can follow a detailed example as outlined in Figures 6A-6B, which depict a syntax diagram of this type of expression using a notation that roughly follows C-style programming, e.g., as used in video coding standards since at least 1993. Bold lines indicate syntax elements present in the bitstream, while non-bold lines often indicate control flow or variable setting.

[0072] As shown in Figure 6A, the tile group header (601) as a typical syntax structure for a header applicable to a (possibly rectangular) portion of a picture can conditionally include the variable-length extended Golomb coding syntax element dec_pic_size_idx (602) (depicted in bold). The presence of this syntax element in the tile group header can be gated upon the use of adaptive resolution (603), a flag value not depicted here in bold, which means that the flag is present in the bitstream at the point where it occurs in the syntax diagram. Whether adaptive resolution is used for this picture or its portion can be signaled in any high-level syntax structure, inside or outside the bitstream. In the example shown, whether adaptive resolution is used for this picture or its portion is signaled in the sequence parameter set, as outlined below.

[0073] Referring to Figure 6B, an excerpt of a sequence parameter set (610) is also shown. The first syntax element shown is adaptive_pic_resolution_change_flag (611). If true, that flag can indicate the use of adaptive resolution, which may require specific control information. In this example, such control information is conditionally present based on the value of the flag based on an if() statement in the parameter set (612) and the tile group header (601).

[0074] If adaptive resolution is used, in this example, it is the output resolution in samples that is coded (613). The number 613 refers to both output_pic_width_in_luma_samples and output_pic_height_in_luma_samples, which together can specify the resolution of the output picture. Elsewhere in the video coding technology or standard, specific limits on either value can be specified. For example, the level specification might limit the number of total output samples, which could be the product of the values ​​of two syntax elements for output samples. Also, a particular video coding technology or standard, or an external technology or standard such as a system standard, might limit the numbering range (e.g., one or both dimensions must be divisible by a power of 2) or the aspect ratio (e.g., width and height must have a relationship such as 4:3 or 16:9). Such limits may be introduced to facilitate hardware implementation or for other reasons and are well known in the art.

[0075] In certain applications, it may be desirable for the encoder to instruct the decoder to use a particular reference picture size rather than implicitly assuming that size is the output picture size. In this example, the syntax element reference_pic_size_present_flag (614) gates the conditional presence of the reference picture dimensions (615) (again, the numbers refer to both width and height).

[0076] Finally, a table of possible decoded picture widths and heights is shown. Such a table may be represented, for example, by the table notation (num_dec_pic_size_in_luma_samples_minus 1) (616). The "minus 1" may refer to the interpretation of the value of that syntax element. For example, if the coded value is 0, there is one table entry. If the value is 5, there are six table entries. For each "line" in the table, the width and height of the decoded picture are included afterwards in the syntax (617).

[0077] The syntax element dec_pic_size_idx (602) in the tile group header can be used to index the presented table entries (617), thereby allowing different decoding sizes—actually zoom ratios—for each tile group.

[0078] Certain video coding techniques or standards, such as VP9, ​​support spatial scalability by implementing a particular type of reference picture resample (signaled quite differently than the disclosed subject matter) in conjunction with temporal scalability to enable spatial scalability. In particular, certain reference pictures may be upsampled to a higher resolution using ARC-type techniques to form the basis of spatial enhancement layers. These upsampled pictures may then be refined using conventional prediction mechanisms at higher resolutions to add detail.

[0079] The embodiments discussed herein may be used in such environments. In some cases, in the same or other embodiments, values ​​in the NAL unit header, e.g., the Temporal ID field, may be used to indicate not only temporal but also spatial layers. Doing so may have certain advantages for certain system designs. For example, existing selective forwarding units (SFUs) that are configured and optimized for temporal layer selective forwarding based on NAL unit header Temporal ID values ​​may be used without modification for scalable environments. To enable this, there may be a requirement for the mapping between coded picture sizes and temporal layers to be indicated by the Temporal ID field in the NAL unit header.

[0080] Recently, the compressed domain aggregation or extraction of multiple semantically independent picture portions into a single video picture has attracted some attention. In particular, for example, in the context of 360 coding or certain surveillance applications, multiple semantically independent source pictures (e.g., six cubic surfaces of a cubic-projected 360 scene or individual camera inputs in the case of a multi-camera surveillance setup) may require separate adaptive resolution settings to address different scene-specific activity at a given time. In other words, an encoder can choose to use different resampling factors for different semantically independent pictures that form the entire 360 ​​scene or surveillance scene at a given time. When combined into a single picture, it requires that resampling of reference pictures be performed and adaptive resolution coding signaling be available for the coded picture portions.

[0081] In an embodiment, not all samples of a reconstructed picture are intended for output. The encoder can use a compatibility window to indicate a rectangular subportion of the picture intended for output. The compatibility window may be described or indicated, for example, by a left and right offset from a picture edge as defined by the picture size. Specific use cases for which the compatibility window may be relevant can be identified, including overscan, spatial assembly of views in a multiview system, or a 360 system, where the compatibility window may indicate one of several cube map planes to be output.

[0082] Because not all applications require the use of a compatibility window, and because compatibility window parameters may require a certain amount of bits in the bitstream and therefore may impair coding efficiency if not used, the presence of such parameters may be gated by a flag.

[0083] In an embodiment, the compatibility window size may be signaled in a picture parameter set (PPS). If the compatibility window size of a reference picture is different from that of the current picture, the compatibility window parameter, which may define the compatibility window size, may be used to calculate the resampling ratio. The decoder may need to know the compatibility window size of each picture to determine whether a resampling process is needed. If the resampling ratio is not equal to 1, the output picture size is not constant within the CVS, and special handling and post-processing of the output picture, such as upscaling / downscaling for display, may be used.

[0084] In an embodiment, a flag indicating whether the decoded / outputted pictures have the same size and the resampling ratio is equal to 1 in the CVS / bitstream may be signaled in a high-level parameter set such as a decoding parameter set (DPS), a video parameter set (VPS), or a sequence parameter set (SPS). The flag may be used for session negotiation or configuration of decoder and display settings for video streaming.

[0085] Referring to Figure 7, a flag constant_pic_size_flag (704) equal to 1 may indicate that the picture sizes of the pictures in the CVS are the same. A constant_pic_size_flag equal to 0 may indicate that the picture sizes of the pictures in the CVS may or may not be the same. If the value of constant_pic_size_flag is equal to 1, then the flag sps_conformance_window_flag (705) may be present in the SPS (701). A sps_conformance_window_flag equal to 1 may indicate that a conformance cropping window offset parameter follows in the appropriate position, e.g., next in the SPS. A sps_conformance_window_flag equal to 0 may indicate that a conformance cropping window offset parameter is not present.

[0086] In an embodiment, sps_conf_win_left_offset (706), sps_conf_win_right_offset (707), sps_conf_win_top_offset (708), and sps_conf_win_bottom_offset (709) may define a sample of the picture in the CVS output from the decoding process with respect to a rectangular area defined in picture coordinates for output.

[0087] In an embodiment, if the syntax elements sps_conf_win_left_offset, sps_conf_win_right_offset, sps_conf_win_top_offset, and sps_conf_win_bottom_offset are not present, then the values ​​of sps_conf_win_left_offset, sps_conf_win_right_offset, sps_conf_win_top_offset, and sps_conf_win_bottom_offset may be inferred to be equal to zero.

[0088] In an embodiment, referring to FIG. 8, pic_width_in_luma_samples (802) may define the width of each decoded picture referencing the PPS (801) in units of luma samples. In an embodiment, pic_width_in_luma_samples may not be equal to 0, may be an integer multiple of Max(8, MinCbSizeY), and may be less than or equal to pic_width_max_in_luma_samples. If not present, the value of pic_width_in_luma_samples may be inferred to be equal to pic_width_max_in_luma_samples. pic_height_in_luma_samples (803) may define the height of each decoded picture referencing the PPS in units of luma samples. pic_height_in_luma_samples may, in some cases, not be equal to 0, may be an integer multiple of Max(8, MinCbSizeY), and may be less than or equal to pic_height_max_in_luma_samples. If not present, the value of pic_height_in_luma_samples may be inferred to be equal to pic_height_max_in_luma_samples.

[0089] In an embodiment, and still referring to Figure 8, conformance_window_flag (804) equal to 1 may indicate that adaptive cropping window offset parameters follow at the appropriate position, e.g., next in PPS (801). Conformance_window_flag equal to 0 may indicate that adaptive cropping window offset parameters are not present. conf_win_left_offset (805), conf_win_right_offset (806), conf_win_top_offset (807), and conf_win_bottom_offset (808) may define the sample of the picture that references the PPS output from the decoding process, with respect to a rectangular area defined in picture coordinates for output.

[0090] In the same embodiment, if the syntax elements conf_win_left_offset, conf_win_right_offset, conf_win_top_offset, and conf_win_bottom_offset are not present, the values ​​of conf_win_left_offset, conf_win_right_offset, conf_win_top_offset, and conf_win_bottom_offset may be inferred to be equal to the values ​​of sps_conf_win_left_offset, sps_conf_win_right_offset, sps_conf_win_top_offset, and sps_conf_win_bottom_offset, respectively.

[0091] In an embodiment, the adaptive cropping window may include luminance samples with horizontal picture coordinates from SubWidthC*conf_win_left_offset to pic_width_in_luma_samples-(SubWidthC*conf_win_right_offset+1) and vertical picture coordinates from SubHeightC*conf_win_top_offset to pic_height_in_luma_samples-(SubHeightC*conf_win_bottom_offset+1), inclusive.

[0092] The value of SubWidthC*(conf_win_left_offset+conf_win_right_offset) may be less than pic_width_in_luma_samples, and the value of SubHeightC*(conf_win_top_offset+conf_win_bottom_offset) may be less than pic_height_in_luma_samples.

[0093] The variables PicOutputWidthL and PicOutputHeightL may be derived as shown in Equations 1 and 2 below. PicOutputWidthL=pic_width_in_luma_samples-SubWidthC*(conf_win_right_offset+conf_win_left_offset) (Formula 1) PicOutputHeightL=pic_height_in_luma_samples-SubHeightC*(conf_win_bottom_offset+conf_win_top_offset) (Formula 2)

[0094] In an embodiment, the fractional interpolation process with reference picture resampling may be handled as follows.

[0095] Inputs to this process may be a luminance position (xSb, ySb) defining the top-left sample of the current coding sub-block relative to the top-left luminance sample of the current picture, a variable sbWidth defining the width of the current coding sub-block, a variable sbHeight defining the height of the current coding sub-block, a motion vector offset mvOffset, a refined motion vector refMvLX, a selected reference picture sample array refPicLX, a half-sample interpolation filter index hpelIfIdx, a bidirectional optical flow flag bdofFlag, and a variable cIdx defining the color component index of the current block.

[0096] The output of this process may be a (sbWidth+brdExtSize)×(sbHeight+brdExtSize) array of predicted sample values, predSamplesLX.

[0097] The prediction block boundary extension size brdExtSize may be derived as shown in Equation 3 below. brdExtSize=(bdofFlag||(inter_affine_flag[xSb][ySb]&&sps_affine_prof_enabled_flag))?2:0 (Formula 3)

[0098] The variable fRefWidth may be set equal to PicOutputWidthL of the reference picture in luma samples. The variable fRefHeight may be set equal to PicOutputHeightL of the reference picture in luma samples. The motion vector mvLX may be set equal to (refMvLX-mvOffset).

[0099] If cIdx is equal to 0, the following may apply: The scaling factors and their fixed-point representations may be defined according to Equations 4 and 5 below: hori_scale_fp=((fRefWidth<<14)+(PicOutputWidthL>>1)) / PicOutputWidthL (Formula 4) vert_scale_fp=((fRefHeight<<14)+(PicOutputHeightL>>1)) / PicOutputHeightL (Formula 5) - (xIntL, yIntL) may be the luminance position given in full samples, and (xFracL, yFracL) may be the offset given in 1 / 16 samples. These variables may be used in this clause to define fractional sample positions within the reference sample array refPicLX. - Reference Sample Padding (xSbInt L , ySbInt L ) may be set equal to (xSb+(mvLX[0]>>4),ySb+(mvLX[1]>>4)). - each luminance sample position (x) in the predicted luminance sample array predSamplesLX L For each (x = 0..sbWidth-1+brdExtSize, y = 0..sbHeight-1+brdExtSize), the corresponding predicted luminance sample value predSamplesLX[x L ][y L ] is derived as follows: -(refxSb L , refySb L ) and (refx L ,refy L ) is the luminance position pointed to by the motion vector (refMvLX, refMvLX) given in 1 / 16 sample units. L ,refx L ,refySb L , and refy L may be derived as shown in Equation 6-9 below. refxSb L =((xSb<<4)+refMvLX[0])*hori_scale_fp (Equation 6) refx L =((Sign(refxSb)*((Abs(refxSb)+128)>>8) +x L *((hori_scale_fp+8)>>4))+32)>>6 (Formula 7) refySb L =((ySb<<4)+refMvLX[1])*vert_scale_fp (Equation 8) refyL=((Sign(refySb)*((Abs(refySb)+128)>>8)+yL* ((vert_scale_fp+8)>>4))+32)>>6 (Equation 9) -variable xInt L , yInt L , xFrac L and yFrac L may be derived as shown in Equation 10-13 below. xInt L =refx L >>4 (Formula 10) yInt L =refy L >>4 (Formula 11) xFrac L =refx L &15 (expression 12) yFrac L =refy L &15 (expression 13) -bdofFlag is equal to TRUE or (sps_affine_prof_enabled_flag is equal to TRUE and inter_affine_flag[xSb][ySb] is equal to TRUE) and one or more of the following conditions are true: L ][y L ] is used as input (xInt L +(xFrac L >>3)-1),yInt L +(yFrac L>>3)-1) may be derived by invoking the luma integer sample fetching process defined in the appropriate clause of the video coding specification, involving refPicLX. 1.x L is equal to 0. 2.x L is equal to sbWidth+1. 3.y L is equal to 0. 4.y L is equal to sbHeight+1. Otherwise, the predicted luma sample values ​​predSamplesLX[xL][yL] may be derived by invoking a luma sample 8-tap interpolation filtering process as defined in the appropriate clause of the video coding specification with (xIntL-(brdExtSize>0?1:0),yIntL-(brdExtSize>0?1:0)),(xFracL,yFracL),(xSbInt L ,ySbInt L ),refPicLX,hpelIfIdx,sbWidth,sbHeight, and (xSb,ySb) as input

[0100] Otherwise (if cIdx is not equal to 0), the following may apply: 1. Let (xIntC, yIntC) be the chrominance position given in full samples, and (xFracC, yFracC) be the offset given in 1 / 32 samples. These variables may be used in this clause to define a general fractional sample position within the reference sample array refPicLX. 2. The top left coordinate of the border block for the reference sample padding (xSbIntC, ySbIntC) is set equal to ((xSb / SubWidthC)+(mvLX[0]>>5), (ySb / SubHeightC)+(mvLX[1]>>5)). 3. For each chrominance sample position (xC=0..sbWidth-1, yC=0..sbHeight-1) in the predicted chrominance sample array predSamplesLX, the corresponding predicted chrominance sample value predSamplesLX[xC][yC] may be derived as follows: -(refxSb C ,refySb C ) and (refx C ,refy C ) is the chrominance position pointed to by the motion vector (mvLX[0], mvLX[1]) given in 1 / 32 sample units. C , refySb C、 refx C and refy C may be derived as shown in Equations 14-17 below. refxSb C =((xSb / SubWidthC<<5)+mvLX[0])*hori_scale_fp (Equation 14) refx C =((Sign(refxSb C) *((Abs(refxSb C )+256)>>9) +xC*((hori_scale_fp+8)>>4))+16)>>5 (Equation 15) refySb C =((ySb / SubHeightC<<5)+mvLX[1])*vert_scale_fp (Equation 16) refy C =((Sign(refySb C )*((Abs(refySb C )+256)>>9) +yC*((vert_scale_fp+8)>>4))+16)>>5 (Equation 17) -variable xInt C , yInt C , xFrac C and yFrac C may be derived as shown in Equations 18-21 below. xIntC =refx C >>5 (Formula 18) yInt C =refy C >>5 (Formula 19) xFrac C =refy C &31 (expression 20) yFrac C =refy C &31 (expression 21)

[0101] The predicted sample values ​​predSamplesLX[xC][yC] may be derived by invoking the process defined above with (xIntC, yIntC), (xFracC, yFracC), (xSbIntC, ySbIntC), sbWidth, sbHeight and refPicLX as inputs.

[0102] Figure 9 is a flowchart of an example process 900 for decoding an encoded video bitstream. In some implementations, one or more process blocks of Figure 9 may be performed by decoder 210. In some implementations, one or more process blocks of Figure 9 may be performed by another device or group of devices, such as encoder 203, that is separate from or includes decoder 210.

[0103] As shown in FIG. 9, process 900 may include obtaining a first flag indicating whether a constant picture size is used in a coded video sequence including the current picture (block 910).

[0104] As further shown in FIG. 9, process 900 may include determining from the first flag whether a fixed picture size is used (block 920).

[0105] As further shown in FIG. 9, process 900 may include decoding the current picture without performing reference picture resampling (block 930) based on a first flag indicating that a constant picture size is used (YES in block 920).

[0106] As further shown in FIG. 9, based on a first flag indicating that a constant picture size is not used (NO at block 920), process 900 may proceed to block 940, block 950, block 960, and block 970.

[0107] As further shown in FIG. 9, process 900 may include obtaining a second flag indicating whether the compatibility window size is signaled (block 940).

[0108] As further shown in FIG. 9, process 900 may include obtaining a compatibility window size based on a second flag indicating that the compatibility window size is signaled (block 950), determining a resampling ratio between the current picture and the reference picture based on the compatibility window size (block 960), and performing reference picture resampling on the current picture using the resampling ratio (block 970).

[0109] In one embodiment, the compatibility window size may be signaled as at least one offset distance from the boundary of the current picture.

[0110] In one embodiment, the first flag may be signaled in a sequence parameter set (SPS) and the second flag may be signaled in one of an SPS and a picture parameter set (PPS).

[0111] In one embodiment, a second flag may be signaled in the SPS and may indicate whether the SPS compatibility window parameter is signaled in the SPS.

[0112] In one embodiment, based on a second flag indicating that the SPS compatibility window parameter is signaled in the SPS, the compatibility window size may be obtained based on the SPS compatibility window parameter.

[0113] In one embodiment, based on the first flag indicating that the picture size is not constant, the process 900 may include obtaining a third flag indicating whether a PPS compatibility window parameter is signaled in the PPS.

[0114] In one embodiment, the compatibility window size may be obtained based on the SPS compatibility window parameters based on a second flag indicating that the SPS compatibility window parameters are signaled in the SPS and a third flag indicating that the PPS compatibility window parameters are not signaled in the PPS.

[0115] In one embodiment, the compatibility window size may be obtained based on the PPS compatibility window parameters based on a second flag indicating that the SPS compatibility window parameters are not signaled in the SPS and a third flag indicating that the PPS compatibility window parameters are signaled in the PPS.

[0116] Although Figure 9 illustrates example blocks of process 900, in some implementations, process 900 may include additional blocks, fewer blocks, different blocks than those shown in Figure 9, or blocks arranged differently than those shown in Figure 9. Additionally or alternatively, two or more of the blocks of process 900 may be performed in parallel.

[0117] Furthermore, the proposed methods may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium to perform one or more of the proposed methods.

[0118] The techniques described above may be implemented using computer-readable instructions and as computer software physically stored on one or more computer-readable media. For example, Figure 10 illustrates a computer system 1000 suitable for implementing certain embodiments of the disclosed subject matter.

[0119] Computer software can be coded using any suitable machine code or computer language, which may be subjected to mechanisms such as assembly, compilation, linking, etc. to create code containing instructions that can be executed by a computer central processing unit (CPU), graphics processing unit (GPU), etc. directly or via interpretation, microcode execution, etc.

[0120] The instructions may be executed by various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, and the like.

[0121] 10 with respect to computer system 1000 are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure. Neither component form should be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of computer system 1000.

[0122] Computer system 1000 may include certain human interface input devices. Such human interface input devices may respond to input by one or more human users via, for example, tactile input (e.g., keystrokes, swipes, data glove movements, etc.), audio input (e.g., voice, clapping), visual input (e.g., gestures), olfactory input (not shown). Human interface devices may also be used to capture certain media not necessarily directly associated with conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0123] The input human interface devices may include one or more of a keyboard 1001, a mouse 1002, a trackpad 1003, a touchscreen 1010 and associated graphics adapter 1050, a data glove, a joystick 1005, a microphone 1006, a scanner 1007, and a camera 1008 (only one of each is depicted).

[0124] Computer system 1000 may also include certain human interface output devices, such as tactile output, sound, light, and smell / taste to stimulate one or more of the human user's senses. Such human interface output devices can include haptic output devices (e.g., haptic feedback via a touchscreen 1010, data gloves, or joystick 1005, although there may also be haptic feedback devices that do not function as input devices), audio output devices (speakers 1009, headphones (not shown)), visual output devices (such as screens 1010 to include cathode ray tube (CRT) screens, liquid crystal display (LCD) screens, plasma screens, organic light emitting diode (OLED) screens, each with or without touchscreen input capability, each with or without haptic feedback capability - some of which may be capable of outputting more than two-dimensional visual output or three-dimensional output through means such as stereoscopic output; virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0125] The computer system 1000 may also include human-accessible storage devices and their associated media such as optical media 1021, including CD / DVD ROM / RW 1020 with media such as CDs / DVDs, thumb drives 1022, removable hard drives or solid state drives 1023, legacy magnetic media such as tape and floppy disks (not shown), and dedicated ROM / ASIC / PLD-based devices such as security dongles (not shown).

[0126] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter of this disclosure does not encompass transmission media, carrier waves, or other transitory signals.

[0127] The computer system 1000 may also include interfaces to one or more communications networks (1155). Networks may be, for example, wireless, wired, or optical. Networks may further be local, wide area, metropolitan, vehicular, and industrial, real-time, delay-tolerant, etc. Examples of networks include local area networks such as Ethernet, wireless LAN, etc.; cellular networks, including Global System for Mobile Communications (GSM), third generation (3G), fourth generation (4G), fifth generation (5G), and long-term evolution (LTE), etc.; television wired or wireless wide area digital networks, including cable TV, satellite TV, and terrestrial broadcast TV; vehicular and industrial networks, including CANBus, etc. Particular networks typically require an external network interface adapter (1154) that attaches to a particular general-purpose data port or peripheral bus (1149) (e.g., a universal serial bus (USB) port on computer system 1000; others are typically integrated into the core of computer system 1000 by attachment to the system bus, as described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). As an example, a network 1055 may be connected to peripheral bus 1049 using network interface 1054. Using any of these networks, computer system 1000 can communicate with other entities. Such communications can be unidirectional, receive only (e.g., broadcast TV), transmit only unidirectionally (e.g., a CANbus to a particular CANbus device), or bidirectional, for example, to other computer systems using local or wide-area digital networks. As previously discussed, particular protocols and protocol stacks can be used with each of these networks and network interfaces (1154).

[0128] The aforementioned human interface devices, human access storage devices, and network interfaces may be attached to core 1040 of computer system 1000 .

[0129] The core 1040 may include one or more central processing units (CPUs) 1041, graphics processing units (GPUs) 1042, dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) 1043, hardware accelerators for specific tasks 1044, etc. These devices may be connected via a system bus 1048, along with read-only memory (ROM) 1045, random access memory (RAM) 1046, internal mass storage devices such as internal non-user-accessible hard drives, solid-state drives (SSDs), etc. 1047. In some computer systems, the system bus 1048 may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be attached directly to the core's system bus 1048 or via a peripheral bus 1049. Peripheral bus architectures include Peripheral Component Interconnect (PCI), USB, etc.

[0130] The CPU 1041, GPU 1042, FPGA 1043, and accelerator 1044 may execute specific instructions that, in combination, may constitute the aforementioned computer code. That computer code may be stored in ROM 1045 or RAM 1046. Transient data may also be stored in RAM 1046, while permanent data may be stored, for example, in internal mass storage 1047. Rapid storage and retrieval from any of the memory devices may be enabled through the use of cache memory, which may be closely associated with one or more of the CPU 1041, GPU 1042, mass storage device 1047, ROM 1045, and RAM 1046, etc.

[0131] The computer-readable medium may bear computer code for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0132] By way of example and not limitation, the architecture, and specifically the computer system 1000 having the core 1040, can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be media associated with user-accessible mass storage devices, as previously introduced, as well as media associated with specific storage devices of the core 1040 that are non-transitory in nature, such as the core's internal mass storage device 1047 or ROM 1045. Software implementing various embodiments of the present disclosure can be stored in such devices and executed by the core 1040. The computer-readable media can include one or more memory devices or chips, according to particular needs. The software enables the core 1040, and specifically the processors therein (including a CPU, GPU, FPGA, etc.), to perform particular processes or portions of particular processes described herein, including defining data structures stored in RAM 1046 and modifying such data structures according to processes defined by the software. Additionally or alternatively, a computer system may provide functionality as a result of logic embodied in hardwired or otherwise circuitry (e.g., accelerator 1044) that can operate in place of or in conjunction with software to perform particular processes or portions of particular processes described herein. References to software may encompass logic, and vice versa, where appropriate. References to computer-readable media may encompass, where appropriate, circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry that embodies logic for execution, or both. The present disclosure encompasses any suitable combination of hardware and software.

[0133] While this disclosure has described several exemplary embodiments, there are alterations, permutations, and various substitute equivalents which fall within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise numerous systems and methods which, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within its spirit and scope. [Explanation of symbols]

[0134] 100 Communication Systems 110 First Terminal 120 Second Terminal 130 terminals 140 terminals 150 Network 201 Source 202 uncompressed video sample streams 203 Encoder 204 Video Bitstream 205 Streaming Server 207 Video Bitstream 208 Streaming Client 209 Video Bitstream 210 decoder 211 Outgoing Video Sample Stream 212 Display 213 Acquisition Subsystem 312 channels 310 Receiver 315 Buffer Memory 320 Parser 321 Symbol 351 Scaler / Descaler Unit 352 Intra Prediction Units 353 Motion Compensation Prediction Unit 355 Aggregator 356 Loop Filter 357 Reference Picture Buffer 358 Current Picture 430 Source Coder 432 Coating Engine 433 decoder 434 Reference Picture Memory 435 Predictor 440 Transmitter 443 Video Sequence 445 Entropy Coder 450 Controller 460 channels 501 Picture Header 502 ARC information 503 ARC (Warping Coordinates) 601 Tile Group Header 602 Syntax element dec_pic_size_idx 603 Adaptive Resolution 610 Sequence Parameter Set 612 parameter sets 613 sample output resolution 614 Syntax element reference_pic_size_present_flag 615 Reference Picture Dimensions 616 table display(num_dec_pic_size_in_luma_samples_minus 1) 617 table entries 1000 Computer Systems 1001 keyboard 1002 Mouse 1003 Trackpad 1005 Joystick 1006 Microphone 1007 Scanner 1008 Camera 1009 Speaker 1010 Touchscreen 1020 CD / DVD ROM / RW 1021 Optical media 1022 thumb drive 1023 removable hard drive or solid state drive 1040 cores 1041 CPU 1042 GPU 1043 Field Programmable Gate Area (FPGA) 1044 Hardware Accelerator 1045 read-only memory (ROM) 1046 Random Access Memory (RAM) 1047 Internal Mass Storage 1048 System Bus 1049 Peripheral Bus 1050 graphics adapter 1054 network interface 1149 Peripheral Bus 1154 External Network Interface Adapter 1155 Communication Network

Claims

1. 1. A method of decoding an encoded video bitstream using at least one processor, the method comprising: obtaining a first flag indicating whether a constant picture size is used in a coded video sequence including the current picture; decoding the current picture without performing reference picture resampling based on the first flag indicating that the constant picture size is used; obtaining a second flag indicating whether a compatibility window size is signaled based on the first flag indicating that the fixed picture size is not used; based on the second flag indicating that the compatibility window size is signaled; obtaining the compatibility window size; determining a resampling ratio between the current picture and a reference picture based on the compatibility window size; performing the reference picture resampling on the current picture using the resampling ratio; A method comprising:

2. The method of claim 1 , wherein the compatibility window size is signaled as at least one offset distance from a boundary of the current picture.

3. the first flag is signaled in a sequence parameter set (SPS); the second flag is signaled in one of the SPS and a picture parameter set (PPS); The method of claim 1.

4. The method of claim 3 , wherein the second flag is signaled in the SPS and indicates whether an SPS compatibility window parameter is signaled in the SPS.

5. The method of claim 4 , wherein the compatibility window size is obtained based on the SPS compatibility window parameter based on the second flag indicating that the SPS compatibility window parameter is signaled in the SPS.

6. 5. The method of claim 4, further comprising: obtaining a third flag indicating whether a PPS compatibility window parameter is signaled in the PPS based on the first flag indicating that the picture size is not constant.

7. 7. The method of claim 6, wherein the compatibility window size is obtained based on the SPS compatibility window parameter, based on the second flag indicating that the SPS compatibility window parameter is signaled in the SPS and the third flag indicating that the PPS compatibility window parameter is not signaled in the PPS.

8. 7. The method of claim 6, wherein the adaptability window size is obtained based on the PPS adaptability window parameter, based on the second flag indicating that the SPS adaptability window parameter is not signaled in the SPS and the third flag indicating that the PPS adaptability window parameter is signaled in the PPS.

9. 1. A device for decoding an encoded video bitstream, the device comprising: at least one memory configured to store program code; at least one processor configured to read the program code and to act as directed by the program code; Equipped with The program code first obtaining code configured to cause the at least one processor to obtain a first flag indicating whether a constant picture size is used in a coded video sequence that includes a current picture; decoding code configured to cause the at least one processor to decode the current picture without performing reference picture resampling based on the first flag indicating that the constant picture size is used; second obtaining code configured to cause the at least one processor to obtain a second flag indicating whether a compatibility window size is signaled based on the first flag indicating that the fixed picture size is not used; and executable code configured to cause the at least one processor to obtain the compatibility window size based on the second flag indicating that the compatibility window size is signaled, determine a resampling ratio between the current picture and a reference picture based on the compatibility window size, and perform reference picture resampling on the current picture using the resampling ratio; Devices containing:

10. The device of claim 9 , wherein the compatibility window size is signaled as at least one offset distance from a boundary of the current picture.

11. the first flag is signaled in a sequence parameter set (SPS); the second flag is signaled in one of the SPS and a picture parameter set (PPS); 10. The device of claim 9.

12. 12. The device of claim 11, wherein the second flag is signaled in the SPS and indicates whether an SPS compatibility window parameter is signaled in the SPS.

13. 13. The device of claim 12, wherein the compatibility window size is obtained based on the SPS compatibility window parameter based on the second flag indicating that the SPS compatibility window parameter is signaled in the SPS.

14. 13. The device of claim 12, wherein the program code further includes third obtaining code configured to cause the at least one processor to obtain a third flag indicating whether a PPS compatibility window parameter is signaled in the PPS based on the first flag indicating that the picture size is not constant.

15. 15. The device of claim 14, wherein the compatibility window size is obtained based on the SPS compatibility window parameter, based on the second flag indicating that the SPS compatibility window parameter is signaled in the SPS and the third flag indicating that the PPS compatibility window parameter is not signaled in the PPS.

16. 15. The device of claim 14, wherein the compatibility window size is obtained based on the SPS compatibility window parameter, based on the second flag indicating that the SPS compatibility window parameter is not signaled in the SPS and the third flag indicating that the PPS compatibility window parameter is signaled in the PPS.

17. 1. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a device for decoding an encoded video bitstream, cause the one or more processors to: obtaining a first flag indicating whether a constant picture size is used in a coded video sequence that includes the current picture; causing the current picture to be decoded without performing reference picture resampling based on the first flag indicating that the constant picture size is used; obtaining a second flag indicating whether a compatibility window size is signaled based on the first flag indicating that the fixed picture size is not used; obtaining the compatibility window size based on the second flag indicating that the compatibility window size is signaled; determining a resampling ratio between the current picture and a reference picture based on the compatibility window size; performing reference picture resampling on the current picture using the resampling ratio; A non-transitory computer-readable medium containing one or more instructions.

18. the first flag is signaled in a sequence parameter set (SPS); the second flag is signaled in one of the SPS and a picture parameter set (PPS); 18. The device of claim 17.

19. 20. The device of claim 18, wherein the second flag is signaled in the SPS and indicates whether an SPS compatibility window parameter is signaled in the SPS.

20. 20. The device of claim 19, wherein the one or more instructions further cause the at least one processor to obtain, based on the first flag indicating that the picture size is not constant, a third flag indicating whether a PPS compatibility window parameter is signaled in the PPS.

Images