Method for reference picture resampling with offset in a video bitstream - Patents.com

Abstract

To provide a method, a device and a computer-readable medium for decoding an encoded video bitstream using at least one processor.SOLUTION: A method includes obtaining a flag indicating that a conformance window is not used for reference picture resampling; if the flag indicates that the conformance window is not used for the reference picture resampling, then determining whether a resampling picture size is signaled; if the resampling picture size is signaled, then determining a resampling ratio on the basis of the resampling picture size; if the resampling picture size is not signaled, then determining the resampling ratio on the basis of an output picture size; and applying the reference picture resampling to a current picture using the resampling ratio.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] [Related Applications] This application claims priority to U.S. Provisional Patent Application No. 62 / 905,319, filed September 24, 2019, and U.S. Patent Application No. 17 / 022,727, filed September 16, 2020, which are incorporated by reference in their entireties.

[0002] [Technical field] The disclosed subject matter relates to video coding and decoding, and more particularly, to signaling reference picture resampling with resampling picture size indication. [Background technology]

[0003] Video coding and decoding using inter-picture prediction with motion compensation is known. Uncompressed digital video can be composed of a sequence of pictures, each having spatial dimensions of, for example, 1920x1080 luminance samples and associated chrominance samples. The sequence of pictures may have a fixed or variable picture rate (also known informally as frame rate), for example, 60 pictures per second or 60 Hz. Uncompressed video has significant bitrate requirements. For example, 1080p60 4:2:0 video (1920x1080 luminance sample resolution at 60 Hz frame rate) with 8 bits per sample requires a bandwidth approaching 1.5 Gbit / s. One hour of such video requires more than 600 Gbytes of storage space.

[0004] One goal of video coding and decoding can be the reduction of redundancy in the input video signal through compression. Compression can help reduce the aforementioned bandwidth or storage space requirements, in some cases by more than two orders of magnitude. Both lossy and lossless compression, and combinations thereof, can be utilized. Lossless compression refers to techniques where an exact copy of the original signal can be reconstructed from the compressed original signal. With lossy compression, the reconstructed signal is not identical to the original signal, but the distortion between the original signal and the reconstructed signal is small enough to produce a reconstructed signal that is useful for the intended application. In the case of video, lossy compression is widely used. The amount of tolerable distortion depends on the application, for example, a user of a particular consumer streaming application can tolerate higher distortion than a user of a television posting application. The achievable compression ratio can reflect that the higher the tolerable / tolerable distortion, the higher the compression ratio that can result.

[0005] Video encoders and decoders can use techniques from several broad categories, including, for example, motion compensation, transform, quantization, and entropy coding, some of which are introduced below.

[0006] Historically, video encoders and decoders tend to work with a given picture size that is often defined and remains constant for a coded video sequence (CVS), Group of Pictures (GOP), or similar multi-picture time frame. For example, in MPEG-2, system designs are known to change the horizontal resolution (and therefore the picture size) depending on factors such as scene activity, not only for I-pictures, but also typically for GOPs. Resampling of reference pictures for use of different resolutions in a CVS is known, for example from ITU-T Rec. H.263 Annex P. However, here the picture size does not change, so only the reference pictures are resampled, resulting in the possibility that only a portion of the picture canvas is used (in the case of downsampling) or only a portion of the scene is captured (in the case of upsampling). Furthermore, H.263 Annex Q allows resampling of individual macroblocks by multiples of two (in each dimension), either upwards or downwards. Again, the picture size remains the same. In H.263 the size of the macroblocks is fixed and therefore does not need to be signaled.

[0007] Picture size changes in predicted pictures have become more mainstream in recent video coding. For example, VP9 allows reference picture resampling and changing the resolution of the whole picture. Similarly, certain proposals targeting VVC (including, for example, Hendry, et. al, “On adaptive resolution change (ARC) for VVC”, Joint Video Team document JVET-M0135-v1, Jan9-19, 2019, which is incorporated herein by reference in its entirety) allow resampling of the whole reference picture to different—higher or lower—resolutions. In the document, different candidate resolutions are proposed to be coded in the sequence parameter set and referenced by per-picture syntax elements in the picture parameter set. Summary of the Invention

[0008] In an embodiment, there is provided a method of decoding an encoded video bitstream using at least one processor, the method comprising: obtaining a flag indicating that a fitting window is not used for reference picture resampling; determining whether a resampling picture size is signaled based on the flag indicating that the adaptive window is not used for the reference picture resampling; determining a resampling ratio based on the resampling picture size based on the determination that the resampling picture size is signaled; determining the resampling ratio based on an output picture size based on a determination that the resampling picture size is not signaled; performing the reference picture resampling on a current picture using the resampling ratio.

[0009] In an embodiment, there is provided an apparatus for decoding an encoded video bitstream, the apparatus comprising: at least one memory configured to store program code; at least one processor configured to read the program code and to act as instructed by the program code; the program code comprising: obtaining code configured to cause the at least one processor to obtain a flag indicating that an adaptive window is not used for reference picture resampling; first decision code configured to cause the at least one processor to determine whether a resampled picture size is signaled based on the flag indicating that the adaptive window is not used for the reference picture resampling; second decision code configured to cause the at least one processor to determine a resampling ratio based on the resampled picture size based on a determination that the resampled picture size is signaled; third decision code configured to cause the at least one processor to determine the resampling ratio based on an output picture size based on a determination that the resampling picture size is not signaled; and executable code configured to cause the at least one processor to perform the reference picture resampling on a current picture using the resampling ratio.

[0010] In an embodiment, a non-transitory computer readable medium is provided having instructions stored thereon that, when executed by one or more processors of an apparatus for decoding an encoded video bitstream, cause the one or more processors to: Obtaining a flag indicating that a fitting window is not used for reference picture resampling; determining whether a resampling picture size is signaled based on the flag indicating that the adaptive window is not used for the reference picture resampling; determining a resampling ratio based on the resampling picture size based on the determination that the resampling picture size is signaled; determining the resampling ratio based on an output picture size based on a determination that the resampling picture size is not signaled; The resampling ratio is used to perform the reference picture resampling on the current picture. [Brief description of the drawings]

[0011] Further features, characteristics, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings.

[0012] [Figure 1] FIG. 1 is a schematic diagram of a simplified block diagram of a communication system, according to an embodiment.

[0013] [Diagram 2] FIG. 1 is a schematic diagram of a simplified block diagram of a communication system, according to an embodiment.

[0014] [Diagram 3] FIG. 2 is a schematic diagram of a simplified block diagram of a decoder according to an embodiment;

[0015] [Figure 4] FIG. 2 is a schematic diagram of a simplified block diagram of an encoder, according to an embodiment;

[0016] [Diagram 5] FIG. 13 is a schematic diagram of options for signaling ARC parameters according to an embodiment.

[0017] [Figure 6A] FIG. 2 is a schematic diagram of an example of a syntax table according to an embodiment. [Figure 6B] FIG. 2 is a schematic diagram of an example of a syntax table according to an embodiment.

[0018] [Figure 7] FIG. 2 is a schematic diagram of picture size and adaptation window signaling within a PPS according to an embodiment;

[0019] [Figure 8] FIG. 2 is a schematic diagram of picture size and adaptation window signaling within a PPS according to an embodiment;

[0020] [Figure 9] 1 is a flowchart of an exemplary process for decoding an encoded video bitstream, according to an embodiment.

[0021] [Figure 10] 1 is a schematic diagram of a computer system, according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] FIG. 1 shows a simplified block diagram of a communication system (100) according to an embodiment of the present invention. The system (100) may include at least two terminals (110-120) interconnected via a network (150). In a unidirectional transmission of data, a first terminal (110) may code video data locally for transmission to another terminal (120) via the network (150). The second terminal (120) may receive the coded video data of the other terminal from the network (150), decode the coded data, and display the recovered video data. Unidirectional data transmission may be common in media serving applications, etc.

[0023] 1 illustrates a second pair of terminals (130, 140) adapted to support bidirectional transmission of coded video, such as may occur during a video conference. In the bidirectional transmission of data, each terminal (130, 140) may code locally captured video data for transmission to the other terminal over the network (150). Each terminal 130, 140 may also receive coded video data transmitted by the other terminal, decode the coded data, and display the recovered video data on a local display device.

[0024] In FIG. 1, the terminal devices (110-140) may be depicted as servers, personal computers, and smartphones, although the principles of the present disclosure are not limited thereto. The embodiments of the present disclosure may be applied to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. The network (150) represents any number of networks that carry coded video data between the terminals (110-140), including, for example, wired and / or wireless communication networks. The communication network (150) may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include electronic communication networks, local area networks, wide area networks, and / or the Internet. For purposes of the present discussion, the architecture and topology of the network (150) may not be important to the operation of the present disclosure, unless otherwise noted below.

[0025] 2 shows an arrangement of a video encoder and a video decoder in a streaming environment as an example of an application of the disclosed subject matter, which is equally applicable to, for example, video conferencing, digital TV, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc., other video-enabled applications, etc.

[0026] The streaming system may include a video source (201), e.g., a capture subsystem (213), which may include a digital camera, that generates an uncompressed video sample stream (202). The sample stream (202) may be processed by an encoder (203), shown in bold to emphasize its high data volume when compared to an encoded video bitstream, coupled to the camera (201). The encoder (203) may include hardware, software, or a combination thereof, and may enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video bitstream (204), shown in thin to emphasize its low data volume when compared to the sample stream, may be stored on a streaming server (205) for future use. One or more streaming clients (206, 208) may access the streaming server (205) to retrieve copies (207, 209) of the encoded video bitstream (204). The client (206) may include a video decoder (210). A video decoder (310) decodes an incoming copy of the encoded bitstream (207) and generates an output video sample stream (211) that can be rendered on a display (212) or other rendering device (not shown). In some streaming systems, the video bitstreams (204, 207, 209) can be encoded according to a particular video coding / compression standard. Examples of these standards include ITU-T Recommendation H.265. The video coding standard under development is known informally as Versatile Video Coding (VVC). The disclosed subject matter may be used in the context of VVC.

[0027] FIG. 3 may be a functional block diagram of a video decoder (210) according to an embodiment of the disclosure.

[0028] The receiver (310) may receive one or more coded video sequences to be decoded by the video decoder (210), one coded video sequence at a time in the same or another embodiment, where the decoding of each coded video sequence is independent of the other coded video sequences. The coded video sequences may be received from a channel (312), which may be a hardware / software link to a storage device that stores the coded video data. The receiver (310) may receive the coded video data along with other data, e.g., coded audio data and / or auxiliary data streams, which may be forwarded to a respective using entity (not shown). The receiver (310) may separate the coded video sequences from the other data. To eliminate network jitter, a buffer memory (315) may be connected between the receiver (310) and the entropy decoder / parser (320) (hereinafter the "parser"). When the receiver (310) is receiving data controllably from a storage / forwarding device of sufficient bandwidth or from an isosynchronous network, the buffer (315) may not be needed or may be small. For use in a best effort packet network such as the Internet, buffer (315) may be necessary and may be relatively large, and may advantageously be adaptively sized.

[0029] The video decoder (210) may include a parser (320) to reconstruct symbols (321) from the entropy coded video sequence. These symbol categories include information used to manage the operation of the decoder (210) and information for controlling a rendering device, such as a display (212), which may in some cases not be an integral part of the decoder but may be connected to the decoder as shown in FIG. 3. The control information for the rendering device may be in the form of a Supplementary Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (320) may parse / entropy decode the received encoded video sequence. The coding of the coded video sequence may follow a video coding technique or standard and may follow principles well known to those skilled in the art, including variable length coding, Huffman coding, arithmetic coding with or without context dependency, etc. The parser (320) may extract a set of subgroup parameters from the coded video sequence for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the subgroup. Subgroups may include Groups of Pictures (GOPs), pictures, subpictures, tiles, slices, bricks, macroblocks, coding tree units (CTUs), coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. A tile may refer to a rectangular region of a CU / CTU within a particular tile column and row in a picture. A brick may refer to a rectangular region of a CU / CTU row within a particular tile. A slice may refer to one or more bricks of a picture contained in a NAL unit. A subpicture may refer to a rectangular region of one or more slices in a picture.The entropy decoder / parser may also extract information such as transform coefficients, quantization parameter values, motion vectors, etc. from the coded video sequence.

[0030] The parser (320) may perform entropy decoding / parsing operations on the video sequence received from the buffer (315) to generate symbols (321).

[0031] The reconstruction of the symbols (321) may include a number of different units, depending on the type of coded video picture or portion thereof (e.g., inter and intra pictures, inter and intra blocks) and other factors. Which units are included and how can be controlled by subgroup control information parsed from the coded video sequence by the parser (320). The flow of such subgroup control information between the parser 320 and the following units is not shown for clarity.

[0032] Beyond the functional blocks already mentioned, the decoder (210) may be conceptually subdivided into a number of functional units, as described below. In an actual implementation operating under commercial constraints, many of these units may interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, the following conceptual subdivision into functional units is adequate.

[0033] The first unit is a scalar / inverse transform unit 351. The scalar / inverse transform unit (351) receives quantized transform coefficients and control information, including which transform should be used, block size, quantization coefficients, quantization scaling matrices, etc., as symbols (321) from the parser (320). It can output blocks containing sample values ​​that can be input to an aggregator (355).

[0034] In some examples, the output samples of the scaler / inverse transform unit (351) may belong to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture but can use prediction information from a previously reconstructed part of the current picture. Such prediction information may be provided by an intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) generates a block of the same size and shape as the block being reconstructed using surrounding already reconstructed information fetched from the current (partially reconstructed) picture (358). The aggregator (355) adds, in some cases, on a sample-by-sample basis, the prediction information generated by the intra-prediction unit (352) to the output sample information provided by the scaler / inverse transform unit (351).

[0035] In other cases, the output samples of the scaler / inverse transform unit (351) may relate to an inter-coded, possibly motion-compensated, block. In such cases, the motion-compensated prediction unit (353) may access the reference picture memory (357) to fetch samples used for prediction. After motion-compensating the fetched samples according to the symbols (321) related to the block, these samples may be added by the aggregator (355) to the output of the scaler / inverse transform unit to generate output sample information (in this case referred to as residual samples or residual signals). The addresses in the reference picture memory from which the motion-compensated prediction unit fetches prediction samples may be controlled by the motion-compensated prediction unit's available motion vectors, e.g., in the form of symbols (321) that may have X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​fetched from the reference picture memory when sub-sample accurate motion vectors are in use, motion vector prediction mechanisms, etc.

[0036] The output samples of the aggregator (355) may be subjected to various loop filtering techniques in a loop filter unit (356). The video compression techniques are controlled by parameters contained in the coded video bitstream and made available to the loop filter unit (356) as symbols (321) from the parser (320), but may also include in-loop filter techniques that are responsive to meta-information obtained during decoding of previous portions (in decoding order) of the coded picture or coded video sequence, and that may also be responsive to previously reconstructed loop filtered sample values.

[0037] The output of the loop filter unit (356) may be a sample stream that can be output to a render device (212) and stored in a reference picture memory for use in future inter-picture prediction.

[0038] Once a particular coding picture is fully reconstructed, it can be used as a reference picture for future prediction. Once a coding picture is fully reconstructed and the coding picture is identified as a reference picture (e.g., by the parser (320)), the current reference picture (358) can become part of the reference picture buffer (357), and fresh current picture memory can be reallocated before beginning reconstruction of a subsequent coding picture.

[0039] The video decoder (210) may perform decoding operations according to a given video compression technique, which may be defined in a standard such as ITU-T Rec. H.265. The coding video sequence may comply with the syntax specified by the video compression technique or standard in use, in the sense that the coding video sequence complies with the syntax of the video compression technique or standard, as specified in the video compression technique or standard, and in particular in a profile document therein. Also, a requirement for compliance may be that the complexity of the coding video sequence is within the limits defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples / second), maximum reference picture size, etc. The limits set by the level may in some cases be further limited through a Hypothetical Reference Decoder (HRD) specification and metadata for HDR buffer management signaled in the coding video sequence.

[0040] In an embodiment, the receiver (310) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the coded video sequence. The additional data may be used by the video decoder 210 to correctly decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or SNR enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0041] FIG. 4 may be a functional block diagram of a video encoder (203) according to one embodiment of the disclosure.

[0042] The encoder (203) may receive video samples from a video source (201) (not part of the encoder) that may capture video images to be coded by the encoder (203).

[0043] The video source (201) may provide a source video sequence to be coded by the encoder (203) in the form of a digital video sample stream of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media presentation system, the video source (201) may be a storage device that stores previously prepared video. In a video conferencing system, the video source (203) may be a camera that captures local image information as a video sequence. The video data may be provided as a number of individual pictures that, when viewed in succession, give the appearance of motion. The pictures themselves may be organized as a spatial array of pixels. Each pixel may contain one or more samples, depending on the sampling structure, color space, etc., being used. Those skilled in the art will readily appreciate the relationship between pixels and samples. The following description focuses on samples.

[0044] According to an embodiment, the encoder (203) may code and compress pictures of a source video sequence into a coded video sequence (443) in real time or under any other time constraint required by the application. Enforcing an appropriate coding rate is one function of the controller (450). The controller controls and is functionally coupled to other functional units as described below. Coupling is not shown for clarity. Parameters set by the controller may include rate control related parameters (picture skip, quantizer, lambda value for rate distortion optimization techniques, ...), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. Those skilled in the art will readily identify other functions of the controller 450 as they may relate to a video encoder (203) optimized for a particular system design.

[0045] Some video encoders operate in what those skilled in the art would immediately recognize as a "coding loop." As a very simplified explanation, the coding loop can include an encoder (430) (hereafter "source coder") (which generates symbols based on an input picture to be coded and reference pictures) and a coding portion of a (local) decoder (433) that is embedded within the encoder (203) and reconstructs the symbols to generate sample data that a (remote) decoder can generate (when any compression between the symbols and the coding video bitstream is lossless among the video compression techniques considered in the disclosed subject matter). The reconstructed sample stream is input to a reference picture memory 434. When the decoding of the symbol stream results in bit-exact results independent of the decoder location (local or remote), the contents of the reference picture buffer are also bit-exact between the local and remote encoders. In other words, the predictive portion of the encoder "sees" exactly the same sample values ​​as the decoder "sees" when using prediction during decoding as reference picture samples. This basic principle of reference picture synchrony (and the resulting drift if synchrony cannot be maintained, for example due to channel errors) is well known to those skilled in the art.

[0046] The operation of the "local" decoder (433) may be the same as that of the "remote" decoder (210), detailed above in connection with Figure 3. Referring also briefly to Figure 4, however, because symbols are available and the encoding / decoding of symbols into a coded video sequence by the entropy coder (445) and parser (320) may be lossless, the entropy decoding portion of the decoder (210), including the channel (312), receiver (310), buffer (315), and parser (320), may not be fully implemented in the local decoder (433).

[0047] An observation to be made at this point is that any decoder techniques, other than parsing / entropy decoding, present in the decoder must also be present in substantially the same functional form as in the corresponding encoder. For this reason, the subject matter of the disclosure focuses on the decoder operation. A description of the encoder techniques can be omitted, as they are the inverse of the decoder techniques, which are described generically. Only in certain areas are more detailed descriptions necessary and are provided below.

[0048] In operation, in some examples, the source coder (430) may perform motion-compensated predictive coding, which predictively codes an input frame with reference to one or more previously coded frames from the video sequence designated as "reference frames." In this method, the coding engine (432) codes differences between pixel blocks of the input frame and pixel blocks of reference frames that may be selected as prediction references for the input frame.

[0049] The local video decoder (433) may decode the coded video data of frames, which may be designated as reference frames, based on symbols generated by the source coder (430). The operation of the coding engine (432) may advantageously be lossy. When the coded video data may be decoded in a video decoder (not shown in FIG. 4), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (433) may replicate the decoding process that may be performed by the video decoder on the reference frames, resulting in reconstructed reference frames to be stored in the reference picture cache (434). In this way, the encoder (203) may locally store copies of reconstructed reference frames that have common content with the reconstructed reference frames that would be obtained by the far-end video decoder (in the absence of transmission errors).

[0050] The predictor (435) may perform a prediction search for the coding engine (432). That is, for a new frame to be coded, the predictor (435) may search the reference picture memory (434) for sample data (such as candidate reference pixel blocks) or specific metadata such as reference picture motion vectors, block shapes, etc. that may serve as suitable prediction references for the new picture. The predictor (435) may operate on a sample block-pixel block basis to find a suitable prediction reference. In some examples, an input picture may have prediction references derived from multiple reference pictures stored in the reference picture memory (434), as determined by the search results obtained by the predictor (435).

[0051] The controller (450) may manage the coding operations of the video coder (430), including, for example, setting parameters and subgroup parameters used for encoding the video data.

[0052] The output of all the aforementioned functional units may undergo entropy coding in an entropy coder (445), which converts the symbols produced by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques well known to those skilled in the art, such as Huffman coding, variable length coding, arithmetic coding, etc.

[0053] The transmitter (440) may buffer the coded video sequence generated by the entropy coder (445) to prepare it for transmission over a communication channel (460), which may be a hardware / software link to a storage device that may store the coded video data. The transmitter (440) may merge the coded video data from the video coder (430) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0054] A control unit (450) may manage the operation of the encoder (203). During coding, the control unit (450) may assign a particular coding picture type to each coding picture, which may affect the coding technique that may be applied to the respective picture. For example, pictures may often be assigned as one of the following picture types:

[0055] An intra picture (I-picture) may be a picture that can be coded and decoded without using any other frame in a sequence as a source of prediction. Some video codecs allow different kinds of intra pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art will recognize the variations of I-pictures and their respective applications and characteristics.

[0056] A predictive picture (P-picture) may be a picture that can be coded and decoded using intra- or inter-prediction, in most cases using one motion vector and reference index to predict the sample values ​​of each block.

[0057] A bidirectionally predicted picture (B-picture) may be a picture that can be coded and decoded using intra- or inter-prediction, using up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, a multi-predictive picture can use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0058] A source picture may commonly be spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples, respectively) and coded block by block. Blocks may be predictively coded with reference to other (already coded) blocks determined by the coding assignment applied to the respective picture of the block. For example, blocks of I pictures may be non-predictively coded, or they may be predictively coded with reference to already coded blocks of the same picture (spatial or intra prediction). Pixel blocks of P pictures may be predictively coded via spatial prediction or via temporal prediction with reference to one previously coded reference picture. Blocks of B pictures may be non-predictively coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference pictures.

[0059] The video coder (203) may perform coding operations according to a given video coding technique or standard, such as ITU-T Rec. H.265. In doing so, the video coder (203) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. The coding video data may therefore conform to a syntax specified by the video coding technique or standard being used.

[0060] In one embodiment, the transmitter (440) may transmit additional data along with the encoded video. The video coder (430) may include such data as part of the coded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other types of redundant data such as redundant pictures and slices, Supplementary Enhancement Information (SEI) messages, Visual Usability Information (VUI) parameter set fragments, etc.

[0061] Recently, compressed domain aggregation or extraction of multiple semantically independent picture parts into a single video picture has attracted attention. In particular, for example, in the context of 360 coding or certain surveillance applications, multiple semantically independent source pictures (e.g., six cubic surfaces of a cubic projected 360-degree scene, or individual camera inputs in the case of a multi-camera surveillance setup) may require separate adaptive resolution settings to accommodate different scene-specific activities at a given time. In other words, an encoder may choose to use different resampling factors for different semantically independent pictures that generate the entire 360-degree or surveillance scene at a given time. When combined into a single picture, this requires that reference picture resampling is performed and that adaptive resolution coding signaling is available for parts of the coding picture.

[0062] The following introduces some terminology that will be referenced in the remainder of this description.

[0063] A sub-picture may represent, in some cases, a rectangular configuration of samples, blocks, macroblocks, coding units, or similar entities that may be semantically grouped and coded independently at a modified resolution. One or more sub-pictures may form a picture. One or more coding sub-pictures may form a coding picture. One or more sub-pictures may be assembled into a picture, and one or more sub-pictures may be extracted from a picture. In certain circumstances, one or more coding sub-pictures may be assembled into a coding picture in the compressed domain without conversion to the sample level, and in the same or other cases, one or more coding sub-pictures may be extracted from a coding picture in the compressed domain.

[0064] Reference Picture Resampling (RPR) or Adaptive Resolution Change (ARC) may refer to a mechanism that allows changing the resolution of pictures or sub-pictures in a coded video sequence, e.g., by reference picture resampling. RPR / ARC parameters hereinafter refer to control information required to perform adaptive resolution change. This may include, e.g., filter parameters, scaling factors, resolution of output and / or reference pictures, various control flags, etc.

[0065] In an embodiment, coding and decoding may be performed on a single semantically independent coded video picture. Before describing the meaning of coding / decoding multiple sub-pictures with independent RPR / ARC parameters and its implied additional complexity, the signaling of the RPR / ARC parameters should be explained.

[0066] 5A-5E, several embodiments for signaling RPR / ARC parameters are shown. As noted with each of the embodiments, they may have certain advantages and certain disadvantages in terms of coding efficiency, complexity, and architecture. A video coding standard or technology may select one or more of these embodiments, or options known from the related art, for signaling RPR / ARC parameters. The embodiments may not be mutually exclusive or may be interchanged based on application needs, technically related standards, or encoder preferences.

[0067] The RPR / ARC parameter classes may include:

[0068] - Separate or combined up / downsample factors in the X and Y dimensions.

[0069] - An up / down sample factor, which indicates a constant speed zoom in / out for a given number of pictures with the addition of a time dimension.

[0070] Either of the two above may involve the coding of one or more possibly short syntax elements which may point to a table containing the factors.

[0071] Resolution in samples, blocks, macroblocks, coding units (CUs), or any other suitable unit of granularity, of input pictures, output pictures, reference pictures, coding pictures, combined or separate, in the X or Y dimension. If there is more than one resolution (e.g. one for the input picture and one for the reference picture), in certain cases one set of values ​​may be inferred from another set of values. This can be controlled, for example, by the use of flags. See below for more detailed examples.

[0072] "Warping" coordinates include those used in H.263 Annex P, again with appropriate granularity as described above. H.263 Annex P defines one efficient way to code such warping coordinates, although other, potentially more efficient ways may be devised. For example, the variable-length reversible Huffman-type coding of Annex P warping coordinates may be replaced by an appropriate-length binary coding, where the length of the binary codewords may, for example, be derived from the maximum picture size, possibly multiplied by a particular coefficient and offset by a particular value, thus allowing "warping" outside the boundaries of the maximum picture size.

[0073] -Up- or Down-sampling Filter Parameters In embodiments, there may be only a single filter for up- and / or down-sampling. However, in embodiments, it may be desirable to allow more flexibility in filter design, which may require signaling of filter parameters. Such parameters may be selected through an index in a list of possible filter designs. The filter may be fully specified (e.g., through a list of filter coefficients, using an appropriate entropy coding technique), the filter may be selected implicitly through the up / down sample ratio, which is signaled according to any of the mechanisms described above, etc.

[0074] In the following, we assume the coding of a finite set of up / downsample factors (the same factors to be used in both X and Y dimensions) indicated through a codeword. The codeword may be a variable length codeword using common Ext-Golomb codes for certain syntax elements in video coding specifications such as H.264 and H.265. One suitable mapping of values ​​to up / downsample factors may, for example, follow Table 1: Table 1 [Table 1]

[0075] Many similar mappings can be devised according to the needs of the application and the capabilities of the up- and down-scaling mechanisms available in the video compression technology or standard. The table can be extended to more values. The values ​​may be represented by entropy coding mechanisms other than the Ext-Golomb code, for example using binary coding. It may have certain advantages when the resampling factor is targeted outside the video processing engine (mainly the encoder and decoder) itself, for example by MANE. It should be noted that in situations where no change in resolution is required, the Ext-Golomb code can be chosen to be short, only a single bit in the above table. It may have coding efficiency advantages over using binary codes in the most general case.

[0076] The number of entries in the table, along with their meanings, may be fully or partially configurable. For example, a basic outline of the table may be conveyed in a "high" parameter set, such as a sequence or decoder parameter set. In an embodiment, one or more such tables may be defined in a video coding technology or standard and may be selected, for example, through a decoder or sequence parameter set.

[0077] The following describes how the upsampling / downsampling factors (ARC information) coded as described above are included in a video coding technique or standard syntax. Similar considerations can be applied to one or several codeword controlled up / downsampling filters. See below for a discussion of when a relatively large amount of data is required for a filter or other data structure.

[0078] As shown in Figure 5, H.263 Annex P includes ARC information (502) in the form of four warping coordinates in the picture header (501), specifically in the H.263 PLUSPTYPE (503) header extension. This may be a wise design choice when (a) there is a picture header available, and (b) frequent changes of ARC information are expected. However, the overhead when using H.263-type signaling can be very large, and the picture header may be of a transient nature, so the scaling coefficients may not fall between picture boundaries.

[0079] In the same or another embodiment, the signaling of ARC parameters is described in detail below, as outlined in Figures 6A-6B. Figures 6A-6B show a syntax diagram of a type expressed using a notation that roughly follows C-style programming as used, for example, in video coding standards since at least 1993. Bolded lines indicate syntax elements that appear in the bitstream. Non-bolded lines may indicate control flow or variable setting.

[0080] As shown in Fig. 6A, the tile group header (601) as an exemplary syntax structure of a header applicable to a (possibly rectangular) picture portion may conditionally contain a variable-length Ext-Golomb coded syntax element dec_pic_size_idx (602) (shown in bold). The presence of this syntax element in the tile group header can be controlled in the use of the value of the adaptive resolution (603), a flag not shown here in bold. This means that the flag is present in the bitstream at the point where it occurs in the syntax diagram. Whether adaptive resolution is used for this picture or portion can be signaled in a high-level syntax structure in or outside the bitstream. In the shown example, it is signaled in the sequence parameter set outlined below.

[0081] Referring to Figure 6B, an excerpt of a sequence parameter set (610) is also shown. The first syntax element shown is adaptive_pic_resolution_change_flag (611). When true, the flag can indicate the use of adaptive resolution, which may require specific control information. In the example, such control information is conditionally present based on the value of the flag based on an if() statement in the parameter set (612) and the tile group header (601).

[0082] When adaptive resolution is used, the output resolution is coded in units of samples, in this example (613). Reference 613 represents both output_pic_width_in_luma_samples and output_pic_height_in_luma_samples, which together may define the resolution of the output picture. In other cases, a video coding technology or standard may define specific limitations on any value. For example, the level definition may limit the number of total output samples, which may be the product of the values ​​of those two syntax elements. Also, a specific video coding technology or standard, or an external technology or standard, such as a system standard, may limit the numbering range (e.g., one or both dimensions must be divisible by a number that is a power of two), or the aspect ratio (e.g., width and height must have a relationship such as 4:3 or 16:9). Such limitations may be introduced to facilitate hardware implementation or for other reasons, and are well known in the art.

[0083] In certain applications, an encoder may be encouraged to instruct the decoder to use a particular reference picture size rather than implicitly assuming that size is the output picture size. In this example, the syntax element reference_pic_size_present_flag (614) controls the conditional presence of the reference picture dimensions (615) (again, the reference numbers represent both width and height).

[0084] Finally, a table of possible decoded pictures with width and height is shown. Such a table can be represented, for example, by a table instruction (num_dec_pic_size_in_luma_samples_minus1) (616). "minus1" may represent an interpretation of the value of the syntax element. For example, if the coded value is 0 (zero), there is one table entry. If the value is 5, there are six table entries. For each "row" in the table, the width and height of the decoded picture are included in the syntax (617).

[0085] Existing table entries (617) can be indexed using the syntax element dec_pic_size_idx (602) in the tile group header, allowing different decoding sizes, effectively zoom factors, per tile group.

[0086] Certain video coding techniques or standards, e.g., VP9, ​​support spatial scalability by implementing a specific form of reference picture resampling (signaled in a completely different way than the disclosed subject matter) in conjunction with temporal scalability to enable spatial scalability. In particular, certain reference pictures may be upsampled to a higher resolution using ARC-type techniques to form the basis of a spatial enhancement layer. These upsampled pictures may be refined using regular prediction mechanisms at higher resolutions to add details.

[0087] The embodiments discussed herein can be used in such environments. In certain cases, in the same or another embodiment, values ​​in the NAL unit header, e.g., the Temporal ID field, can be used to indicate not only temporal but also spatial layers. Doing so may provide certain advantages for certain system designs. For example, existing Selected Forwarding Units (SFUs) that are generated and optimized for temporal layer selective forwarding based on NAL unit header Temporal ID values ​​can be used without modification in scalable environments. To be effective, there may be a requirement that the mapping between coding picture size and temporal layers is indicated by the Temporal ID field in the NAL unit header.

[0088] In an embodiment, the adaptation window size may be signaled in the PPS. The adaptation window parameter may be used to calculate the resampling ratio when the adaptation window size of the reference picture is different from the adaptation window size of the current picture. The decoder may need to know the adaptation window size of each picture to determine whether a resampling process is necessary.

[0089] In an embodiment, the scale factor for reference picture resampling (RPR) may be calculated based on the output width and output height between the current picture and the reference picture, which may be derived from an adaptation window parameter. This may allow the scaling factor to be calculated more accurately compared to using the decoded picture size. This may work well for most video sequences where the output picture size is nearly the same as the decoded picture size, with small padding areas.

[0090] However, this may also cause various problems. For example, for immersive media applications (e.g., 360 cube map, stereoscopic, point cloud), when the adaptation window size is quite different from the decoded picture size due to a large offset value, the calculation of the scaling factor based on the adaptation window size may not guarantee the quality of inter prediction with different resolutions. In extreme cases, the co-location region of the current CU in the reference picture may not exist. When RPR is used for scalability with multiple layers, the adaptation window offset may not be used for the calculation of the reference region across layers. Note that in SHVC (HEVC Scalability Extension), the reference region of each directly dependent layer may be explicitly signaled in the PPS extension. When a sub-bitstream targeting a specific region (sub-picture) is extracted from the entire bitstream, the adaptation window size does not quite match the picture size. Note that the adaptation window parameter cannot be updated once the bitstream is coded, as long as the parameter is used for scaling calculation.

[0091] Based on the above potential problems, the calculation of the scaling factor based on the adaptation window size may have corner cases where alternative parameters are needed. As an alternative, when the adaptation window parameters are not available for the calculation of the scaling factor, it is proposed to signal a reference region parameter that can be used to calculate the scaling and scalability of the RPR.

[0092] In an embodiment, referring to Figure 7, conformance_window_flag may be signaled in the PPS. Conformance_window_flag equal to 1 may indicate that a conformance cropping window offset parameter follows in the PPS. Conformance_window_flag equal to 0 may indicate that a conformance cropping window offset parameter is not present.

[0093] In an embodiment, and still referring to Figure 7, conf_win_left_offset, conf_win_right_offset, conf_win_top_offset, and conf_win_bottom_offset specify the sample of the picture that refers to the PPS output from the decoding process in terms of a rectangular area specified in the picture coordinates for output. When conformance_window_flag is equal to 0, the values ​​of conf_win_left_offset, conf_win_right_offset, conf_win_top_offset, and conf_win_bottom_offset may be inferred to be equal to 0.

[0094] In an embodiment, a flag may be present in the PPS or another parameter set and may indicate whether the resampling picture size (width and height) is explicitly signaled in the PPS or another parameter set. If the resampling picture size parameter is explicitly signaled, the resampling ratio between the current picture and the reference picture may be calculated based on the resampling picture size parameter.

[0095] In an embodiment, referring to FIG. 7, use_conf_win_for_rpr_flag equal to 0 may indicate that resampled_pic_width_in_luma_samples and resampled_pic_height_in_luma_samples follow next in the appropriate location, e.g., within the PPS.

[0096] In an embodiment, use_conf_win_for_rpr_flag equal to 1 may indicate that resampling_pic_width_in_luma_samples and resampling_pic_height_in_luma_samples are not present.

[0097] In an embodiment, resampling_pic_width_in_luma_samples may specify the width of each reference picture that references a PPS for resampling in units of luma samples. resampling_pic_width_in_luma_samples may not be equal to 0, may be an integer multiple of Max(8,MinCbSizeY), and may be less than or equal to pic_width_max_in_luma_samples.

[0098] In an embodiment, resampling_pic_height_in_luma_samples may specify the height of each reference picture referencing a PPS in units of luma samples for resampling. resampling_pic_height_in_luma_samples may not be equal to 0, may be an integer multiple of Max(8,MinCbSizeY), and may be less than or equal to pic_height_max_in_luma_sample.

[0099] In an embodiment, when the syntax element resampling_pic_width_in_luma_samples is not present, the value of resampling_pic_width_in_luma_samples may be inferred to be equal to PicOutputWidthL.

[0100] When the syntax element resampling_pic_height_in_luma_samples is not present, the value of resampling_pic_height_in_luma_samples may be inferred to be equal to PicOutputHeightL.

[0101] In an embodiment, an example of fractional interpolation processing with reference picture resampling may be processed as follows.

[0102] The variable fRefWidth may be set equal to the resampling_pic_width_in_luma_samples of the reference picture in luma samples.

[0103] The variable fRefHeight may be set equal to resampling_pic_height_in_luma_samples of the reference picture in luma samples.

[0104] The motion vector mvLX may be set equal to (refMvLX-mvOffset).

[0105] When cIdx is equal to 0, the scaling coefficients and their fixed point representations may be defined according to Equation 1 and Equation 2 below.

number

[0106] In an embodiment, referring to Figure 8, use_conf_win_for_rpr_flag equal to 0 may specify that resampled_pic_width_in_luma_samples and resampled_pic_height_in_luma_samples follow next in the PPS. use_conf_wid_for_rpr_flag equal to 1 may specify that resampling_pic_width_in_luma_samples and resampling_pic_height_in_luma_samples are not present.

[0107] In an embodiment, ref_region_left_offset may specify the horizontal offset between the top-left luma samples of the reference region in the decoded picture. The value of ref_region_left_offset is −2 inclusive. 14 ~2 14Should be in the range -1. When not present, the value of ref_region_left_offset may be inferred to be equal to conf_win_left_offset.

[0108] In an embodiment, ref_region_top_offset may specify the vertical offset between the top-left luma samples of the reference region in the decoded picture. The value of ref_region_top_offset can be −2 inclusive. 14 ~2 14 Should be in the range -1. When not present, the value of ref_region_top_offset may be inferred to be equal to conf_win_right_offset.

[0109] In an embodiment, ref_region_right_offset may specify the horizontal offset between the bottom right luma samples of the reference region in the decoded picture. The value of ref_layer_right_offset is −2 inclusive. 14 ~2 14 Should be in the range -1. When not present, the value of ref_region_right_offset may be inferred to be equal to conf_win_top_offset.

[0110] In an embodiment, ref_region_bottom_offset may specify the vertical offset between the bottom right luma samples of the reference region in the decoded picture. The value of ref_layer_bottom_offset is −2 inclusive. 14 ~2 14 Should be in the range -1. When not present, the value of ref_region_bottom_offset[ref_loc_offset_layer_id[i]] may be inferred to be equal to conf_win_bottom_offset.

[0111] The variables PicRefWidthL and PicRefHeightL may be derived as shown in Equations 3 and 4, as follows:

number

[0112] The variable fRefWidth may be set equal to the PicRefWidthL of the reference picture in luma samples.

[0113] The variable fRefHeight may be set equal to the PicRefHeightL of the reference picture in luma samples.

[0114] The motion vector mvLX may be set equal to (refMvLX-mvOffset).

[0115] When cIdx is equal to 0, the scaling coefficients and their fixed-point representations may be defined as shown in Equation 5 and Equation 6 below.

number

[0116] Top-left coordinate of the border block for reference sample padding (xSbInt L ,ySbInt L ) may be set equal to (xSb + (mvLX[0] >> 4), ySb + (mvLX[1] >> 4)).

[0117] For each luma sample position (x L =0..sbWidth-1+brdExtSize,y L For each of the predicted luma samples, predSamplesLX[x L ][y L ] may be derived as follows: (refxSb L ,refySb L ) and (refx L ,refy LLet refxSb be the luma position pointed to by the motion vector (refMvLX, refMvLX) given in 1 / 16 sample units. L , refx L , refySb L , and refy L may be derived as shown in Equations 7 to 10 below.

number

[0118] Figure 9 is a flow chart of an example process 900 for decoding an encoded video bitstream. In some implementations, one or more of the processing blocks in Figure 9 may be performed by the decoder 210. In some implementations, one or more of the processing blocks in Figure 9 may be performed by another device or group of devices that are separate from or include the decoder 210, such as the encoder 203.

[0119] As shown in FIG. 9, the process 900 may include obtaining a flag indicating that an adaptive window is not used for reference picture resampling.

[0120] As further shown in FIG. 9, process 900 may include determining whether a resampling picture size is signaled based on the flag indicating that an adaptive window is not used for reference picture resampling (block 920).

[0121] If the second flag is determined to indicate that the resampled picture size is signaled (YES at block 920), process 900 may proceed to block 930 and then to block 950. At block 930, process 900 may include determining a resampling ratio based on the resampled picture size.

[0122] If it is determined that the second flag does not indicate that an adaptive window is used for reference picture resampling (NO at block 920), process 900 may proceed to block 940 and then to block 950. At block 940, process 900 may include determining a resampling ratio based on the output picture size.

[0123] As further shown in FIG. 9, process 900 may include performing reference picture resampling on the current picture using the resampling ratio (block 950).

[0124] In an embodiment, the flag may be signaled in the picture parameter set.

[0125] In an embodiment, the resampled picture size may be signaled in the coded video bitstream as at least one of a resampled picture size width and a resampled picture size height.

[0126] In an embodiment, at least one of the width and the height may be signaled in a picture parameter set.

[0127] In an embodiment, at least one of the width and the height may be expressed as a number of luma samples contained in the at least one of the width and the height.

[0128] In an embodiment, at least one of the width and the height may be determined based on at least one offset distance between an edge of the current picture and a given luma sample of the reference region.

[0129] In an embodiment, the at least one offset distance may be signaled in a picture parameter set.

[0130] In an embodiment, the at least one offset distance includes at least one of the following: a horizontal offset distance between the left edge of the current picture and the top-left luma sample of the reference area, a vertical offset distance between the top edge of the current picture and the top-left luma sample of the reference area, a horizontal offset distance between the right edge of the current picture and the bottom-right luma sample of the reference area, and a vertical offset distance between the bottom edge of the current picture and the bottom-right luma sample of the reference area.

[0131] 9 illustrates example blocks of process 900, which in some implementations may include more, fewer, or a different arrangement of blocks than those illustrated in FIG 9. Additionally or alternatively, two or more of the blocks of process 900 may be performed in parallel.

[0132] Additionally, the proposed methods may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium to perform one or more of the proposed methods.

[0133] The techniques described above may be implemented as computer software using computer readable instructions or physically stored on one or more computer readable media. For example, FIG. 10 illustrates a computer system 1000 suitable for implementing certain embodiments of the subject matter of this disclosure.

[0134] The computer software can be coded using any suitable machine code or computer language that can be processed by mechanisms such as assembly, compilation, linking, etc. to generate code containing instructions that can be executed by a computer central processing unit (CPU), graphics processing unit (GPU), etc., directly or through interpretation, microcode execution, etc.

[0135] The instructions may be executed in a variety of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, and the like.

[0136] 10 of the computer system 1000 are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing the embodiments of the present disclosure. Furthermore, the arrangement of components should not be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of the computer system 1000.

[0137] The computer system 1000 may include certain human interface input devices. Such human interface input devices may be responsive to input by one or more human users, for example, through sensory input (e.g., keystrokes, swipes, data grabbing actions), audio input (e.g., voice, clapping), visual input (e.g., gestures), olfactory input (not shown). Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as sound (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a digital camera), and video (including, for example, two-dimensional video, three-dimensional video, stereoscopic video).

[0138] The input human interface devices may include one or more of a keyboard 1001, a mouse 1002, a trackpad 1003, a touch screen 1010 and associated graphics adapter 1050, a data glove, a joystick 1005, a microphone 1006, a scanner 1007, and a camera 1008 (only one of which is shown).

[0139] The computer system 1000 may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses, for example, through sensory output, sound, light, and smell / taste. Such human interface output devices may include sensory output devices (e.g., sensory feedback via a touch screen 1010, data glove, or joystick 1005, although there may also be sensory feedback devices that do not function as input devices), audio output devices (e.g., speakers 1009, headphones (not shown), visual output devices (e.g., screen 1010, including a cathode ray tube (CRT) screen, a liquid crystal display (LCD) screen, a plasma screen, an organic light emitting diode (OLED) screen, each with or without touch screen input capability, each with or without sensory feedback capability, some of which may be capable of outputting more outputs, such as stereoscopic output, virtual reality glasses (not shown), holographic displays, and smoke generator tanks (not shown), and printers (not shown)).

[0140] The computer system 1000 may also include human accessible storage and associated media such as optical media such as CD / DVDROM / RW 1020 with media 1021 such as CD / DVDs, thumb drives 1022, removable hard drives or solid state drives 1023, legacy magnetic media such as tapes and floppy disks (not shown), dedicated ROM / ASIC / PLD based devices such as security dongles (not shown), etc.

[0141] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter of this disclosure does not encompass transmission media, carrier waves, or other transitory signals.

[0142] The computer system 1000 may also include an interface (1155) to one or more communication networks. The networks may be, for example, wireless, wired, or optical. The networks may further be local, wide area, metropolitan area, vehicular and industrial, real-time, delay tolerant, etc. Examples of networks include local area networks such as Ethernet, wireless LANs, global systems for mobile communications (GSM), cellular networks including 3G, 4G, 5G, LTE, etc., TV wired or wireless wide area digital networks including cable TV, satellite TV, terrestrial broadcast TV, vehicular and industrial including CANBus, etc. A particular network generally requires an external network interface adapter (1154) that is attached to a particular general-purpose data port or peripheral bus (1149) (e.g., a Universal Serial Bus (USB) port on the computer system 1000). Others are typically integrated into the core of the computer system 1000 by attachment to a system bus as described below (e.g., an Ethernet interface to a PC computer system, or a cellular network interface to a smartphone computer system). As an example, a network 1055 may be connected to the peripheral bus 1049 using a network interface 1054. Using these networks, the computer system 1000 can communicate with other entities. Such communications may be one-way receive only (e.g., broadcast TV), one-way transmit only (e.g., CANbus to a specific CANbus device), or bidirectional to other computer systems, for example, using local or wide area digital networks. Specific protocols and protocol stacks may be used with each of these networks and network interfaces (1154).

[0143] The aforementioned human interface devices, human accessible storage devices, and network interfaces may be attached to a core 1040 of the computer system 1000 .

[0144] The core 1040 may include one or more central processing units (CPUs) 1041, graphics processing units (GPUs) 1042, dedicated programmable processing units 1043 in the form of FPGAs, hardware accelerators 1044 for specific tasks, etc. These devices may be connected through a system bus 1048, along with read only memory (ROM) 1045, random access memory (RAM) 1046, internal mass storage devices 1047 such as internal non-user accessible hard drives, SSDs, etc. In some computer systems, the system bus 1048 is accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripherals can be attached directly to the core's system bus 1048 or through a peripheral bus 1049. Peripheral bus architectures include peripheral component interconnect (PCI), USB, etc.

[0145] The CPU 1041, GPU 1042, FPGA 1043, and accelerator 1044 may execute certain instructions that may be combined to generate the aforementioned computer code. The computer code may be stored in ROM 1045 or RAM 1046. Temporary data may also be stored in RAM 1046, while permanent data may be stored, for example, in internal mass storage device 1047. Rapid storage and retrieval from any of the memory devices may be made possible through the use of cache memory, which may be closely associated with one or more of the CPU 1041, GPU 1042, mass storage device 1047, ROM 1045, RAM 1046, etc.

[0146] The computer-readable medium may bear computer code for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0147] As an example and not by way of limitation, computer system 1000 having the architecture, and core 1040 in particular, can provide functionality as a result of processors (including CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer readable media. Such computer readable media can be specific storage of core 1040 of a non-transitory nature, such as core internal mass storage 1047 or ROM 1045, and media associated with user accessible mass storage as described above. Software implementing various embodiments of the disclosure can be stored in such devices and executed by core 1040. Computer readable media can include one or more memory devices or chips, according to specific needs. The software can cause core 1040, and specifically the processors therein (including CPU, GPU, FPGA, etc.) to perform specific operations or specific portions of specific operations described herein, including defining and modifying data structures stored in RAM 1046 according to software-defined operations. Additionally or alternatively, a computer system may provide functionality as a result of implementation in hardwired or other circuitry (e.g., accelerator 1044) that can operate in conjunction with or in place of software to perform certain processes or certain portions of certain processes described herein. References to software include logic, and vice versa, where appropriate. References to computer-readable media may include, where appropriate, circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry that implements logic for execution, or both. The present disclosure includes any appropriate combination of hardware and software.

[0148] While this disclosure has described several exemplary embodiments, there are alterations, permutations, and various substitute equivalents, which are encompassed within the scope of this disclosure. It will be apparent to those skilled in the art that numerous systems and methods can be devised that, although not explicitly shown or described herein, embody the principles of the disclosure and thus are encompassed within the spirit and scope of the disclosure. [Explanation of symbols]

[0149] 100 Systems 110, 120, 130, 140 terminals 150 Network

Claims

1. 1. A method for generating an encoded video bitstream using at least one processor, comprising: generating the encoded video bitstream including (i) a flag indicating whether a fitting window is used for reference picture resampling; and (ii) reference picture resampling parameters; the flag indicates that the adaptation window is not used for reference picture resampling; Based on a determination that the adaptive window is not used for the reference picture resampling and a determination that the reference picture resampling parameters should be signaled, the reference picture resampling parameters include a plurality of reference region parameters used to calculate a scaling for the reference picture resampling, the plurality of reference region parameters including at least a reference region left offset and a reference region top offset; a current picture is encoded using the reference picture resampling parameters including the plurality of reference region parameters; method.

2. The method of claim 1 , wherein the flag is signaled in a picture parameter set.

3. The method further comprises the step of determining whether a resampling picture size is signaled based on the flag indicating that the fitting window is not used for the reference picture resampling; Based on a determination that the resampled picture size is signaled, a resampling ratio is determined based on the resampled picture size; the encoded video bitstream further comprising at least one of a width of the resampled picture size and a height of the resampled picture size. The method of claim 1.

4. The method of claim 3 , wherein the at least one of the width and the height is signaled in a picture parameter set.

5. The method of claim 3 , wherein the at least one of the width and the height is expressed as a number of luma samples contained in the at least one of the width and the height.

6. The method of claim 3 , wherein the at least one of the width and the height is determined based on at least one offset distance between a boundary of the current picture and a given luma sample of a reference region.

7. The method of claim 6 , wherein the at least one offset distance is signaled in a picture parameter set.

8. The at least one offset distance is: the horizontal offset distance between the left boundary of the current picture and the top-left luma sample of the reference area; a vertical offset distance between a top boundary of the current picture and the top-left luma sample of the reference region; the horizontal offset distance between the right boundary of the current picture and the bottom right luma sample of the reference area; and The vertical offset distance between the bottom boundary of the current picture and the bottom right luma sample of the reference area. The method of claim 6, comprising at least one of:

9. 1. A device for generating an encoded video bitstream, comprising: at least one memory configured to store program code; and at least one processor configured to read the program code and to operate as instructed by the program code, the processor executing the program code to perform the steps of the method according to any one of claims 1 to 8. device.

10. A program for causing a computer to execute each step of the method according to any one of claims 1 to 8.

11. A non-transitory computer readable medium storing the program of claim 10.

12. 1. A method for decoding an encoded video bitstream using at least one processor, comprising: (i) a flag indicating whether a fitting window is used for reference picture resampling; and (ii) a reference picture resampling parameter, the flag indicates that the adaptation window is not used for reference picture resampling; Based on a determination that the adaptive window is not used for the reference picture resampling and a determination that the reference picture resampling parameters should be signaled, the reference picture resampling parameters include a plurality of reference region parameters used to calculate a scaling for the reference picture resampling, the plurality of reference region parameters including at least a reference region left offset and a reference region top offset; reference picture resampling is performed on the current picture using the reference picture resampling parameters including the plurality of reference region parameters; method.

Images