Scaled Intra Reference Picture

Abstract

Aspects of the present disclosure include a method and apparatus for video coding. The apparatus includes a processing circuit that receives a bitstream including a first sub-bitstream corresponding to a scaled version of a current picture and a second sub-bitstream corresponding to a full-scale current picture. The processing circuit reconstructs the scaled version of the current picture from the first sub-bitstream and reconstructs a second block of the full-scale current picture based on (i) partitioning information of one or more first blocks in the scaled version of the current picture or (ii) intra-prediction information of the one or more first blocks.

Description

[Technical Field]

[0001] [Related Applications] This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 425,557, "Scaled Intra Reference Picture," filed November 15, 2022, which claims the benefit of priority to U.S. Patent Application No. 18 / 376,333, "SCALED INTRA REFERENCE PICTURE," filed October 3, 2023. The disclosures of the foregoing applications are incorporated herein by reference in their entireties.

[0002] [Technical field] This disclosure generally describes embodiments related to video coding. [Background technology]

[0003] The background description provided herein is intended to provide a general background to the present disclosure. The work of the presently named inventors is not expressly or implicitly admitted as prior art to the present disclosure, to the extent that the work described in this background section, as well as aspects of the description that may not be considered prior art at the time of filing, is not admitted as prior art to the present disclosure.

[0004] Image / video compression helps transfer image / video data between various devices, storage, and networks with minimal loss of quality. In some examples, video codec techniques can compress video based on spatial and temporal redundancy. For example, video codecs can use a technique called intra-prediction, which can compress images based on spatial redundancy. For example, intra-prediction can use reference data from the current picture being reconstructed for sample prediction. In another example, video codecs can use a technique called inter-prediction, which can compress images based on temporal redundancy. For example, inter-prediction can predict samples in a current picture from a previously reconstructed picture using motion compensation. Motion compensation is indicated by a motion vector (MV). Summary of the Invention

[0005] Aspects of the disclosure include methods and devices for video encoding / decoding. In some examples, the device for video decoding includes a processing circuit. The processing circuit receives a bitstream including a first sub-bitstream corresponding to a scaled version of a current picture having a first spatial resolution and a second sub-bitstream corresponding to a full-scale current picture, the full-scale having a second spatial resolution higher than the first spatial resolution. The processing circuit reconstructs the scaled version of the current picture from the first sub-bitstream and reconstructs a second block of the full-scale current picture based on (i) partitioning information of one or more first blocks in the scaled version of the current picture or (ii) intra-prediction information of one or more first blocks of the scaled version.

[0006] In the example, a first region in the scaled version of the current picture is co-located with a second block in the full-scale current picture.

[0007] In the example, a first sample in a second block can be predicted based on a reconstructed sample in a first region in a scaled version of the current picture. The first sample in the second block is co-located with the reconstructed sample in the first region. The second sample in the second block can be predicted by interpolation based on at least the predicted first sample in the second block and the reconstructed sample to the top left of the second block. The predictor for the second block includes (i) the predicted first sample in the second block, (ii) the reconstructed sample to the top left of the second block, and (iii) the predicted second sample in the second block. The second block is reconstructed from the predictor for the second block.

[0008] In an example, samples in the second block are predicted using intra prediction. (i) The predicted samples in the second block and (ii) the corresponding upsampled reconstructed samples in the first region in the scaled version of the current picture can be blended using a weighted average. The weight of the blended samples in the second block depends on the position of the blended samples within the second block.

[0009] In an example, reconstructed samples in a first region of a scaled version of a current picture are upsampled, the upsampled samples in the first region of the scaled version are filtered, and a second block can be reconstructed with the filtered upsampled samples in the first region as a predictor of the second block.

[0010] In an example, residuals of samples in the second block can be predicted based on residuals of reconstructed samples in the first region. Whether samples in the second block have non-zero residuals can be predicted based on the distribution of non-zero residuals in the first region. In an example, the distribution of non-zero residuals in the second block is arithmetically decoded, and the distribution of non-zero residuals in the first region is used as a context for the arithmetic decoding.

[0011] In an embodiment, the partitioning information of the one or more first blocks indicates whether each of the one or more first blocks is to be divided into smaller blocks. The second block can be reconstructed based on the partitioning information of the one or more first blocks by determining whether to partition the second block in the full-scale current picture based on the partitioning information of the one or more first blocks. The processing circuit reconstructs the second block based on the determination of whether to partition the second block.

[0012] In an example, the second sub-bitstream includes a flag for the second block. In response to a decision to partition the second block, the flag indicates whether to apply partitioning to blocks partitioned from the second block. In response to a decision to partition the second block, the flag indicates whether to apply partitioning to the second block.

[0013] In an example, the partitioning information includes a flag for each of the one or more first blocks indicating whether to divide each first block into smaller blocks, and the partitioning information for the second block can be entropy decoded, and the flag for each of the one or more first blocks can be used as context for the entropy decoding.

[0014] In an example, the intra prediction information includes intra prediction mode (IPM) information for one or more first blocks. A most probable mode (MPM) list for a second block can be constructed based on the IPM information for one or more first blocks. The second block can be reconstructed based on the MPM list.

[0015] In an example, the intra prediction information includes reference line index information of one or more first blocks. The reference line index of the second block can be determined based on the reference line index information of the one or more first blocks, and the second block can be reconstructed based on the reference line index of the second block.

[0016] In the example, the second sub-bitstream indicates that intra-skip mode is used for the second block. The intra-prediction information indicates a prediction mode of one of the one or more first blocks. The prediction mode of the one or more first blocks can be used for the second block, and the second block can be reconstructed based on the prediction mode.

[0017] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by a computer, cause the computer to perform a method for video encoding / decoding. [Brief explanation of the drawings]

[0018] Further features, characteristics, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings.

[0019] [Figure 1] FIG. 1 is a schematic diagram of an exemplary block diagram of a communication system (100).

[0020] [Figure 2] FIG. 2 is a schematic diagram of an exemplary block diagram of a decoder.

[0021] [Figure 3] FIG. 2 is a schematic diagram of an exemplary block diagram of an encoder.

[0022] [Figure 4] 1 illustrates the locations of spatial merge candidates according to an embodiment of the present disclosure.

[0023] [Figure 5] 1 illustrates candidate pairs considered for a spatial merge candidate redundancy check according to one embodiment of the present disclosure.

[0024] [Figure 6] 10 illustrates exemplary motion vector scaling for temporal merge candidates.

[0025] [Figure 7] 10 illustrates exemplary candidate positions for temporal merge candidates for the current CU.

[0026] [Figure 8] 1 illustrates an example of an intra-template matching prediction (IntraTMP) mode according to one embodiment of the present disclosure.

[0027] [Figure 9] An example of a reference region for coding CTU(m,n) is shown below.

[0028] [Figure 10] 1 illustrates a full-scale current picture and a scaled version of the current picture according to an embodiment of the present disclosure.

[0029] [Figure 11] 1 shows a flowchart outlining a decoding process according to certain embodiments of the present disclosure.

[0030] [Figure 12] 1 shows a flowchart outlining an encoding process according to certain embodiments of the present disclosure.

[0031] [Figure 13] FIG. 1 is a schematic diagram of a computer system, according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0032] 1 shows a block diagram of a video processing system 100 in some examples. The video processing system 100 is a video encoder and video decoder in a streaming environment, which is one example of an application of the disclosed subject matter. The disclosed subject matter is equally applicable to, for example, video conferencing, digital TV, streaming services, storing compressed video on digital media including CDs, DVDs, memory sticks, etc., other video-enabled applications, etc.

[0033] The video processing system 100 includes a video source 101, e.g., a capture subsystem 113, which may include a digital camera, that generates an uncompressed video picture stream 102. In one example, the video picture stream 102 includes samples captured by the digital camera. The video picture stream 102, shown in bold to emphasize its high data volume when compared to the encoded video data 104 (or coded video bitstream), may be processed by an electronic device 120 that includes a video encoder 103 coupled to the video source 101. The video encoder 103 may include hardware, software, or a combination thereof, and may enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video data 104 (or coded video bitstream), shown in thin to emphasize its low data volume when compared to the video picture stream 102, may be stored on a streaming server 105 for future use. One or more streaming client subsystems, such as the client subsystems 106 and 108 of FIG. 1, can access the streaming server 105 to retrieve copies 107 and 109 of the encoded video data 104. The client subsystem 106 may include a video decoder 110, for example, within an electronic device 130. The video decoder 110 decodes the input copy 107 of the encoded video data and generates an output video picture stream 111 that can be rendered on a display 112 (e.g., a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data 104, 107, and 109 (e.g., a video bitstream) may be encoded according to a particular video coding / compression standard. Examples of these standards include ITU-T Recommendation H.265. In one example, a video coding standard under development is known informally as Versatile Video Coding (VVC). The disclosed subject matter may be used in the context of VVC.

[0034] It should be noted that electronic devices 120 and 130 may include other components (not shown). For example, electronic device 120 may include a video decoder (not shown), and electronic device 130 may also include a video encoder (not shown).

[0035] 2 shows an exemplary block diagram of a video decoder (210). The video decoder (210) may be included in an electronic device (230). The electronic device (230) may include a receiver (231) (e.g., a receiving circuit). The video decoder (210) may be used in place of the video decoder (110) in the example of FIG. 1.

[0036] The receiver (231) can receive one or more coded video sequences, e.g., in a bitstream, to be decoded by the video decoder (210). In an embodiment, one coded video sequence is received at a time, with the decoding of each coded video sequence being independent of the decoding of other coded video sequences. The coded video sequences may be received from a channel (201), which may be a hardware / software link to a storage device that stores the coded video data. The receiver (231) may receive the coded video data along with other data, e.g., coded audio data and / or auxiliary data streams, which may be forwarded to respective using entities (not shown). The receiver (231) may separate the coded video sequences from other data. To eliminate network jitter, a buffer memory (215) may be coupled between the receiver (231) and the entropy decoder / parser (220) (hereinafter, "parser (220)"). In certain applications, the buffer memory (215) is part of the video decoder (210). Alternatively, it may be external to the video decoder 210 (not shown). Still alternatively, there may be a buffer memory (not shown) external to the video decoder 210, e.g., to remove network jitter, in addition to another buffer memory 215 internal to the video decoder 210, e.g., to handle playout timing. When the receiver 231 is receiving data controllably from a store / forward device of sufficient bandwidth or from an isosynchronous network, the buffer memory 215 may not be needed or may be small. For use with best-effort packet networks such as the Internet, the buffer memory 215 may be needed, but it may be relatively large, advantageously of adaptive size, and implemented at least in part in an operating system or similar element (not shown) external to the video decoder 210.

[0037] The video decoder (210) may include a parser (220) to reconstruct symbols (221) from the coded video sequence. These symbol categories include information used to manage the operation of the video decoder (210) and information for controlling a rendering device, such as a render device (212) (e.g., a display screen), which may not be an integral part of the electronic device (230) but may be coupled to the electronic device (230) as shown in FIG. 2. The control information for the rendering device may be in the form of a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (220) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may follow a video coding technique or standard and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context dependency, etc. The parser (220) may extract from the coded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the group. The subgroups may include groups of pictures (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The parser (220) may also extract information such as transform coefficients, quantization parameter values, motion vectors, etc. from the coded video sequence.

[0038] The parser (220) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (215) to generate symbols (221).

[0039] The reconstruction of the symbols (221) may include several different units, depending on the type of coded video picture or portion thereof (e.g., inter- and intra-picture, inter- and intra-block) and other factors. Which units are included and how can be controlled by group control information parsed by the parser (220) from the coded video sequence. The flow of such subgroup control information between the parser (220) and the following units is not shown for clarity.

[0040] Beyond the functional blocks already mentioned, the video decoder (210) can be conceptually subdivided into a number of functional units, as described below. In an actual implementation operating under commercial constraints, many of these units will interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, the following conceptual subdivision into functional units is appropriate.

[0041] The first unit is a scalar / inverse transform unit 251. The scalar / inverse transform unit (251) receives quantized transform coefficients and control information from the parser (220) as symbols (221), including which transform to use, block size, quantization coefficients, quantization scaling matrices, etc. The scalar / inverse transform unit (251) can output blocks containing sample values ​​that can be input to an aggregator (255).

[0042] In some cases, the output samples of the scaler / inverse transform unit (251) may relate to intra-coded blocks. Intra-coded blocks are blocks that do not use prediction information from a previously reconstructed picture but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information can be provided by the intra-picture prediction unit (252). In some cases, the intra-picture prediction unit (252) generates a block of the same size and shape as the block being reconstructed using surrounding, already reconstructed information fetched from the current picture buffer (258). The current picture buffer (258), for example, buffers the reconstructed current picture partially and / or completely. The aggregator (255), in some cases, adds the prediction information generated by the intra-prediction unit (252) to the output sample information provided by the scaler / inverse transform unit (251) on a sample-by-sample basis.

[0043] In other cases, the output samples of the scaler / inverse transform unit (251) may relate to an inter-coded, possibly motion-compensated, block. In such cases, the motion-compensated prediction unit (253) can access the reference picture memory (257) to fetch samples used for prediction. After motion-compensating the fetched samples according to the symbols (221) associated with the block, these samples may be added by the aggregator (255) to the output of the scaler / inverse transform unit (251) to generate output sample information (in this case, referred to as residual samples or residual signals). The addresses in the reference picture memory (257) from which the motion-compensated prediction unit (253) fetches prediction samples can be controlled by the motion-compensated prediction unit (253)'s available motion vectors, e.g., in the form of symbols (221) that may have X, Y, and reference picture components. Motion compensation may include interpolation of sample values ​​fetched from the reference picture memory (257) when sub-sample accurate motion vectors are in use, motion vector prediction mechanisms, etc.

[0044] The output samples of the aggregator (255) may undergo various loop filtering techniques in a loop filter unit (256). Video compression techniques may include in-loop filtering techniques controlled by parameters contained in the coded video sequence (also called the coded video bitstream) and made available to the loop filter unit (256) as symbols (221) from the parser (220). Video compression may not only be responsive to previously reconstructed, loop-filtered sample values, but may also be responsive to meta-information obtained during the decoding of previous portions (in decoding order) of the coded picture or coded video sequence.

[0045] The output of the loop filter unit (256) may be a sample stream that can be output to a render device (212) and stored in a reference picture memory (257) for use in future inter-picture prediction.

[0046] Once a particular coded picture is fully reconstructed, it can be used as a reference picture for future prediction. For example, once the coded picture corresponding to the current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (220)), the current picture buffer (258) can become part of the reference picture memory (257), and a fresh current picture buffer can be reallocated before beginning reconstruction of a subsequent coded picture.

[0047] The video decoder 210 may perform decoding operations in accordance with a standard or predetermined video compression technology, such as ITU-T Rec. H.265. The coded video sequence may conform to the syntax specified by the video compression technology or standard in use, in the sense that the coded video sequence conforms to both the video compression technology or standard and a profile documented in the video compression technology or standard. Specifically, a profile may select certain tools from the full set of tools available in the video compression technology or standard as tools usable only under the profile. Compliance may also require that the complexity of the coded video sequence be within limits defined by the level of the video compression technology or standard. In some cases, the level may limit the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level may, in some cases, be further constrained through a Hypothetical Reference Decoder (HRD) specification and metadata for HRD buffer management signaled within the coded video sequence.

[0048] In embodiments, the receiver (231) may receive additional (redundant) data along with the coded video. The additional data may be included as part of the coded video sequence. The additional data may be used by the video decoder (210) to correctly decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0049] 3 shows an exemplary block diagram of a video encoder (303). The video encoder (303) is included in an electronic device (320). The electronic device (320) includes a transmitter (340) (e.g., a transmitting circuit). The video encoder (303) can be used in place of the video encoder (103) in the example of FIG. 1.

[0050] The video encoder (303) may receive video samples from a video source (301) (which, in the example of FIG. 3, is not part of the electronic device (320)) that may capture video images to be coded by the video encoder (303). In another example, the video source (301) is part of the electronic device (320).

[0051] The video source (301) may provide a source video sequence to be coded by the video encoder (303) in the form of a digital video sample stream of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media presentation system, the video source (301) may be a storage device that stores previously prepared video. In a video conferencing system, the video source (301) may be a camera that captures local image information as a video sequence. The video data may be provided as multiple individual pictures that, when viewed sequentially, give the appearance of motion. The pictures themselves may be organized as a spatial array of pixels. Each pixel may contain one or more samples, depending on the sampling structure, color space, etc., in use. The following discussion focuses on samples.

[0052] According to an embodiment, the video encoder (303) may code and compress pictures of a source video sequence into a coded video sequence (343) in real time or under any other required time constraints. Enforcing the appropriate coding rate is one function of the controller (350). In some embodiments, the controller (350) controls and is operatively coupled to other functional units, described below. The coupling is not shown for clarity. Parameters set by the controller (350) may include rate control-related parameters (picture skip, quantizer, lambda value for rate-distortion optimization techniques, ...), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (350) may be configured with other appropriate functionality associated with the video encoder (303) optimized for a particular system design.

[0053] In some embodiments, the video encoder (303) is configured to operate within a coding loop. As a highly simplified explanation, in one example, the coding loop may include a source coder (330) (e.g., responsible for generating symbols, such as a symbol stream, based on an input picture to be coded and a reference picture) and a (local) decoder (333) built into the video encoder (303). The decoder (333) reconstructs the symbols to create sample data in a manner similar to that created by a (remote) decoder. The reconstructed sample stream (sample data) is input to a reference picture memory (334). When decoding the symbol stream yields bit-exact results independent of the decoder location (local or remote), the contents of the reference picture memory (334) are also bit-exact between the local and remote encoders. In other words, the predictive portion of the encoder "sees" exactly the same sample values ​​as the decoder "sees" when using prediction during decoding. This basic principle of reference picture synchronism (and the resulting drift when synchronism cannot be maintained, for example due to channel errors) is similarly used in several related techniques.

[0054] The operation of the "local" decoder (333) may be the same as that of a "remote" decoder, such as the video decoder (210) described in detail above in connection with Figure 2. However, and referring briefly to Figure 2 as well, the entropy decoding portion of the video decoder (210), including the buffer memory (215) and parser (220), may not be fully implemented in the local decoder (333) because symbols are available and the encoding / decoding of the symbols into a coded video sequence by the entropy coder (345) and parser (220) may be lossless.

[0055] In embodiments, decoder techniques, excluding analysis / entropy decoding, present in a decoder are present in the corresponding encoder in the same or substantially the same functional form. Therefore, the subject matter of the described disclosure focuses on decoder operation. A description of the encoder techniques can be omitted, as they are the reverse of the decoder techniques, which are described generically. In certain areas, more detailed descriptions are provided below.

[0056] In operation, in some examples, the source coder (330) may perform motion-compensated predictive coding, which predictively codes an input picture with reference to one or more previously coded pictures from a video sequence designated as "reference pictures." In this method, the coding engine (332) codes differences between pixel blocks of the input picture and pixel blocks of reference pictures that may be selected as prediction references for the input picture.

[0057] The local video decoder (333) may decode coded video data of pictures that may be designated as reference pictures based on symbols generated by the source coder (330). The operation of the coding engine (332) may advantageously be lossy. When the coded video data is decoded in a video decoder (not shown in FIG. 3), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (333) may replicate the decoding process that may be performed by a video decoder on the reference pictures, resulting in reconstructed reference pictures to be stored in the reference picture memory (334). In this way, the video encoder (303) may store copies of reconstructed reference pictures that have content in common with reconstructed reference pictures obtained by a far-end video decoder (absent transmission errors).

[0058] The predictor (335) may perform a predictive search for the coding engine (332). That is, for a new picture to be coded, the predictor (335) may search the reference picture memory (334) for sample data (such as candidate reference pixel blocks) or specific metadata such as reference picture motion vectors, block shapes, etc. that can serve as appropriate prediction references for the new picture. The predictor (335) may operate on a sample block-pixel block basis to find an appropriate prediction reference. In some examples, an input picture may have prediction references drawn from multiple reference pictures stored in the reference picture memory (334), as determined by the search results obtained by the predictor (335).

[0059] The control unit (350) may manage the coding operations of the source coder (330), including, for example, setting parameters and subgroup parameters used for encoding the video data.

[0060] The output of all of the aforementioned functional units may undergo entropy coding in an entropy coder (345), which converts the symbols produced by the various functional units into a coded video sequence by applying lossless compression to the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0061] The transmitter (340) may buffer the coded video sequence produced by the entropy coder (345) for transmission over a communication channel (360), which may be a hardware / software link to a storage device that may store the coded video data. The transmitter (340) may merge the coded video data from the video encoder (303) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0062] The controller (350) may manage the operation of the video encoder (303). During coding, the controller (350) may assign a particular coded picture type to each coded picture, which may affect the coding technique that may be applied to each picture. For example, pictures may often be assigned as one of the following picture types:

[0063] Intra-pictures (I-pictures) may be coded and decoded without using any other picture in the sequence as a source of prediction. Some video codecs allow different types of intra-pictures, including, for example, Independent Decoder Refresh (IDR) pictures.

[0064] A predictive picture (P-picture) may be a picture that can be coded and decoded using intra- or inter-prediction, in most cases using motion vectors and reference indices to predict the sample values ​​of each block.

[0065] A Bi-directionally Predictive Picture (B Picture) may be coded and decoded using intra- or inter-prediction, using two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, a multi-predictive picture can use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0066] A source picture is generally spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and may be coded block by block. Blocks may be predictively coded with reference to other (already coded) blocks determined by the coding assignment applied to each picture of the block. For example, blocks of an I-picture may be non-predictively coded, or they may be predictively coded with reference to already coded blocks of the same picture (spatial prediction or intra-prediction). Pixel blocks of a P-picture may be predictively coded via spatial prediction or via temporal prediction with reference to one previously coded reference picture. Blocks of a B-picture may be predictively coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference pictures.

[0067] The video encoder (303) may perform coding operations according to a predetermined video coding technique or standard, such as ITU-T Rec. H.265. In doing so, the video encoder (303) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. The coded video data may therefore conform to a syntax specified by the video coding technique or standard being used.

[0068] In one embodiment, the transmitter (340) may transmit additional data along with the coded video. The source coder (330) may include such data as part of the coded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0069] Video may be captured as multiple source pictures (video pictures) in a time sequence. Intra-picture prediction (sometimes abbreviated as intra-prediction) exploits spatial correlation within a given picture, while inter-picture prediction exploits correlation (temporal or other) between pictures. In one example, a particular picture being encoded / decoded is called the current picture and is partitioned into blocks. When a block in the current picture is similar to a reference block in a previously coded and still buffered reference picture in the video, the block in the current picture can be coded by a vector called a motion vector. A motion vector points to a reference block within the reference picture and may have a third dimension that identifies the reference picture if multiple reference pictures are in use.

[0070] In some embodiments, bi-prediction techniques can be used in inter-picture prediction. Bi-prediction techniques use two reference pictures, such as a first reference picture and a second reference picture, both of which precede the current picture in the video in decoding order (but may be past and future, respectively, in display order). A block in the current picture can be coded with a first motion vector that points to a first reference block in the first reference picture and a second motion vector that points to a second reference block in the second reference picture. A block can be predicted by a combination of the first and second reference blocks.

[0071] Furthermore, merge mode techniques can be used in inter-picture prediction to improve coding efficiency.

[0072] According to some embodiments of the present disclosure, prediction, such as inter-picture prediction and intra-picture prediction, is performed in units of blocks. For example, according to the HEVC standard, pictures in a video picture sequence are partitioned into coding tree units (CTUs) for compression. CTUs within a picture have the same size, such as 64x64 pixels, 32x32 pixels, or 16x16 pixels. Typically, a CTU includes three coding tree blocks (CTBs): one luma CTB and two chroma CTBs. Each CTU can be recursively quadtree-decomposed into one or more coding units (CUs). For example, a 64x64 pixel CTU can be partitioned into one CU of 64x64 pixels, four CUs of 32x32 pixels, or 16 CUs of 16x16 pixels. In one example, each CU is analyzed to determine the CU's prediction type, such as an inter-prediction type or an intra-prediction type. A CU is divided into one or more prediction units (PUs) depending on temporal and / or spatial predictability. Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In one embodiment, prediction operations in coding (encoding / decoding) are performed in units of prediction blocks. Using a luma prediction block as an example of a prediction block, the prediction block includes a matrix of values ​​(e.g., luma values) for pixels, such as 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 pixels, etc.

[0073] It is noted that the video encoders (103) and (303) and the video decoders (110) and (210) may be implemented using any suitable technology. In one embodiment, the video encoders (103) and (303) and the video decoders (110) and (210) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (103) and (303) and the video decoders (110) and (210) may be implemented using one or more processors executing software instructions.

[0074] VVC allows various inter-prediction modes. For inter-predicted CUs, motion parameters can include MVs, one or more reference picture indices, a reference picture list usage index, and additional information about the specific coding function used to generate the inter-predicted samples. Motion parameters can be signaled explicitly or implicitly. If a CU is coded in skip mode, the CU can be associated with a PU and may not have significant residual coefficients, coded motion vector deltas or MV differences (e.g., MVDs), or reference picture indices. Merge mode can be specified when the motion parameters of the current CU are obtained from neighboring CUs, including spatial and / or temporal candidates, and optionally additional information such as those introduced in VVC. Merge mode can be applied not only to skip mode but also to inter-predicted CUs. In an example, an alternative to merge mode is explicit transmission of motion parameters, where the MVs, the corresponding reference picture indices of each reference picture list, and a reference picture list usage flag and other information are explicitly signaled for each CU.

[0075] In embodiments such as VVC, the VVC Test model (VTM) reference software includes enhanced merge prediction, merge motion vector difference (MMVD) mode, adaptive motion vector prediction (AMVP) mode with symmetric MVD signaling, affine motion compensation prediction, subblock-based temporal motion vector prediction (SbTMVP), adaptive motion vector resolution (AMVR), motion field storage (1 / 16 luma sample MV storage and 8x8 motion field compression), bi-prediction with CU-level weights (BCW), bi-directional optical flow (BDOF), prediction refinement using optical flow (PROF), decoder side motion vector refinement (DMVR), combined inter and intra prediction It includes one or more sophisticated inter-prediction coding tools, including inter-prediction coding (CIIP), geometric partitioning mode (GPM), etc. Inter-prediction and related methods are described in detail below.

[0076] In some instances, enhanced merge prediction can be used. For example, in VTM4, such a merge candidate list is constructed by including five types of candidates, in order: spatial MVPs from spatially neighboring CUs, temporal MVPs from co-located CUs, history-based MVPs (HMVPs) from a first-in-first-out (FIFO) table, pairwise average MVPs, and zero MVs.

[0077] The size of the merge candidate list can be signaled in the slice header. In the example, the maximum allowed size of the merge candidate list is 6 in VTM4. For each CU coded in merge mode, the index of the best merge candidate (e.g., merge index) can be coded using truncated unary binarization (TU). The first bin of the merge index can be coded with context (e.g., context-adaptive binary arithmetic coding (CABAC)), and bypass coding can be used for the other bins.

[0078] Some examples of the process for generating each category of merge candidates are given below. In an embodiment, spatial candidates are derived as follows: The derivation of spatial merge candidates in VVC can be the same as that in HEVC. In an example, up to four merge candidates are selected from the candidates at the positions shown in FIG. 4. FIG. 4 illustrates the positions of spatial merge candidates according to an embodiment of the present disclosure. Referring to FIG. 4, the derivation order is B1, A1, B0, A0, B2. Position B2 is considered only if the CUs at positions A0, B0, B1, and A1 are unavailable (e.g., because the CUs belong to another slice or another tile) or are intra-coded. After the candidate at position A1 is added, the addition of the remaining candidates is subject to a redundancy check, which ensures that candidates with the same motion information are removed from the candidate list, thus improving coding efficiency.

[0079] To reduce the amount of computation, not all possible candidate pairs are considered in the aforementioned redundancy check. Instead, only pairs linked by arrows in FIG. 5 are considered, and a candidate is added to the candidate list only if the corresponding candidates used in the redundancy check do not have the same motion information. FIG. 5 illustrates candidate pairs considered for a spatial merge candidate redundancy check according to one embodiment of the present disclosure. Referring to FIG. 5, each arrow-linked pair includes A1 and B1, A1 and A0, A1 and B2, B1 and B0, and B1 and B2. This allows candidates at positions B1, A0, and / or B2 to be compared with candidates at position A1, and candidates at positions B0 and / or B2 to be compared with candidates at position B1.

[0080] In an embodiment, temporal candidates are derived as follows: In the example, only one temporal merge candidate is added to the candidate list. Figure 6 shows exemplary motion vector scaling for temporal merge candidates. To derive a temporal merge candidate for a current CU (611) in a current picture (601), a scaled MV (621) (e.g., shown by a dotted line in Figure 6) can be derived based on a co-located CU (612) belonging to a co-located reference picture (604). In the example, a co-located reference picture (also referred to as a co-located picture) is, for example, a specific reference picture used for temporal motion vector prediction. The co-located reference picture used for temporal motion vector prediction can be indicated by a reference index in syntax such as a high-level syntax (e.g., a picture header, a slice header).

[0081] The reference picture list used to derive the co-located CU (612) can be explicitly signaled in the slice header. The scaled MV (621) for the temporal merge candidate can be obtained as shown by the dotted line in Figure 6. The scaled MV (621) can be scaled from the MV of the co-located CU (612) using picture order count (POC) distances tb and td. The POC distance tb can be defined as the POC difference between the current reference picture (602) of the current picture (601) and the current picture (601). The POC distance td can be defined as the POC difference between the co-located reference picture (604) of the co-located reference picture (603) and the co-located reference picture (603). The reference picture index of the temporal merge candidate can be set to 0.

[0082] FIG. 7 shows exemplary candidate positions (e.g., C0 and C1) for temporal merge candidates for the current CU. The position of the temporal merge candidate can be selected from candidate positions C0 and C1. Candidate position C0 is located at the bottom right corner of the co-located CU (710) of the current CU. Candidate position C1 is located at the center of the co-located CU of the current CU. If the CU at candidate position C0 is unavailable, intra-coded, or outside the current row of the CTU, candidate position C1 is used to derive the temporal merge candidate. Otherwise, for example, if the CU at candidate position C0 is available, intra-coded, and in the current row of the CTU, candidate position C0 is used to derive the temporal merge candidate.

[0083] Intra block copy (IBC) mode can be used in video coding such as HEVC and VVC. For example, as in HEVC, the IBC concept requires additional memory in the DPB, and external memory is used in hardware implementations. The additional external memory accesses result in increased memory bandwidth. For example, as in VVC, the IBC mode uses fixed memory, which allows IBC to be implemented using on-chip memory, significantly reducing memory bandwidth requirements and hardware complexity. A reference sample memory (RSM) can be used to hold samples for a single CTU. A special feature of RSM is its continuous update mechanism, which replaces the reconstructed samples of the left-neighboring CTU with the reconstructed samples of the current CTU. Block vector (BV) coding in IBC employs the concept of a merge list for inter-prediction. The IBC list construction process considers two spatially neighboring BVs and five history-based BVs (HBVPs). For example, only the first HBVP is compared with a spatial candidate when it is added to a candidate list. While conventional inter prediction uses two different candidate lists, one for merge mode and the other for normal mode, IBC's candidate list is for both cases. Merge mode may use up to six candidates from the list, while normal mode uses only the twelfth candidate. Block vector difference (BVD) coding employs motion vector difference (MVD) processing, allowing the final BV to be arbitrarily large. The reconstructed BV may point to an area outside the reference sample area, which requires correction by removing the absolute offset in each direction using modulo arithmetic on the width and height of the RSM.

[0084] 8 illustrates an example of an intra-template matching prediction (IntraTMP) mode according to one embodiment of the present disclosure. In an embodiment, as in ECM software, IntraTMP is a special intra-prediction mode that can copy a best predicted block (821) from the reconstructed portion of the current frame (or current picture), and the template (e.g., an L-shaped template) (820) of the best predicted block (821) can match the current template (810) of the current block (811). For a given search range, the encoder can search for the template (820) that is most similar to the current template (810) in the reconstructed portion of the current frame and use the corresponding block (821) as the prediction block. The encoder can signal the use of IntraTMP mode, and the same prediction operation can be performed at the decoder side.

[0085] The prediction signal can be generated by matching a current template (810), such as an L-shaped causal neighborhood of the current block (811), with a template (e.g., (820)) of another block (e.g., (821)) within a predetermined search area. The exemplary search area shown in FIG. 8 can include multiple CTUs (or SBs). Referring to FIG. 8, the search area can include the current CTU R1 (e.g., a portion of the current CTU R1), the upper-left CTU R2, the upper CTU R3, and the left CTU R4. The cost function can include any suitable cost function, such as the sum of absolute differences (SAD).

[0086] Within each region, the decoder can search for the template (e.g., (820)) that has the smallest cost (e.g., smallest SAD) relative to the current template (810), and can use the block (e.g., (821)) associated with the template with the smallest SAD as the predicted block.

[0087] The dimensions of the region indicated by (SearchRange_w, SearchRange_h) can be set to be proportional to the block dimensions (BlkW, BlkH) to have a fixed number of SAD comparisons per pixel. For example,

number

[0088] The parameter a can be a constant that controls the tradeoff between gain and complexity. In the example, a is 5.

[0089] The intra template matching tool can be enabled for CUs of a certain size, such as sizes with width and height less than or equal to 64. The maximum CU size for IntraTMP mode is configurable.

[0090] IntraTMP mode can be signaled at the CU level via a dedicated flag, for example, if decoder-side intra mode derivation (DIMD) is not currently used for the CU.

[0091] In an example, as in ECM5, IntraTMP mode accesses 320 top samples and 320 left samples to support a 64x64 block. A memory size such as 320 top samples and 320 left samples per block can improve coding efficiency in IBC mode. The reference region or search range for IBC mode can be expanded. In an example, the reference region for IBC mode is expanded to two CTU rows above. Figure 9 shows an example of a reference region for coding CTU(m,n). The integers m and n are indices representing the position of the CTU. To code CTU(m,n), the reference region can include CTUs with indices (m-2,n-2), ..., (W-1,n-2), (0,n-1), ..., (W-1,n-1), (0,n), ..., and (m,n). Here, W indicates the maximum horizontal index of the CTU within the current tile, slice, picture, etc. The setting (e.g., predicting a block by accessing 320 top samples and 320 left samples) can ensure that the IBC mode does not require extra memory in the current test of the Essential Video Coding Model (ETM) platform when the CTU size is 128x128. The per-sample block vector search range (also called the local search range) can be limited to [-(C<<1),C>>2] (or [-2C,1 / 4C]) horizontally and [-C,C>>2] (or [-C,1 / 4C]) vertically to accommodate the reference region extension, where C represents the CTU size, such as 128. For example, the BV of a block is limited to [-2C,1 / 4C]) horizontally and [-C,1 / 4C]) vertically.

[0092] Scalable video coding can be applied to intra-pictures. In various examples, such as the AVC and HEVC scalable coding standards, scalable video coding can include one or more of temporal scalability, spatial scalability, and SNR scalability. For scalable video coding techniques, in some examples, both the base layer and the enhancement layer are required for display purposes.

[0093] Intra prediction has a limitation in some examples in that predicted samples generated from the upper-left reconstruction region have low correlation with the lower-right region of the current block. Using the disclosed methods (e.g., the embodiments described in FIGS. 10-12) can generate more accurate predicted samples. Furthermore, in some examples, blocks of a higher resolution picture (e.g., (1001)) can be generated based on blocks of a lower resolution picture (e.g., (1002)), thus transmitting fewer bits and improving coding efficiency.

[0094] The disclosed method can utilize a scaled intra-coded picture of the same content as a reference for predicting the current picture. A scaled version of the current picture or a scaled picture can represent a downsampled version of the current picture. A full-scale current picture (also called a full-scale current picture) can represent the current picture with its original resolution or the original resolution of the current picture.

[0095] The scaled version of the current picture can be compressed as part of an intra-coding framework. In a bitstream, the coded representation of the current picture has two sub-bitstreams. In one example, a sub-bitstream representing a scaled version of the current picture (e.g., the first sub-bitstream) is transmitted first, followed by a sub-bitstream representing the full-scale current picture (e.g., the second sub-bitstream).

[0096] In one embodiment, the bitstream may include a first sub-bitstream corresponding to a scaled version of the current picture and a second sub-bitstream corresponding to the full-scale current picture, as shown in Figure 10. Figure 10 illustrates a full-scale current picture (1001) and a scaled version of the current picture (1002) according to an embodiment of the present disclosure. The full-scale current picture (1001) and the scaled version of the current picture (1002) may correspond to the same current picture and may have different spatial resolutions. The scaled version (1002) may have a first spatial resolution, such as Ws x Hs samples, where Ws and Hs may be the width and height of the scaled version (1002). The full-scale current picture (1001) may have a second spatial resolution, such as Wf x Hf samples, where Ws and Hs may be the width and height of the full-scale current picture (1001). The second spatial resolution may be higher than the first spatial resolution. The original resolution of the scaled version (1002) may represent a second spatial resolution.

[0097] In this framework, a scaled version of the current picture, after decoding, is upsampled to its original resolution and can be used as a reference to predict the full-scale current picture. The upscaling (or upsampling) procedure applied to the decoded scaled version of the current picture can be specified so that it can produce the same resampled reconstructed image. In one embodiment, referring to Figure 10, the scaled version of the current picture (1002), after decoding, is upsampled to the original resolution of the full-scale current picture (1001), and can then be used as a reference or predictor to predict the full-scale current picture (1001). In another embodiment, the scaled version of the current picture (1002) is not upsampled and can be used as a reference or predictor to predict the full-scale current picture (1001).

[0098] The scaled version of the current picture may have: 1) a smaller picture width and the same picture height; 2) a smaller picture height and the same picture width; or 3) a smaller picture width and a smaller picture height. The scaled version of the current picture may have a picture width that is smaller than the picture width of the full-scale current picture and a picture height that is the same as the picture height of the full-scale current picture. The scaled version of the current picture may have a picture height that is smaller than the picture height of the full-scale current picture and a picture width that is the same as the picture width of the full-scale current picture. As shown in Figure 10, the scaled version of the current picture may have a picture width that is smaller than the picture width of the full-scale current picture and a picture height that is smaller than the picture height of the full-scale current picture.

[0099] The scaled version of the current picture and the full-scale current picture may have the same partitioning structure, so that each block in the full-scale current picture is in the same position as each block in the scaled version.

[0100] The scaled version of the current picture and the full-scale current picture may have different partitioning structures, and therefore, blocks in the full-scale current picture may not be co-located with blocks in the scaled version. Referring to Figure 10, the full-scale current picture (1001) includes a second block (1010). The second block (1010) may be co-located with a first region (1020) in the scaled version (1002) of the current picture; for example, the second block (1010) in the full-scale current picture (1001) and the first region (1020) in the scaled version (1002) correspond to the same physical region in the current picture. The second block (1010) in the full-scale current picture (1001) and the first region (1020) in the scaled version (1002) may have the same shape. The size of the first region (1020) can be scaled to the size of the second block (1010) based on, for example, Ws, Hs, Wf, and Hf.

[0101] According to an embodiment of the present disclosure, a first region (1020) in the scaled version of the current picture (1002) can be co-located with a second block (1010) in the full-scale current picture (1001). The first region (1020) can overlap one or more first blocks in the scaled version of the current picture (1002). In the example shown in FIG. 10, the first region (1020) overlaps with first blocks (1021)-(1022) in the scaled version of the current picture (1002). In the example shown in FIG. 10, the first blocks (1021)-(1022) completely overlap the first region (1020), and the first region (1020) includes the first blocks (1021)-(1022).

[0102] In some embodiments, due to different partitioning structures, one or more of the first blocks partially overlap the first region (1020), for example, one of the one or more first blocks includes samples that are outside the first region (1020).

[0103] A scaled version (1002) of the current picture from the first sub-bitstream can be reconstructed, for example, by reconstructing samples in the scaled version (1002). A second block (1010) in the full-scale current picture (1001) can be reconstructed based on one of: (i) partitioning information of one or more first blocks (1021)-(1022) in the scaled version (1002), or (ii) intra-prediction information of one or more first blocks (1021)-(1022) in the scaled version (1002).

[0104] In one embodiment, prediction regarding partitioning using the partitioning result of the scaled picture, for example, after expanding the block size as indicated by the scaling ratio, can be used as a basis for predicting the partitioning of the full-scale current picture. For example, a bitstream may include a first sub-bitstream corresponding to a scaled version of the current picture having a first spatial resolution and a second sub-bitstream corresponding to the full-scale current picture, where the full-scale has a second spatial resolution higher than the first spatial resolution. Furthermore, one or more first blocks in the scaled version of the current picture are co-located with second blocks in the full-scale current picture. An example method can reconstruct one or more first blocks in the scaled version of the current picture from the first sub-bitstream. Then, the disclosed method can reconstruct a second block of the full-scale current picture based on (i) partitioning information of the one or more first blocks in the scaled version or (ii) intra-prediction information of the one or more first blocks in the scaled version.

[0105] In one example, the scaling ratio is 2, and blocks in the full-scale current picture (e.g., 16x16 blocks) are co-located with and correspond to blocks in the scaled picture (e.g., 8x8 blocks). If an 8x8 block in the scaled picture is coded as an 8x8 block without being split, then the 16x16 block in the full-scale current picture can be inferred to be unsplit and coded as a 16x16 block. If an 8x8 block in the scaled picture is split and coded as four 4x4 blocks, then the 16x16 block in the full-scale current picture can be inferred to be split into four 8x8 blocks, and each of the four 8x8 blocks is coded separately.

[0106] Referring to FIG. 10, the partitioning information of the scaled version (1002) can be used to predict the partitioning information of the full-scale current picture (1001). The partitioning information of the first block (1021) can indicate whether the first block (1021) is to be divided into smaller blocks. The partitioning information of the first block (1022) can indicate whether the first block (1022) is to be divided into smaller blocks. Therefore, whether to partition the second block (1010) of the full-scale current picture (1001) can be determined based on the partitioning information of the first blocks (1021) to (1022). Based on the determination of whether to partition the second block (1010), the second block (1010) can be reconstructed.

[0107] In one embodiment, one flag for each block size may be signaled to indicate whether further division is required.

[0108] In one embodiment, a flag for the second block (1010) is signaled. If it is determined that the second block (1010) is to be partitioned based on the partitioning information of the first blocks (1021) to (1022), the flag may indicate whether splitting is applied to the blocks partitioned from the second block (1010). If it is determined that the second block (1010) is not to be partitioned based on the partitioning information of the first blocks (1021) to (1022), the flag may indicate whether splitting is applied to the second block (1010).

[0109] In the example, a flag for each of the first blocks (1021) to (1022) is signaled to indicate whether the first blocks (1021) to (1022) are to be further divided.

[0110] In one embodiment, the flag indicating the partitioning mode of the scaled picture can be used as a context for entropy coding of the partitioning mode of the full-scale current picture.

[0111] In an example, the partitioning information includes a flag for each of the one or more first blocks (1021)-(1022) indicating whether to divide each first block (e.g., (1021) or (1022)) into smaller blocks. The partitioning information for the second block (e.g., (1021) or (1022)) can be entropy coded (e.g., entropy encoded or entropy decoded), and the flag for each of the one or more first blocks (1021)-(1022) can be used as context for the entropy coding (e.g., entropy encoding or entropy decoding).

[0112] The reconstructed samples of the scaled picture can be used in the intra prediction process of the full-scale current picture.

[0113] The bottom row, right column, or bottom right corner of the full-scale current picture can be predicted using the same location in the reconstructed samples of the scaled picture. Along with the top and left reconstructed samples of the full-resolution picture, the remaining samples of the current block can be interpolated and predicted. Examples of such interpolation include planar mode, bilateral interpolation, etc.

[0114] In one embodiment, referring to Figure 10, a first sample (e.g., sample (1043)) in a second block (1010) is predicted based on a reconstructed sample (e.g., (1073)) in a first region (1020) of a scaled version (1002) of a current picture. The first sample in the second block (1010) may be co-located with the reconstructed sample in the first region (1020). Referring to Figure 10, sample (1043) is located in the lower right corner of the second block (1010) and is co-located with the reconstructed sample (1073) located in the lower right corner of the first region (1020). The second sample in the second block (1010) (e.g., the sample at position (1051) or the sample at position (1052)) can be predicted by interpolation based on at least the predicted first sample (1043) in the second block (1010) and the reconstructed sample in the second block (1010) (e.g., the upper-left reconstructed sample (1044)). The predictor for the second block (1010) can include (i) the predicted first sample (1043) in the second block (1010), (ii) the reconstructed sample in the second block (1010) (e.g., the upper-left reconstructed sample (1044)), and (iii) the predicted second sample in the second block (1010) (e.g., the predicted sample at position (1051) or the predicted sample at position (1052)). The second block (1010) can be reconstructed from the predictor for the second block (1010).

[0115] 10, the bottom row (1041), right column (1042), or bottom right corner (1043) of the second block (1010) can be predicted based on the reconstructed samples at the same locations in the first region (1020) of the scaled version (1002). For example, the bottom row (1041) is predicted based on the reconstructed samples in the bottom row (1071) in the first region (1020). The right column (1042) is predicted based on the reconstructed samples in the right column (1072) in the first region (1020).

[0116] The predicted bottom row (1041), predicted right column (1042), and / or predicted bottom right corner (1043) of the second block (1010) can be used, along with the above and left reconstructed samples (e.g., sample (1044)) of the second block (1010) in the full-scale current picture (1001), to predict the remaining samples of the second block (1010) by interpolation. As mentioned above, in some instances of intra prediction, a limitation exists in that predicted samples generated from the top-left reconstructed region may have low correlation with the bottom-right region of the current block, and therefore the prediction of the bottom-right region may not be accurate. The described method can use the reconstructed lower right region (e.g., (1071), (1072), and / or (1073)) of the scaled version (1002) to predict the lower right region (e.g., (1041), (1042), and / or (1043)) of the second block (1010), thereby making the predicted samples of the second block (1010) more accurate.

[0117] A regular intra-predicted block of the full-scale current picture can be blended with the upsampled reconstructed block at the same location, with prediction from the scaled picture given more weight in the bottom right corner of the current block and prediction from the full resolution picture given more weight in the top left corner of the current block.

[0118] Referring to Figure 10, samples of the second block (1010) can be used to obtain an intra-predicted block of the second block (1010) of the full-scale current picture (1001) using intra prediction. (i) Predicted samples (e.g., intra-prediction samples) of the second block (1010) obtained from intra prediction and (ii) corresponding upsampled reconstructed samples of the first region (1020) can be blended using a weighted average. The weight of the blended samples in the second block (1010) can depend on the position of the blended samples in the second block (1010). In one example, the closer the blended samples are to the bottom right corner (1043) of the second block (1010), the greater the weight given to the reconstructed samples of the first region (1020). In one example, the closer the blended samples are to the top left corner of the second block (1010), the greater the weight given to the intra-predicted samples of the second block (1010). For example, position (1051) is associated with a first weight for intra-predicted samples in the second block (1010), and position (1052) is associated with a second weight for intra-predicted samples in the second block (1010). The first weight is greater than the second weight.

[0119] The described method uses position-dependent weights, thus giving more weight to samples that are more likely to be accurate, and therefore the predicted samples in the second block (1010) may be more accurate.

[0120] Prediction of intra prediction mode (IPM) from the scaled picture to the full resolution picture can be performed for the same position. The prediction can be used as an additional candidate in a list such as a most probable mode (MPM) list. Alternatively, the prediction can be used together with IPM from spatially neighboring blocks to derive an MPM list. For chroma components, the direct mode (DM) can be either from the IPM associated with the co-located luma block from the full-scale picture or the scaled picture.

[0121] In one example, the intra prediction information includes IPM information for one or more first blocks (1021)-(1022), such as an IPM used to code the one or more first blocks (1021)-(1022). The IPM for the second block (1010) may be predicted based on the IPM information for the one or more first blocks (1021)-(1022). The MPM list for the second block (1010) can be constructed based on the IPM information for the one or more first blocks (1021)-(1022). In one example, the predicted IPM is used as an additional candidate for the MPM list for the second block (1010). In another example, the prediction can be used together with IPMs from spatially neighboring blocks of the second block (1010) to derive the MPM list. For the chroma components, the DM direct mode can be either the full-scale current picture (1001) or an IPM associated with the co-located luma block from a scaled version of the current picture (1002). The second block (1010) can be reconstructed based on the MPM list.

[0122] In Multiple Reference Lines (MRL) mode, in addition to the directly adjacent lines of the neighboring samples, one of two non-adjacent reference lines can be used as a reference line for intra-picture prediction of a luma sample. The non-adjacent reference lines may be only two or three lines away from the current block. Prediction of reference line index can be done from the co-located scaled picture to the full-resolution picture. The co-located reference line from the scaled picture can be used to predict the reference line of the current block in the full-scale picture.

[0123] In one embodiment, the intra prediction information includes reference line index information for one or more first blocks (1021)-(1022), for example, each reference line or each reference line index used for each of the one or more first blocks (1021)-(1022). Based on the reference line index information for the one or more first blocks (1021)-(1022), the reference line index or reference line of the second block can be predicted. Based on the reference line index (or reference line) of the second block (1010), the second block (1010) can be reconstructed.

[0124] The intra-skip mode is used to code the current block in the full-scale picture. In one example, when the intra-skip mode is used, all prediction modes are inherited from the prediction modes associated with the co-located samples in the scaled picture. Furthermore, when the intra-skip mode is applied, the residual samples can be assumed to be 0, and no syntax related to residual coding is signaled.

[0125] In one embodiment, intra-skip mode is used to code the second block (1010) in the full-scale current picture (1001). The prediction mode of one of the one or more first blocks (1021)-(1022) in the scaled version (1002) (e.g., (1021) or (1022)) can be inherited by the second block (1010). The second block (1010) can be reconstructed based on at least the inherited prediction mode. When intra-skip mode is applied to the second block (1010) in the full-scale current picture (1001), the residual samples of the second block (1010) can be assumed to be zero, and no syntax related to residual coding is signaled to the second block (1010).

[0126] The reconstructed samples of the scaled picture can be used as predictors in the inter-layer prediction process of the full-scale current picture. In one example, the scaled picture is in the base layer and the full-scale current picture is in the enhancement layer.

[0127] A co-located upsampled reconstructed sample in the scaled picture can be used as a predictor for a current block in the full-scale current picture. In this case, the displacement vector pointing from the current block position to the reference block position in the scaled picture is 0. As an example, referring to FIG. 10, a first region (1020) of co-located upsampled reconstructed samples can be used as a predictor for a second block (1010) in the full-scale current picture (1001). A vector (e.g., displacement vector) (1003) points from the second block (1010) to the reference block (the upsampled and reconstructed first region (1020) in the scaled version (1002)) and can be 0, for example, when the second block (1010) and the reference block are co-located.

[0128] This predicted block can be filtered before being applied as a predictor. Reference samples above and to the left of the current block in the full-scale current picture can be used to extend this predicted block.

[0129] In one embodiment, referring to Figure 10, an upsampled and reconstructed first region (1020) co-located with a second block (1010) can be filtered before being used as a predictor for the second block (1010). Reference samples (1061)-(1062) above and / or to the left of the second block (1010) can be used to enhance (e.g., filter) the upsampled and reconstructed first region (1020).

[0130] This predicted block can be combined with another predictor generated from an intra-prediction process within the full-scale current picture as described above in the blending process.

[0131] In one example, equal weighting is applied to each sample in the block.

[0132] In another example, different weightings can be applied depending on the sample position within the block, e.g., if the current position is far from the above and / or left reference samples, more weight can be assigned to the predictors from the scaled picture.

[0133] Referring to Figure 10, the first predictor is an upsampled and reconstructed first region (1020) at the same location as the second block (1010). The second predictor can be generated from the second block (1010) in the full-scale current picture (1001) using intra-prediction. The first predictor and the second predictor can be combined using a weighted average. In one example, equal weighting is applied to each sample in the second block (1010). In one example, different weighting can be applied depending on the location within the second block (1010). For example, if the current location is further away from the above and / or left reference samples, more weight can be assigned to the first predictor from the scaled version (1002).

[0134] The residual signal of the scaled picture can be used as a predictor for coding the residual signal of the full-scale current picture. In one example, the residual of samples in the second block (1010) in the full-scale current picture (1001) can be predicted based on the residual of reconstructed samples in the first region (1020) in the scaled version (1002).

[0135] The significance map, which is the distribution of non-zero residuals within a block, can have a correlation between the same location in the scaled picture and the full-scale picture. Here, the distribution of non-zero residuals within a block can be either sample-based 0 or 1 signaling or sub-block-based signaling, for example, one bit for each 4x4 sub-block is used to indicate whether there is non-zero residual within this region. Referring to Figure 10, the distribution of non-zero residuals within a first region (1020) of the scaled version (1002) can be correlated with the distribution of non-zero residuals within a second block (1010) in the full-scale current picture (1001). Here, the second block (1010) is at the same location as the first region (1020).

[0136] In one embodiment, the significance map (or distribution of non-zero residuals in the block) of the block at the same location in the scaled picture can be used to predict whether the block at the same location in the full-scale current picture has a non-zero residual. As an example, whether a sample in the second block (1010) has a non-zero residual is predicted based on the distribution of non-zero residuals in the first region (1020).

[0137] In another embodiment, the significance map (distribution of non-zero residuals in the block) of the co-located block in the scaled picture can be used as a context for arithmetically coding the significance map of the co-located block in the full-scale current picture. In the example, the distribution of non-zero residuals in the second block (1010) is arithmetically decoded. The distribution of non-zero residuals in the first region (1020) can be used as a context for arithmetic decoding.

[0138] In the above method, the scaled picture may not be required to code all blocks / CTUs in the full-scale current picture. If some blocks in the scaled picture are not used for prediction, their reconstructed values ​​do not provide useful information, and these blocks can be skipped or processed coarsely during coding of the scaled picture. In one example, unused blocks are coded using constant sample values. In another example, syntax can be designed at the block or CTU level to signal that this block / CTU does not have any content and is skipped. In the embodiment described in the disclosure, the scaled version (1002) cannot be used to code all blocks and / or all CTUs in the full-scale current picture (1001). If a block (1030) in the scaled version (1002) is not used for prediction, the reconstructed samples in the block (1030) may not provide useful information. Therefore, the block (1030) can be skipped or processed coarsely during coding of the scaled version (1002). In one example, for blocks in the scaled version (1002) that are not used (including, for example, block (1030)), the blocks are coded using constant sample values. In another example, syntax can be designed at the block level or CTU level to signal that the corresponding block or CTU in the scaled version (1002) does not have any content and is skipped.

[0139] In one example, the first sub-bitstream includes the entire scaled version (1002), and each block in the scaled version (1002) is coded and included in the first sub-bitstream. In one example, the first sub-bitstream includes a portion of the scaled version (1002), and a first block in the scaled version (1002) is coded and included in the first sub-bitstream, and a second block in the scaled version (1002) is not coded and not included in the first sub-bitstream. In one example, the second block is processed coarsely (e.g., using constant sample values).

[0140] In one example, the second sub-bitstream includes the entire full-scale current picture (1001), and each block of the entire full-scale current picture (1001) is coded and included in the second sub-bitstream. In one example, the second sub-bitstream includes a portion of the full-scale current picture (1001), and a first sample of the full-scale current picture (1001) is coded and included in the second sub-bitstream, and a sample of the full-scale current picture (1001) is not coded and not included in the second sub-bitstream.

[0141] 11 shows a flowchart outlining a process (1100) according to one embodiment of the present disclosure. The process (1100) can be used in a video decoder. In various embodiments, the process (1100) is performed by a processing circuit, such as a processing circuit that performs the functions of the video decoder (110), a processing circuit that performs the functions of the video decoder (210), etc. In some embodiments, the process (1100) is implemented with software instructions, and thus the processing circuit performs the process (1100) when it executes the software instructions. The process (1100) begins at (S1101) and proceeds to (S1110).

[0142] At (S1110), a bitstream is received, which may include a first sub-bitstream corresponding to a scaled version of a current picture having a first spatial resolution and a second sub-bitstream corresponding to the current picture at full scale, the full scale having a second spatial resolution higher than the first spatial resolution.

[0143] At (S1120), a scaled version of the current picture can be reconstructed from the first sub-bitstream.

[0144] At (S1130), a second block in the full-scale current picture can be reconstructed based on (i) partitioning information of one or more first blocks in the scaled version of the current picture, or (ii) intra-prediction information of one or more first blocks in the scaled version.

[0145] In the example, as illustrated in FIG. 10, a first region in the scaled version of the current picture is co-located with a second block in the full-scale current picture.

[0146] In the example, a first sample in a second block can be predicted based on a reconstructed sample in a first region in a scaled version of the current picture. The first sample in the second block is co-located with the reconstructed sample in the first region. The second sample in the second block can be predicted by interpolation based on at least the predicted first sample in the second block and the reconstructed sample to the top left of the second block. The predictor for the second block includes (i) the predicted first sample in the second block, (ii) the reconstructed sample to the top left of the second block, and (iii) the predicted second sample in the second block. The second block is reconstructed from the predictor for the second block.

[0147] In an example, samples in the second block are predicted using intra prediction. (i) The predicted samples in the second block and (ii) the corresponding upsampled reconstructed samples in the first region in the scaled version of the current picture can be blended using a weighted average. The weight of the blended samples in the second block depends on the position of the blended samples within the second block.

[0148] In an example, reconstructed samples in a first region of a scaled version of a current picture are upsampled, the upsampled samples in the first region of the scaled version are filtered, and a second block can be reconstructed with the filtered upsampled samples in the first region as a predictor of the second block.

[0149] In an example, residuals of samples in the second block can be predicted based on residuals of reconstructed samples in the first region. Whether samples in the second block have non-zero residuals can be predicted based on the distribution of non-zero residuals in the first region. In an example, the distribution of non-zero residuals in the second block is arithmetically decoded, and the distribution of non-zero residuals in the first region is used as a context for the arithmetic decoding.

[0150] Next, the process proceeds to (S1199) and ends.

[0151] The process 1100 may be adapted as appropriate. Steps of the process 1100 may be modified and / or omitted. Additional steps may be added. Any suitable order of implementation may be used.

[0152] In an embodiment, the partitioning information of the one or more first blocks indicates whether each of the one or more first blocks is to be divided into smaller blocks. The second block can be reconstructed based on the partitioning information of the one or more first blocks by determining whether to partition a second block in the full-scale current picture based on the partitioning information of the one or more first blocks, and reconstructing the second block based on the determination of whether to partition the second block.

[0153] In an example, the second sub-bitstream includes a flag for the second block. In response to a decision to partition the second block, the flag indicates whether to apply partitioning to blocks partitioned from the second block. In response to a decision to partition the second block, the flag indicates whether to apply partitioning to the second block.

[0154] In an example, the partitioning information includes a flag for each of the one or more first blocks indicating whether to divide each first block into smaller blocks, and the partitioning information for the second block can be entropy decoded, and the flag for each of the one or more first blocks can be used as context for the entropy decoding.

[0155] In an example, the intra prediction information includes intra prediction mode (IPM) information for one or more first blocks. A most probable mode (MPM) list for a second block can be constructed based on the IPM information for one or more first blocks. The second block can be reconstructed based on the MPM list.

[0156] In an example, the intra prediction information includes reference line index information of one or more first blocks. The reference line index of the second block can be determined based on the reference line index information of the one or more first blocks, and the second block can be reconstructed based on the reference line index of the second block.

[0157] In the example, the second sub-bitstream indicates that intra-skip mode is used for the second block. The intra-prediction information indicates a prediction mode of one of the one or more first blocks. The prediction mode of the one or more first blocks can be used for the second block, and the second block can be reconstructed based on the prediction mode.

[0158] 12 shows a flowchart outlining a process (1200) according to one embodiment of the present disclosure. The process (1200) can be used in a video encoder. In various embodiments, the process (1200) is performed by a processing circuit, such as a processing circuit performing the functions of the video encoder (103), a processing circuit performing the functions of the video encoder (303), etc. In some embodiments, the process (1200) is implemented by software instructions, and thus the processing circuit performs the process (1200) when it executes the software instructions. The process begins at (S1201) and proceeds to (S1210).

[0159] At (S1210), one or more first blocks in a scaled version of the current picture having a first spatial resolution are encoded, where a first region in the scaled version overlaps with the one or more first blocks.

[0160] (S1220) may encode a second block in the full-scale current picture based on (i) partitioning information of one or more first blocks in the scaled version or (ii) intra-prediction information of one or more first blocks in the scaled version. The full-scale current picture has a second spatial resolution higher than the first spatial resolution. The first region in the scaled version is co-located with the second block in the full-scale current picture.

[0161] In (S1230), the bitstream may include an encoded scaled version including one or more encoded first blocks and a full-scale encoded current picture including one or more encoded second blocks.

[0162] Next, the process proceeds to (S1299) and ends.

[0163] Process 1200 may be adapted as appropriate. Steps of process 1200 may be modified and / or omitted. Additional steps may be added. Any suitable order of implementation may be used. The embodiments described in Figures 10-11 may be adapted for use with process 1200 as appropriate.

[0164] The embodiments of the present disclosure may be used separately or combined in any order. Furthermore, each of the methods (or embodiments), encoders, and decoders may be implemented by processing circuitry (e.g., one or more processors, or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium.

[0165] The techniques described above can be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 13 illustrates a computer system (1300) suitable for implementing certain embodiments of the subject matter of this disclosure.

[0166] Computer software can be coded using any suitable machine code or computer language that can be processed by mechanisms such as assembly, compilation, linking, etc. to generate code containing instructions that can be executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., directly or through interpretation, microcode execution, etc.

[0167] The instructions may be executed by a variety of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, and the like.

[0168] 13 of the computer system (1300) are exemplary in nature and do not suggest any limitation on the scope of use or functionality of the computer software implementing the embodiments of the present disclosure. Furthermore, the arrangement of components should not be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of the computer system (1300).

[0169] The computer system (1300) may include certain human interface input devices. Such human interface input devices may be responsive to input by one or more human users, for example, through sensory input (e.g., keystrokes, swipes, data grabbing actions), audio input (e.g., voice, clapping), visual input (e.g., gestures), or olfactory input (not shown). Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a digital camera), and video (including, for example, two-dimensional video, three-dimensional video, and stereoscopic video).

[0170] The input human interface devices may include one or more of a keyboard (1301), a mouse (1302), a trackpad (1303), a touchscreen (1310), a data grab (not shown), a joystick (1305), a microphone (1306), a scanner (1307), and a camera (1308) (only one of which is shown).

[0171] The computer system (1300) may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses through, for example, sensory output, sound, light, and smell / taste. Such human interface output devices may include sensory output devices (e.g., sensory feedback via a touchscreen (1310), a data grab (not shown), or a joystick (1305; however, sensory feedback devices that do not function as input devices may also exist), audio output devices (e.g., speakers (1309), headphones (not shown)), and visual output devices (e.g., a screen (1310), including a CRT screen, an LCD screen, a plasma screen, and an OLED screen, each with or without touchscreen input capability and each with or without sensory feedback capability, some of which may be capable of outputting two-dimensional visual output or three-dimensional or higher-dimensional output through means such as stereoscopic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown), and printers (not shown)).

[0172] The computer system (1300) may also include human-accessible storage devices and associated media such as optical media including CD / DVD ROM / RW (1320) with media such as CD / DVD (1321), thumb drives (1322), removable hard drives or solid state drives (1323), legacy magnetic media such as tape and floppy disks (not shown), dedicated ROM / ASIC / PLD based devices such as security dongles (not shown), etc.

[0173] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter of this disclosure does not encompass transmission media, carrier waves, or other transitory signals.

[0174] The computer system 1300 may also include interfaces 1354 to one or more communications networks 1355. Networks may be, for example, wireless, wired, or optical. Networks may further be local, wide-area, metropolitan, vehicular, and industrial, real-time, latency-tolerant, and the like. Examples of networks include local area networks such as Ethernet; cellular networks, including WLAN, GSM, 3G, 4G, 5G, LTE, and the like; TV wired or wireless wide-area digital networks, including cable TV, satellite TV, and terrestrial broadcast TV; and vehicular and industrial networks, including CAN Bus. Particular networks generally require external network interfaces that are attached to particular general-purpose data ports or peripheral buses 1349 (e.g., USB ports on the computer system 1300). Others are generally integrated into the core of the computer system 1300 by attachment to a system bus, as described below (e.g., an Ethernet interface to a PC computer system, or a cellular network interface to a smartphone computer system). Using these networks, the computer system 1300 can communicate with other entities. Such communication can be one-way receive only (e.g., broadcast TV), one-way transmit only (e.g., CANbus to a specific CANbus device), or two-way to other computer systems, for example, using local or wide-area digital networks. Specific protocols and protocol stacks can be used with each of the above networks and network interfaces.

[0175] The aforementioned human interface devices, human accessible storage devices, and network interfaces may be attached to the core (1340) of the computer system (1300).

[0176] The core (1340) may include one or more central processing units (CPUs) (1341), graphics processing units (GPUs) (1342), dedicated programmable processing units (1343) in the form of FPGAs, task-specific hardware accelerators (1344), graphics adapters (1350), etc. These devices may be connected through a system bus (1348), along with read-only memory (ROM) (1345), random access memory (1346), and internal mass storage devices (1347) such as internal non-user-accessible hard drives, SSDs, etc. In some computer systems, the system bus (1348) is accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices can be attached directly to the core's system bus (1348) or through a peripheral bus (1349). In an example, a screen (1310) can be connected to the graphics adapter (1350). Peripheral bus architectures include PCI, USB, and the like.

[0177] The CPU (1341), GPU (1342), FPGA (1343), and accelerator (1344) can execute specific instructions that can be combined to generate the aforementioned computer code. The computer code can be stored in ROM (1345) or RAM (1346). Temporary data can also be stored in RAM (1346), while permanent data can be stored, for example, in an internal mass storage device (1347). Rapid storage and retrieval from any of the memory devices can be enabled through the use of cache memory, which can be closely associated with one or more of the CPU (1341), GPU (1342), mass storage device (1347), ROM (1345), RAM (1346), etc.

[0178] The computer-readable medium may bear computer code for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0179] As an example and not by way of limitation, the computer system (1300) having the architecture, and specifically the core (1340), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be specific storage of the core (1340) of a non-transitory nature, such as the core's internal mass storage (1347) or ROM (1345), as well as media associated with user-accessible mass storage devices such as those described above. Software implementing various embodiments of the present disclosure can be stored on such devices and executed by the core (1340). The computer-readable media can include one or more memory devices or chips, depending on the particular needs. The software can cause the core (1340) and specifically the processor (including a CPU, GPU, FPGA, etc.) therein to perform specific processes or portions of specific processes described herein, including defining and modifying data structures stored in RAM (1346) according to software-defined operations. Additionally or alternatively, a computer system may provide functionality as a result of implementation in hardwired logic or other circuitry (e.g., accelerator (1344)) that can operate in conjunction with or in place of software to perform particular processes or portions of particular processes described herein. References to software include logic, and vice versa, where appropriate. References to computer-readable media may include, where appropriate, circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry that implements logic for execution, or both. The present disclosure includes any appropriate combination of hardware and software.

[0180] The use of "at least one" or "one" in this disclosure is intended to include any one or combination of the listed elements. For example, reference to at least one of A, B, or C; at least one of A, B, and C; at least one of A, B, and / or C; and at least one of A through C is intended to include A only, B only, C only, or any combination thereof. Reference to one of A or B, and one of A and B is intended to include A or B or (A and B). The use of "one" does not exclude any combination of the listed elements, where applicable, such as when the elements are not mutually exclusive.

[0181] While this disclosure has described several exemplary embodiments, alterations, permutations, and various substitute equivalents exist, and are encompassed within the scope of this disclosure. Those skilled in the art will appreciate that numerous systems and methods can be devised that, although not explicitly shown or described herein, embody the principles of the present disclosure and therefore are within the spirit and scope of the present disclosure.

Claims

1. 1. A method for video decoding, comprising: receiving a bitstream including a first sub-bitstream corresponding to a scaled version of a current picture having a first spatial resolution and a second sub-bitstream corresponding to a full-scale current picture having a second spatial resolution higher than the first spatial resolution; reconstructing the scaled version of the current picture from the first sub-bitstream; reconstructing second blocks of the full-scale current picture based on (i) partitioning information of one or more first blocks in the scaled version of the current picture, or (ii) intra-prediction information of the one or more first blocks in the scaled version; A method comprising:

2. the partitioning information of the one or more first blocks indicates whether each of the one or more first blocks is divided into smaller blocks; The step of reconstructing the second block includes: determining whether to partition the second block in the full-scale current picture based on the partitioning information of the one or more first blocks; reconstructing the second block based on the determination of whether to partition the second block.

2. The method of claim 1, further comprising: reconstructing the second block based on the partitioning information of the one or more first blocks by:

3. the second sub-bitstream includes a flag for the second block; In response to a decision to partition the second block, the flag indicates whether division is applied to a block partitioned from the second block; In response to determining not to partition the second block, the flag indicates whether partitioning is applied to the second block. The method of claim 2.

4. the partitioning information includes, for each of the one or more first blocks, a flag indicating whether each of the first blocks is divided into smaller blocks; 3. The method of claim 2, further comprising entropy decoding partitioning information of the second block, wherein the flag for each of the one or more first blocks is used as a context for the entropy decoding.

5. the intra prediction information includes intra prediction mode (IPM) information of the one or more first blocks; The step of reconstructing the second block includes: constructing a most probable mode (MPM) list for the second block based on the IPM information of the one or more first blocks; reconstructing the second block based on the MPM list; The method of claim 1 , comprising:

6. the intra prediction information includes reference line index information of the one or more first blocks; The step of reconstructing the second block includes: determining a reference line index for the second block based on the reference line index information of the one or more first blocks; reconstructing the second block based on the reference line index of the second block; The method of claim 1 , comprising:

7. the second sub-bitstream indicates that an intra-skip mode is used for the second block; the intra prediction information indicates a prediction mode of one of the one or more first blocks; 2. The method of claim 1, wherein reconstructing the second block comprises: using a prediction mode of one of the one or more first blocks for the second block; and reconstructing the second block based on the prediction mode.

8. 2. The method of claim 1, wherein a first region in the scaled version of the current picture is co-located with the second block in the full-scale current picture, and the first region overlaps with the one or more first blocks in the scaled version of the current picture.

9. The step of reconstructing the second block includes: predicting a first sample in the second block based on a reconstructed sample in the first region in the scaled version of the current picture, the first sample in the second block being co-located with the reconstructed sample in the first region; predicting a second sample in the second block by interpolation based on at least the predicted first sample in the second block and an upper-left reconstructed sample in the second block, wherein a predictor for the second block includes (i) the predicted first sample in the second block, (ii) the upper-left reconstructed sample in the second block, and (iii) the predicted second sample in the second block, and the second block is reconstructed from the predictor for the second block; The method of claim 8 further comprising:

10. The step of reconstructing the second block includes: predicting samples within the second block using intra prediction; blending (i) the predicted samples in the second block with (ii) the corresponding upsampled reconstructed samples in the first region in the scaled version of the current picture using a weighted average; The method of claim 8, comprising:

11. The method of claim 10 , wherein a weight of a blended sample in the second block depends on a position of the blended sample within the second block.

12. The step of reconstructing the second block includes: upsampling reconstructed samples within the first region of the scaled version of the current picture; filtering the upsampled samples in the first domain of the scaled version; reconstructing the second block using the filtered upsampled samples in the first domain as a predictor of the second block; The method of claim 8, comprising:

13. The step of reconstructing the second block includes:

9. The method of claim 8, further comprising predicting residuals of samples in the second block based on residuals of reconstructed samples in the first region.

14. The step of predicting the residual includes:

14. The method of claim 13, comprising predicting whether samples in the second block have non-zero residuals based on a distribution of non-zero residuals in the first region.

15. The step of predicting the residual includes:

14. The method of claim 13, comprising arithmetically decoding a distribution of non-zero residuals in the second block, wherein the distribution of non-zero residuals in the first region is used as a context for the arithmetic decoding.

16. The method of claim 8 , wherein the first region includes the one or more first blocks.

17. 1. An apparatus for video decoding, comprising: An apparatus comprising processing circuitry configured to carry out the method of any one of claims 1 to 16.

18. 1. A method for video encoding, comprising: including in the bitstream a first sub-bitstream corresponding to a scaled version of a current picture having a first spatial resolution and a second sub-bitstream corresponding to a full-scale current picture having a second spatial resolution higher than the first spatial resolution; A method in which a second block of the full-scale current picture is encoded based on (i) partitioning information of one or more first blocks in the scaled version of the current picture, or (ii) intra-prediction information of the one or more first blocks in the scaled version.

19. 1. A method for video encoding, comprising: transmitting a bitstream including a first sub-bitstream corresponding to a scaled version of a current picture having a first spatial resolution and a second sub-bitstream corresponding to a full-scale current picture having a second spatial resolution higher than the first spatial resolution; A method in which a second block of the full-scale current picture is encoded based on (i) partitioning information of one or more first blocks in the scaled version of the current picture, or (ii) intra-prediction information of the one or more first blocks in the scaled version.

Images