METHODS AND APPARATUS FOR RESAMPLING REFERENCE IMAGES
Patent Information
- Authority / Receiving Office
- MX · MX
- Patent Type
- Patents
- Current Assignee / Owner
- BEIJING DAJIA INTERNET INFORMATION TECH CO LTD
- Filing Date
- 2022-06-24
- Publication Date
- 2026-06-12
Smart Images
Figure MX434825B0
Abstract
Description
METHODS AND APPARATUS FOR RESAMPLING REFERENCE IMAGES CROSS-REFERENCE TO RELATED APPLICATION This application is based on and claims priority from Provisional Application No. 62 / 953,471 filed on December 24, 2019, and its full content is incorporated herein by reference in its entirety. TECHNICAL FIELD This description relates to video encoding and compression. More specifically, this description relates to methods and equipment in reference image resampling technology for video encoding. BACKGROUND OF THE INVENTION Several video coding techniques can be used to compress video data. Video coding is performed according to one or more video coding standards. Examples of video coding standards include Versatile Video Coding (WC), Joint Scan Test Model (JEM), High Efficiency Video Coding (H.265 / HEVC), Advanced Video Coding (H.264 / AVC), Moving Picture Experts Group (MPEG) coding, and similar standards. Video coding typically uses predictive methods (e.g., interprediction, intraprediction, or similar) that take advantage of redundancy present in the images or video sequences. A key goal of video coding techniques is to compress video data in a way that uses a lower bit rate while avoiding or minimizing video quality degradation. BRIEF DESCRIPTION OF THE INVENTION Examples in this description provide reference image resampling methods and apparatus. In accordance with the first aspect of this description, a method for decoding a video signal is provided. The method may include a decoder that obtains a reference image 1 associated with a video block within the video signal. The decoder may also obtain reference samples Ai, ij of the video block from a reference block in the reference image. The i and j may represent a coordinate of a sample within the video block. The decoder may further obtain a first downsampling filter and a second downsampling filter to generate brightness and color interprediction samples, respectively, of the video block when the video block is encoded in a non-affine mode and the resolution of the reference image l is greater than that of a current image.The decoder can also obtain a third downsampling filter and a fourth downsampling filter to generate brightness and color interprediction samples of the video block, respectively, when the video block is affine-encoded and the reference image resolution is higher than that of the current image. The decoder can also obtain video block interprediction samples based on the third and fourth downsampling filters being applied to the reference samples Zú.yl. Pursuant to a second aspect of this description, a computing device is provided. The computing device includes one or more processors and a non-transient, computer-readable memory that stores instructions executable by the one or more processors. The one or more processors can be configured to obtain a reference image 7 associated with a video block within the video signal. The one or more processors can also be configured to obtain reference samples (i, i, j) of the video block from a reference block in the reference image. The i and j can represent a coordinate of a sample within the video block.The one or more processors can also be configured to obtain a first downsampling filter and a second downsampling filter to generate brightness and color interprediction samples of the video block, respectively, when the video block is encoded in a non-affine mode and the reference image resolution is higher than that of the current image. The one or more processors can also be configured to obtain a third downsampling filter and a fourth downsampling filter to generate brightness and color interprediction samples of the video block, respectively, when the video block is encoded in a non-affine mode and the reference image resolution is higher than that of the current image.The one or more processors can also be configured to obtain the video block interprediction samples based on the third and fourth downsampling filters being applied to the reference samples / ii j J. According to a third aspect of this description, a non-transient, computer-readable storage medium is provided that has instructions stored therein. When the instructions are executed by one or more processors of the device, the instructions can cause the device to obtain a reference image 1 associated with a video block within the video signal. The instructions can also cause the device to obtain reference samples h:u of the video block from a block of Λ. The iyj can represent a coordinate of a sample with the video block.The instructions may also cause the device to obtain a first downsampling filter and a second downsampling filter to generate brightness and color interprediction samples, respectively, of the video block when the video block is encoded in a non-affine mode and the reference image resolution is greater than that of the current image. The instructions may also cause the device to obtain a third downsampling filter and a fourth downsampling filter to generate brightness and color interprediction samples, respectively, of the video block when the video block is encoded in an affine mode and the reference image resolution is greater than that of the current image. The instructions may cause the device to obtain interprediction samples of the video block based on the third and fourth downsampling filters being applied to the reference samples. It should be understood that both the above general description and the following detailed description are only examples and are not restrictive of the present description. BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated herein and form a part of this specification, illustrate examples consistent with the present description and, together with the description, serve to explain the principles of the description. FIG. 1 is a block diagram of an encoder, according to an example in the present description. FIG. 2 is a block diagram of a decoder, according to an example in the present description. FIG. 3A is a diagram illustrating block divisions in a multi-type tree structure, according to an example in the present description. FIG. 3B is a diagram illustrating block divisions in a multi-type tree structure, according to an example in the present description. FIG. 3C is a diagram illustrating block divisions in a multi-type tree structure, according to an example in the present description. FIG. 3D is a diagram illustrating block divisions in a multi-type tree structure, according to an example in the present description. FIG. 3E is a diagram illustrating block divisions in a multi-type tree structure, according to an example in the present description. FIG. 4A is a diagrammatic illustration of a 4-parameter affine model according to an example in the present description. FIG. 4B is a diagrammatic illustration of a 4-parameter affine model, according to an example in the present description. FIG. 5 is a diagrammatic illustration of a 6-parameter affine model according to an example in the present description. FIG. 6 is a diagrammatic illustration of an adaptive bit depth switch according to an example in the present description. FIG. 7 is a method for decoding a video signal, according to an example in the present description. FIG. 8 is a method for decoding a video signal, according to the example in this description FIG. 9 is a diagram illustrating a computer environment coupled with a user interface, as per an example in the present description. DETAILED DESCRIPTION OF THE INVENTION Reference will now be made in detail to examples of embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings in which the same numbers in different drawings represent identical or similar elements unless otherwise stated. The implementations set forth in the following description of example embodiments do not represent all implementations consistent with the description. Instead, they are merely examples of apparatus and methods consistent with aspects related to the description as listed in the accompanying claims. The terminology used herein is intended to describe particular modalities only and is not intended to limit the scope of this description. As used herein and in the appended claims, the singular forms a, an, and the are intended to include the plural forms as well, unless the context clearly indicates otherwise. The term and / or as used herein shall also be understood to mean and include any or all possible combinations of one or more of the associated enumerated elements. It is understood that, although the terms first, second, third, etc., may be used herein to describe diverse information, the information should not be limited by these terms. These terms are used only to distinguish one category of information from another. For example, without departing from the scope of this description, first information may be referred to as second information; and similarly, second information may also be referred to as first information. As used herein, the term "if" may be understood to mean "when," "about," or "in response to a judgment," depending on the context. The first version of the HEVC standard was finalized in October 2013, offering approximately 50% savings in bit rate or equivalent perceptual quality compared to the previous generation H.264 / MPEG AVC video coding standard. Although the HEVC standard provides significant coding improvements over its predecessor, there is evidence that superior coding efficiency can be achieved with additional coding tools on top of HEVC. Based on this, both VECG and MPEG began exploring new coding technologies for future video coding standardization. A Joint Video Exploration Team (JVET) was formed in October 2015 by ITU-T VECG and ISO / IEC MPEG to initiate a significant study of advanced technologies that could enable substantial improvements in coding efficiency.A reference software called the Joint Exploration Model (JEM) was maintained by the JVET by integrating many additional coding tools on top of the HEVC test model (HM). In October 2017, the board issued a call for proposals (CfPs) from ITU-T and ISO / IEC for video compression with capabilities beyond HEVC. In April 2018, 23 CfP responses were received and evaluated at the 10th JVET meeting, demonstrating a compression efficiency gain over HEVC of approximately 40%. Based on these evaluation results, JVET launched a new project to develop the next-generation video coding standard, named Versatile Video Coding (WC). That same month, a reference software codebase, called the WC Test Model (VTM), was established to demonstrate a reference implementation of the WC standard. Like HEVC, WC is built on the block-based hybrid video coding framework. Figure 1 shows a general diagram of a block-based video encoder for the WC. Specifically, Figure 1 shows a typical encoder 100. The encoder 100 has video input 110, motion compensation 112, motion estimation 114, intra / inter mode decision 116, block predictor 140, adder 128, transform 130, quantization 132, prediction-related information 142, intraprediction 118, image buffer 120, inverse quantization 134, inverse transform 136, adder 126, memory 124, loop filter 122, entropy coding 138, and bitstream 144. In the 100 encoder, a video frame is divided into a plurality of video blocks for processing. For each given video block, a prediction is formed based on either an interprediction or an intraprediction approach. A prediction residual, representing the difference between a current video block, video input part 110, and its predictor, block predictor part 140, is sent to a transform 130 from the adder 128. The transform coefficients are then sent from the transform 130 to a quantization 132 for entropy reduction. The quantized coefficients are then fed to an entropy coding 138 to generate a compressed video bitstream. As shown in FIG. 1, information 142 related to the prediction of an intra / inter mode decision 116, such as video block splitting information, motion vectors (MVs), reference frame index, and intraprediction mode, is also fed through the entropy coding 138 and stored in a compressed bitstream 144. The compressed bitstream 144 includes a video bitstream. In the encoder 100, decoder-related circuitry is also required to reconstruct pixels for prediction purposes. First, a prediction residue is reconstructed using Inverse Quantization 134 and Inverse Transform 136. This reconstructed prediction residue is then combined with a Block Predictor 140 to generate unfiltered reconstructed pixels for a current video block. Spatial prediction (or intra prediction) uses pixels from samples of neighboring blocks already encoded (which are called reference samples) in the same video frame as the current video block to predict the current video block. Temporal prediction (also referred to as inter-prediction) uses reconstructed pixels from previously encoded video images to predict the current video block. Temporal prediction reduces the inherent temporal redundancy in the video signal. The temporal prediction signal for a given encoding unit (CU) or encoding block is usually signaled by one or more MVs, which indicate the amount and direction of movement between the current CU and its temporal reference. Additionally, if multiple reference images are supported, a reference image index is also sent, which is used to identify which reference image in the reference image store the temporal prediction signal originates from. Motion estimation 114 takes video input 110 and a signal from the picture buffer 120 and outputs a motion estimation signal to motion compensation 112. Motion compensation 112 takes video input 110, a signal from the picture buffer 120, and a motion estimation signal from motion estimation 114 and outputs a motion compensation signal to intra / inter mode decision 116. After the spatial and / or temporal prediction is executed, an intra / inter mode decision 116 in the encoder 100 chooses the best prediction mode, for example, based on the distortion rate optimization method. The block predictor 140 is then subtracted from the current video block, and the resulting prediction residual is decorrelated using the transform 130 and quantization 132. The resulting quantized residual coefficients are inversely quantized by inverse quantization 134 and inversely transformed by the inverse transform 136 to form the reconstructed residual, which is then added back to the prediction block to form the reconstructed CU signal.Additional loop filtering 122, such as unblocking filtering, adaptive sample shifting (SAO), and / or adaptive loop filtering (ALF), can be applied to the reconstructed CU before it is placed in the image buffer reference image store 120 and used to encode future video blocks. To form the output video bitstream 144, all encoding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are sent to the entropy encoding unit 138 for further compression and packing to form the bitstream. Figure 1 provides the block diagram of a hybrid video coding system based on generic blocks. The input video signal is processed block by block (called CUs). In WC, a CU can be up to 128x128 pixels. However, unlike HEVC, which divides blocks only based on quaternary trees, in WC, a coding tree unit (CTU) is divided into CUs to accommodate varying local characteristics based on the quaternary / binary / ternary tree structure. Furthermore, the concept of multiple division unit types found in HEVC is eliminated; that is, the separation of CU, prediction unit (PU), and transform unit (TU) no longer exists in WC. Instead, each CU is always used as the basic unit for both prediction and transform without further division. In the multiple-type tree structure, a CTU is first divided by a quaternary tree structure.Then, each leaf node of the quaternary tree can be further divided by a binary and ternary tree structure. As shown in FIG. 3A, 3B, 3C, 3D and 3E, there are five types of division: quaternary division, horizontal binary division, vertical binary division, horizontal ternary division, and vertical ternary division. FIG. 3A shows a diagram illustrating the quaternary block divisions in a multi-type tree structure, according to the present description. FIG. 3B shows a diagram illustrating the vertical binary block splitting in a multi-type tree structure, according to the present description. FIG. 3C shows a diagram illustrating the horizontal binary block splitting in a multi-type tree structure, according to the present description. FIG. 3D shows a diagram illustrating the vertical ternary block division in a multi-type tree structure, according to the present description. FIG. 3E shows a diagram illustrating the horizontal ternary block division in a multi-type tree structure, according to the present description. In FIG. 1, spatial and / or temporal prediction can be performed. Spatial prediction (or intra-prediction) uses pixels from previously encoded neighboring block samples (called reference samples) in the same image / portion of video to predict the current video block. Spatial prediction reduces the inherent spatial redundancy in the video signal. Temporal prediction (also referred to as inter-prediction or motion-compensated prediction) uses reconstructed pixels from previously encoded video images to predict the current video block. Temporal prediction reduces the inherent temporal redundancy in the video signal. The temporal prediction signal for a given CU is typically signaled by one or more motion vectors (MVs), which indicate the amount and direction of motion between the current CU and its temporal reference.Also, if multiple reference images are supported, a reference image index is additionally sent, which is used to identify which reference image in the reference image store the temporal prediction signal originates from. After spatial and / or temporal prediction, the mode decision block in the encoder chooses the best prediction mode, for example, based on the distortion rate optimization method. The prediction block is then subtracted from the current video block; and the residual prediction is decorrelated using a transform and quantized. The quantized residual coefficients are inversely quantized and inversely transformed to form the reconstructed residual, which is then added back to the prediction block to form the reconstructed CU signal.Additionally, loop filtering, such as unblocking, SAO, and ALF filters, can be applied to the rebuilt CU before it is placed on the reference image and used to encode future video blocks. To form the output video bitstream, the encoding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to the entropy encoding unit for further compression and packaging to form the bitstream. Figure 2 shows a general block diagram of a video decoder for the WC. Specifically, Figure 2 shows a typical decoder block diagram 200. The decoder 200 has bit stream 210, entropy decoding 212, inverse quantization 214, inverse transform 216, adder 218, intra / inter mode selection 220, intra prediction 222, memory 230, loop filter 228, motion compensation 224, picture buffer 226, prediction-related information 234, and video output 232. Decoder 200 is similar to the reconstruction-related section residing in encoder 100 of FIG. 1. In decoder 200, an incoming video bitstream 210 is first decoded via an Entropy Decoder 212 to derive quantized coefficient levels and prediction-related information. The quantized coefficient levels are then processed via an Inverse Quantization 214 and an Inverse Transform 216 to obtain a reconstructed prediction residual. A block predictor mechanism, implemented in an Intra / Inter Mode Selector 220, is configured to perform either an Intra Prediction 222 or a Motion Compensation 224, based on the decoded prediction information.A set of unfiltered reconstructed pixels is obtained by adding the reconstructed prediction residual from Inverse Transform 216 and a predictive output generated by the block predictor mechanism, using an adder 218. The reconstructed block can also pass through a Loop Filter 228 before being stored in an Image Buffer 226, which functions as a reference image store. The reconstructed video in Image Buffer 226 can be sent to drive a display device, as well as used to predict future video blocks. In situations where a Loop Filter is activated, a filtering operation is performed on these reconstructed pixels to derive the final reconstructed Video Output 232. Figure 2 shows a general block diagram of a block-based video decoder. The video bitstream is first entropy-decoded in an entropy decoding unit. The encoding mode and prediction information are sent to the spatial prediction unit (if intracoded) or the temporal prediction unit (if intercoded) to form the prediction block. Residual transform coefficients are sent to the inverse quantization unit and the inverse transform unit to reconstruct the residual block. The prediction block and the residual block are then combined. The reconstructed block may undergo further loop filtering before being stored in the reference image storage. The reconstructed video in the reference image storage is then sent to control a display device and is also used to predict future video blocks. The focus of this description is to improve and simplify the existing reference image resampling design supported in the WC. It then briefly reviews current encoding tools in the WC that are closely related to the technologies proposed in this description. Affinity Mode In HEVC, only the translational motion model is used for motion-compensated prediction. However, in the real world, there are many types of motion, such as inward / outward movement, rotation, perspective movements, and other irregular motion. In WC, affine motion-compensated prediction is applied by setting a flag for each intercoding block to indicate whether the translational or affine motion model is used for interprediction. In the current WC design, two affine modes—a 4-parameter affine mode and a 6-parameter affine mode—are supported for each affine coding block. The 4-parameter affine model has the following parameters: two parameters for translational movement in the horizontal and vertical directions, respectively, one parameter for zoom movement, and one parameter for rotational movement in both directions. The horizontal zoom parameter is equal to the vertical zoom parameter. The horizontal rotation parameter is equal to the vertical rotation parameter. To achieve more efficient affine parameter signaling, in the WC, these affine parameters are derived through two MVs (also called control point motion vectors (CPMVs)) located at the upper-left and upper-right corners of a current block. As shown in FIGS. 4A and 4B, the affine motion field of the block is described by two MV control points (Vo, Vi). Figure 4A shows an illustration of a 4-parameter affine model. Figure 4B shows an illustration of a 4-parameter affine model. Based on the control point motion, the motion field (vx, Vy) of an affine-coded block is described as (%--%) (Vlv-V0v) = ——— --;--- y + % VV W vv= —:xh—-—— y+v0w w The 6-parameter affine mode has the following parameters: two parameters for translational motion in the horizontal and vertical directions, respectively; one parameter for approach motion and one parameter for rotational motion in the horizontal direction; and one parameter for approach motion and one parameter for rotational motion in the vertical direction. The 6-parameter affine motion model is coded with three CPMVs. Figure 5 illustrates a 6-parameter affine model. As shown in Figure 5, three control points of a 6-parameter affine block are located at the upper left, upper right, and lower left corners of the block. Movement at the upper left control point is related to translational motion, movement at the upper right control point is related to rotational and zooming motion in the horizontal direction, and movement at the lower left control point is related to rotational and zooming motion in the vertical direction. Compared to the 4-parameter affine motion model, the horizontal rotational and zooming motion in the 6-parameter model may not be the same as the vertical rotational and zooming motion.Assuming that (Vo, Vi, V2) are the MVs of the upper left, upper right, and lower left corners of the current block in FIG. 5, the movement vector of each sub-block (14, Vy) is derived using three MVs at control points as follows: * . .and r. = 4-l vr— J * — 4- 't— Vor। *— W ' / 7 R = 1¾ + (vly- 1¾. í + Uy - i-:yh |(2) In the WC, CPMVs of affine-encoded blocks are stored in a separate buffer. The stored CPMVs are used only for generating affine CPMV predictors in affine integration mode (i.e., inheriting affine CPMVs from neighboring affine blocks) and explicit affine mode (i.e., signaling affine CPMVs based on a prediction-based scheme). Subblock MVs derived from CPMVs are used for motion compensation, MV prediction of translational MVs, and unlocking. Similar to regular interblock motion compensation, the MVs of each affine subblock can be directed toward reference samples at fractional sample positions. In such a case, the interpolation filtering process is needed to generate fractional pixel position reference samples. To control the worst-case memory bandwidth requirement and worst-case computational complexity of interpolation, a set of 6-tap interpolation filters is used for affine subblock motion compensation. Tables 1 and 2 illustrate the interpolation filters used for regular interblock and affine block motion compensation, respectively.As can be seen, the 6-tap interpolation filters used for affine mode are derived directly from the 8-tap filters used for regular interblocks by directly incorporating two more outer filter coefficients on each side of 8-tap filters into a single filter coefficient for 6-tap filters, i.e., the filter coefficients P0 and P5 in Table 2 are equal to the sum of the filter coefficients P0 and P1 and the sum of the filter coefficients P6 and P7 in Table 1, respectively. TABLE 1 Brightness interpolation filters used for regular interblocks. Fractional Position Interpolation Filter Coefficients 'O P1 P2 P3 P4 P5 P6 P7 1 0 1 -3 63 4 -2 1 0 2 -1 2 -5 62 8 -3 1 0 3 -1 3 -8 60 13 -4 1 0 4 -1 4 -10 58 17 -5 1 0 5 -1 4 -11 52 26 -8 3 -1 6 -1 3 -9 47 31 -10 4 -1 7 -1 4 -11 45 34 -10 4 -1 8 -1 4 -11 40 40 -11 4 -1 9 -1 4 -10 34 45 -11 4 -1 10 -1 4 -10 31 47 -9 3 -1 11 -1 3 -8 26 52 -11 4 -1 12 0 1 -5 17 58 -10 4 -1 13 0 1 -4 13 60 -8 3 -1 14 0 1 -3 8 62 -5 2 -1 15 0 1 -2 4 63 -3 1 0 TABLE 2 Brightness interpolation filters used for affine blocks Fractional Position Interpolation Filter Coefficients PO P1 P2 P3 P4 P5 1 1 -3 63 4 -2 1 2 1 -5 62 8 -3 1 3 2 -8 60 13 -4 1 4 3 -10 58 17 -5 1 5 3 -11 52 26 -8 2 6 2 -9 47 31 -10 3 7 3 -11 45 34 -10 3 8 3 -11 40 40 -11 3 9 3 -10 34 45 -11 3 10 3 -10 31 47 -9 2 11 2 -8 26 52 -11 3 12 1 -5 17 58 -10 3 13 1 -4 13 60 -8 2 14 1 -3 8 62 -5 1 15 1 -2 4 63 -3 1 Additionally, for color sample motion compensation, the same 4-take interpolation filters (as illustrated in Table 3) that are used for regular interblocks are used for affine blocks. TABLE 3 Color interpolation filters used for interblocks (i.e., affine blocks vs. non-affine blocks) Fractional sample interpolation filter coefficients P0 P1 P2 P3 1 -1 63 2 0 2 -2 62 4 0 3 -2 60 7 -1 4 -2 58 10 -2 5 -3 57 12 -2 6 -4 56 14 -2 7 -4 55 15 -2 8 -4 54 16 -2 9 -5 53 18 -2 10 -6 52 20 -2 11 -6 49 24 -3 12 -6 46 28 -4 13 -5 44 29 -4 14 -4 42 30 -4 15 -4 39 33 -4 16 -4 36 36 -4 17 -4 33 39 -4 18 -4 30 42 -4 19 -4 29 44 -5 20 -4 28 46 -6 21 -3 24 49 -6 22 -2 20 52 -6 23 -2 18 53 -5 24 -2 16 54 -4 25 -2 15 55 -4 26 -2 14 56 -4 27 -2 12 57 -3 28 -2 10 58 -2 29 -1 7 60 -2 30 0 4 62 -2 31 0 2 63 -1 Reference image resampling Unlike HEVC, the emerging WC standard supports rapid spatial resolution switching within the bitstream of the same content. This capability is referred to as Reference Image Resampling (RPR) or Adaptive Resolution Shifting (ARC). In real-time video applications, enabling resolution switching within an encoded video stream without requiring the insertion of a random access point (IRAP) image (e.g., an IDR or CRA image) not only adapts compressed video data to dynamic communication channel conditions but also avoids the bandwidth burst caused by the relatively large size of IDR or CRA images. Specifically, the following typical user scenarios could benefit from the RPR feature: Adaptation index in video telephony and follow-up meetings: To adapt encoded video to changing network conditions, when network conditions worsen and available bandwidth decreases, the encoder can adapt by using lower encoding resolutions. Currently, changing the image resolution can only be done after an IRAP image; this presents several problems. An IRAP image of reasonable quality will be much larger than an inter-encoded image and will be correspondingly more complex to decode, consuming time and resources. This is problematic if the resolution change is requested by the decoder for load reasons. It can also break low-latency buffering conditions, forcing audio resynchronization, and the end-to-end latency of the stream will increase, at least temporarily. This can result in a poor user experience. Active speaker changes in multi-party video conferencing: In multi-party video conferencing, it's common for the active speaker to appear larger in the video feed than the other participants. When the active speaker changes, the image resolution for each participant may also need to be adjusted. The need for an ARC (Audit Response) feature becomes more critical when such active speaker changes occur frequently. Faster Startup in Real-Time Streaming: For real-time streaming applications, it's common for the application to accumulate a certain length of decoded image before starting to display. Starting the bitstream at a lower resolution can allow the application to have enough images in the buffer to begin displaying faster. Adaptive Streaming Switching in Continuous Streaming: Dynamic Adaptive Streaming over HTTP (DASH) includes a feature called @mediaStreamStructureId. This enables switching between different representations on open-GOP random access points with non-decodable main images, for example, CRA images with associated RASL images in HEVC. When two different representations of the same video have different bitrates but the same spatial resolution, and both have the same @mediaStreamStructureId value, switching between the two representations can occur within a CRA image with associated RASL images. The RASL images associated with the switch to CRA images can then be decoded with acceptable quality, thus allowing for seamless switching. With ARC, the @mediaStreamStructureId feature could also be used to switch between DASH representations with different spatial resolutions. At the 15th JVET meeting, the RPR feature was formally supported by the WC standard. The main aspects of the existing RPR design in the WC are summarized as follows: High-level RPR signage In accordance with the current RPR design, in the sequence parameter set (SPS), two syntax elements, pic_width_maxjnjuma_samples and pic_height_maxjn_luma_samples, are designated to specify the maximum width and height of the MA / IZ / ¿U¿¿ / UOOin O encoded images that refer to SPS. Then, when the image resolution is changed, a new image parameter set (PPS) needs to be set when the related syntax elements pic_width_luma_samples and pic_height_luma_samples are signaled to specify different image resolutions of the images referring to the PPS. There is bitstream conformance that the values of pic_width_luma_sample and pic_height_luma_sample should not exceed that of pic_width_max_luma_samples and pic_height_max_luma_samples. Table 4 illustrates the RPR-related signaling in the SPS and PPS TABLE 4 RPR signaling in the SPS and PPS seq parameter set rbsp() { Descriptor pic_width_max_in_luma_samples ue(v) pic_height_max_in_luma_samples ue(v)} foot parameter set rbsp() { Descriptor pic_width_in_luma_samples ue(v) pic_heig ht_i n_l u ma_sa m feet ue(v) ...} Reference image resampling process When a resolution change occurs within a bitstream, a current image can have one or more reference images at different sizes. According to the current RPR design, when the image resolution changes, all MVs for the current image are normalized to the sample grid of the current image rather than that of the reference images. This can make image resolution changes transparent to the MV prediction process. When the image resolution changes, in addition to MVs, the samples in a reference block must be sampled up / down during the motion compensation of the current block. In the WC, the scaling index, i.e., refPicWidthlnLumaSample / picWidthlnLuma and refPicHeightlnLumaSample / picHeightlnLumaSample, is limited to the interval [1 / 8, 2]. In the current RPR design, different interpolation filters are applied to interpolate the reference samples when the current image and its reference image have different resolutions. Specifically, when the reference image resolution is equal to or less than that of the current image, the default 8-shot and 4-shot interpolation filters are used to generate the interprediction samples for the brightness and color samples, respectively. However, the default motion interpolation filters do not exhibit strong low-pass characteristics. When the reference image resolution is higher than that of the current image, using the default motion interpolation filters will lead to significant overlap, which becomes more severe as the downsampling rate increases.Consequently, to improve the RPR's interprediction efficiency, two different sets of downsampling filters are applied when the reference image has a resolution 10 times higher than the current image. Specifically, when the downsampling index is equal to or greater than a certain value. 1.5: 1, the following Lanczos 8-pin and 4-pin filters are used as shown in Table 5 and Table 6. TABLE 5 Brightness interpolation filters when the sampling ratio down is equal to or greater than 1.5:1 Fractional sample interpolation filter coefficients P0 P1 P2 P3 P4 P5 P6 P7 0 -1 -5 17 42 17 -5 -1 0 1 0 -5 15 41 19 -5 -1 0 2 0 -5 13 40 21 -4 -1 0 3 0 -5 11 39 24 -4 -2 1 4 0 -5 9 38 26 -3 -2 1 5 0 -5 7 38 28 -2 -3 1 6 1 -5 5 36 30 -1 -3 1 7 1 -4 3 35 32 0 -4 1 8 1 -4 2 33 33 2 -4 1 9 1 -4 0 32 35 3 -4 1 10 1 -3 -1 30 36 5 -5 1 11 1 -3 -2 28 38 7 -5 0 12 1 -2 -3 26 38 9 -5 0 13 1 -2 -4 24 39 11 -5 0 14 0 -1 -4 21 40 13 -5 0 15 0 -1 -5 19 41 15 -5 0 TABLE 6 Color interpolation filters when the sampling ratio towards below is equal to or greater than 1.5:1 Fractional sample interpolation filter coefficients PO P1 P2 P3 0 12 40 12 0 1 11 40 13 0 2 10 40 15 -1 3 9 40 16 -1 4 8 40 17 -1 5 8 39 18 -1 6 7 39 19 -1 7 6 38 21 -1 8 5 38 22 -1 9 4 38 23 -1 10 4 37 24 -1 11 3 36 25 0 12 3 35 26 0 13 2 34 28 0 14 2 33 29 0 15 1 33 30 0 16 1 31 31 1 17 0 30 33 1 18 0 29 33 2 19 0 28 34 2 20 0 26 35 3 21 0 25 36 3 22 -1 24 37 4 23 -1 23 38 4 24 -1 22 38 5 25 -1 21 38 6 26 -1 19 39 7 27 -1 18 39 8 28 -1 17 40 8 29 -1 16 40 9 30 -1 15 40 10 31 0 13 40 11 When the downsampling ratio is equal to or greater than 2:1, the following 8-tap and 4-tap downsampling filters are used, which are derived by applying a cosine window function to the 12-tap SHM downsampling filters. TABLE 7 Brightness interpolation filters when the downsampling ratio is equal to or greater than 2:1 Fractional sample interpolation filter coefficients PO P1 P2 P3 P4 P5 P6 P7 0 -4 2 20 28 20 2 -4 0 1 -4 0 19 29 21 5 -4 -2 2 -4 -1 18 29 22 6 -4 -2 3 -4 -1 16 29 23 7 -4 -2 4 -4 1 16 28 24 7 -4 -2 5 -4 -1 14 28 25 8 -4 -2 6 -3 -3 14 27 26 9 -3 -3 7 -3 1 12 28 25 10 -4 -3 8 -3 3 11 27 TI 11 -3 -3 9 -3 4 10 25 28 12 -1 -3 10 -3 3 9 26 27 14 -3 -3 11 -2 -4 8 25 28 14 -1 -4 12 -2 -4 7 24 28 16 -1 -4 13 -2 -4 7 23 29 16 -1 -4 14 -2 -4 6 22 29 18 -1 -4 15 -2 -4 5 21 29 19 0 -4 TABLE 8 Color interpolation filters when the downsampling ratio is equal to or greater than 2:1 Fractional sample interpolation filter coefficients P0 P1 P2 P3 0 17 30 17 0 1 17 30 18 -1 2 16 30 18 0 3 16 30 18 0 4 15 30 18 1 5 14 30 18 2 6 13 29 19 3 7 13 29 19 3 8 12 29 20 3 9 11 28 21 4 10 10 28 22 4 11 10 27 22 5 12 9 27 23 5 13 9 26 24 5 14 8 26 24 6 15 7 26 25 6 16 7 25 25 7 17 6 25 26 7 19 18 6 24 26 8 19 5 24 26 9 20 5 23 27 9 21 5 22 27 10 22 4 22 28 10 23 4 21 28 11 24 3 20 29 12 25 3 19 29 13 26 3 19 29 13 27 2 18 30 14 28 1 18 30 15 29 0 18 30 16 30 0 18 30 16 31 -1 18 30 17 Finally, the downsampling filters described above are only applied to generate the brightness and color prediction samples for non-affine interblocks. For affine mode, the default 8-shot and 4-shot motion interpolation filters are still applied for downsampling. Problems in the existing RPR design The goal of this description is to improve the efficiency of affine-mode coding when RPR is applied. Specifically, the following problems are identified in the existing RPR design in the WC: First, as discussed earlier, when the reference image resolution is higher than that of the current image, additional downsampling filters are applied only to the motion compensation of non-affine modes. For affine mode, 6-take and 4-take motion interpolation filters are applied. Since these filters are derived from the default motion interpolation filters, they do not exhibit strong low-pass characteristics. Therefore, compared to non-affine mode, the affine mode prediction samples will exhibit more significant overlap artifacts due to the well-known Nyquist-Shannon sampling theorem. Thus, to achieve better encoding performance, it is desirable to also apply appropriate low-pass filters to the affine mode motion compensation when downsampling is required. Secondly, based on the existing RPR design, the fractional pixel position of the reference sample is determined based on the current sample position, the MV, and the resolution scaling index between the reference image and the current image. As such, when downsampling the reference block, this results in higher memory bandwidth consumption and computational complexity for interpolating the reference samples from the current block. Assuming the current block is of size M (width) x N (height), when the reference image is the same size as the current image, integer samples of size (M+7) x (N+7) need to be accessed from the reference image, and 8x(Mx(N+7)) + 8xMxN multiplications are required for motion compensation of the current block.If the downsampling scaling index is s, the corresponding memory bandwidth and multiplications increase to (s x M + 7) x (s x N + 7) and 8 x (M x (s x N + 7)) + 8 x M x N. Table 9 and Table 10 compare the integer number of samples and the number of multiplications per sample used for motion compensation of various block sizes when the downsampling scaling index RPR is 1.5X and 2X, respectively. In Table 9 and Table 10, the columns under the name RPR IX correspond to the case where the resolutions of the reference image and the current image are the same; that is, RPR is not applied. The index column for RPR IX illustrates the corresponding memory bandwidth / multiplication index with a downsampling RPR index greater than 1 for the corresponding worst-case number (i.e., 16x4 bi-prediction) under regular inter-mode without the RPR.As can be seen, compared to the worst-case complexity of regular interprediction, there is a significant increase in memory bandwidth and computational complexity when the reference image has a higher resolution than the current image. The peak increase comes from 16x4 bi-prediction, where the memory bandwidth and number of multiplications are 231% and 127%, respectively, of those of the worst-case bi-prediction. TABLE 9 Memory bandwidth consumption per sample when the RPR index is 1.5X and 2X Memory Bandwidth RPR IX RPR 1.5X RPR2X Memory Bandwidth Memory Bandwidth Index for RPR IX Memory Bandwidth Index for RPR IX 4 8 5.16 7.72 98% 10.78 136% 8 4 5.16 1J7. 98% 10.78 136% 4 16 7.91 12.59 159% 18.28 231% 16 4 7.91 12.59 159% 18.28 231% 4 32 6.70 11.17 141% 16.64 210% 32 4 6.70 11.17 141% 16.64 210% 4 64 6.10 10.46 132% 15.82 200% 64 4 6.10 10.46 132% 15.82 200% 4 128 5.80 10.11 128% 15.41 195% 128 4 5.80 10.11 128% 15.41 195% 8 8 7.03 11.28 143% 16.53 209% 8 16 5.39 9.20 116% 14.02 177% 16 8 5.39 9.20 116% 14.02 177% 8 32 4.57 8.16 103% 12.76 161% 32 8 4.57 8.16 103% 12.76 161% 8 64 4.16 7.64 97% 12.13 153% 64 8 4.16 7.64 97% 12.13 153% 8 128 3.96 7.38 93% 11.81 149% 128 8 3.96 7.38 93% 11.81 149% 16 16 4.13 7.51 95% 11.88 150% 16 32 3.50 6.66 84% 10.82 137% 32 16 3.50 6.66 84% 10.82 137% 16 64 3.19 6.24 79% 10.28 130% 64 16 3.19 6.24 79% 10.28 130% 16 128 3.03 6.02 76% 10.02 127% 128 16 3.03 6.02 76% 10.02 127% 32 32 2.97 5.91 75% 9.85 125% 32 128 2.57 5.34 68% 9.12 115% 128 32 2.57 5.34 68% 9.12 115% 64 64 2.46 5.18 66% 8.90 113% 64 128 2.34 5.00 63% 8.67 110% 128 64 2.34 5.00 63% 8.67 110% 128 128 2.22 4.83 61% 8.44 107% MA / IZ / ¿U¿¿ / UOOin O TABLE 10 Number of multiplications per sample when the RPR index is 1.5X vs 2X Width Height RPR IX RPR 1.5X RPR 2X Mu Mu index for RPR IX Mu index for RPR IX 4 8 23.00 27.00 45% 31.00 52% 8 4 30.00 34.00 57% 38.00 63% 4 16 39.00 47.00 78% 55.00 92% 16 4 60.00 68.00 113% 76.00 127% 4 32 35.50 43.50 73% 51.50 86% 32 4 60.00 68.00 113% 76.00 127% 4 64 33.75 41.75 70% 49.75 83% 64 4 60.00 68.00 113% 76.00 127% 4 128 32.88 40.88 68% 48.88 81% 128 4 60.00 68.00 113% 76.00 127% 8 8 46.00 54.00 90% 62.00 103% 8 16 39.00 47.00 78% 55.00 92% 16 8 46.00 54.00 90% 62.00 103% 8 32 35.50 43.50 73% 51.50 86% 32 8 46.00 54.00 90% 62.00 103% 8 64 33.75 41.75 70% 49.75 83% 64 8 46.00 54.00 90% 62.00 103% 8 128 32.88 40.88 68% 48.88 81% 128 8 46.00 54.00 90% 62.00 103% 16 16 39.00 47.00 78% 55.00 92% 16 32 35.50 43.50 73% 51.50 86% 32 16 39.00 47.00 78% 55.00 92% 16 64 33.75 41.75 70% 49.75 83% 64 16 39.00 47.00 78% 55.00 92% 16 128 32.88 40.88 68% 48.88 81% 128 16 39.00 47.00 78% 55.00 92% 32 32 35.50 43.50 73% 51.50 86% 32 128 32.88 40.88 68% 48.88 81% 128 32 35.50 43.50 73% 51.50 86% 64 64 33.75 41.75 70% 49.75 83% 64 128 32.88 40.88 68% 48.88 81% 128 64 33.75 41.75 70% 49.75 83% 128 128 32.88 40.88 68% 48.88 81% MA / IZ / ¿U¿¿ / UOOin O Third, in the existing RPR design, WC only supports adaptive changes in image resolutions within the same bitstream, while the bit depth used for encoding the video sequence remains the same. However, in accordance with the requirements for a Future Video Coding Standard to issue the WC standard's CfP, it is clearly stated that the standard must support rapid representation changes in the case of adaptive streaming services that offer multiple representations of the same content, each with different properties (e.g., spatial resolution or sample bit depth).In practical video applications, allowing the encoding bit depth to be changed within an encoded video sequence can offer more flexible performance / complexity advantages for video encoders / decoders, especially for software encoder-decoder implementations due to single instruction multiple data (SIMD) operations. Improvements to RPR Coding This description proposes solutions to improve the efficiency and reduce the memory bandwidth and computational complexity of RPR encoding in the WC. More specifically, the technologies proposed in this description can be summarized as follows: First, to improve the RPR encoding efficiency of affine mode, new low-pass interpolation filters are proposed to replace the existing 8-shot brightness and 4-shot color interpolation filters used for affine when the reference image is at a higher resolution than the current image, i.e., when downsampling is required. Second, for the simplification of RPR, it is proposed to disable RPR-based interprediction for certain CU sizes that lead to significant memory bandwidth and increased computational complexity compared to that of regular intermode without RPR being applied. Third, an approach is proposed to allow dynamic changes in the internal bit depth used to encode a video sequence. Downsampling filters for affine mode As mentioned previously, the default 6-slice and 4-slice motion interpolation filters are always applied in affine mode, regardless of whether the resolutions of the current image and its reference image are the same. Similar to the interpolation filters used in HEVC, the default motion interpolation filters in WC do not exhibit strong low-pass characteristics. When the spatial scaling ratio is close to 1, the default motion interpolation filter can provide acceptable quality prediction samples. However, as the downsampling resolution from the reference image to the current image increases, based on the Nyquist-Shannon sampling theorem, overlap artifacts will be much more severe if the same default motion interpolation filter is used.Specifically, when the applied MV targets reference samples at integer sample positions, the default move interpolation does not apply any filtering operations. This could result in a significant drop in the quality of prediction samples for related blocks. To mitigate the overlap artifacts resulting from downsampling, as described above, it is proposed to use different interpolation filters with stronger low-pass characteristics to replace the existing default 6-tap / 4-tap interpolation filters for affine mode motion compensation. Additionally, to maintain the same memory bandwidth and computational complexity as the regular motion compensation process, the proposed downsampling filters are of the same length as the existing interpolation filters used for affine mode—that is, 6 taps for the brightness component and 4 taps for the color component. Figure 7 shows a method for decoding a video signal. The method can be applied, for example, to a decoder. In step 710, the decoder can obtain a reference image i associated with a video block within the video signal. In step 712, the decoder can obtain reference samples iü.jj of the video block from a reference block in reference image 1. The iyj, for example, can represent a coordinate of a sample within the video block. In step 714, the decoder can obtain a first downsampling filter and a second downsampling filter to generate respectively brightness and color interprediction samples of the video block when the video block is encoded in a non-affine intermode and a reference image resolution I is greater than that of a current image. In step 716, the decoder can obtain a third downsampling filter and a fourth downsampling filter to generate respectively the brightness and color interprediction samples of the video block when the video block is affine-mode encoded and the reference image resolution is higher than that of the current image. In step 718, the decoder can obtain interprediction samples of the video block based on the third and fourth downsampling filters being applied to the reference samples;jj. Downsampling filters of affine brightness There can be multiple ways to derive downsampling filters for brightness in affine mode. Method 1: In one or more of the described modalities, it is proposed to directly derive the affine-mode brightness downsampling filters from the existing regular intermode (i.e., non-affine-mode) brightness downsampling filters. Specifically, using this method, the new 6-tap brightness downsampling filters are derived from the 6-tap brightness downsampling filters in Table 5 (for a scaling index of 1.5X) and Table 7 (for a scaling index of 2X) by adding two far-left / far-right filter coefficients from the 8-tap filter to a single filter coefficient, respectively, for the 6-tap filter. Table 11 and Table 12 illustrate the proposed 6-tap brightness downsampling filters when the spatial scaling index is 1.5:1 and 2:1, respectively. TABLE 11 6-shot downsampling filters for brightness reduction, scaling ratio equal to or greater than 1.5:1 Fractional sample interpolation filter coefficients PO P1 P2 P3 P4 P5 0 -4 17 42 17 -5 -1 1 -5 15 41 19 -5 -1 2 -5 13 40 21 -4 -1 3 -5 11 39 24 -4 -1 4 -5 9 38 26 -3 -1 5 -5 7 38 28 -2 -2 6 -4 5 36 30 -1 -2 7 -3 3 35 32 0 -3 8 -3 2 33 33 2 -3 9 -3 0 32 35 3 -3 10 -2 -1 30 36 5 -4 11 -2 -2 28 38 7 -5 12 -1 -3 26 38 9 -5 13 -1 -4 24 39 11 -5 14 -1 -4 21 40 13 -5 15 -1 -5 19 41 15 -5 TABLE 12 6-shot downsampling filters reduce brightness when the scaling ratio is equal to or greater than 2:1 Fractional sample interpolation filter coefficients P0 P1 P2 P3 P4 P5 0 -2 20 28 20 2 -4 1 -4 19 29 21 5 -6 2 -5 18 29 22 6 -6 3 -5 16 29 23 7 -6 4 -5 16 28 24 7 -6 5 -5 14 28 25 8 -6 6 -6 14 27 26 9 -6 7 -4 12 28 25 10 -7 8 -6 11 27 U 11 -6 9 -7 10 25 28 12 -4 10 -6 9 26 27 14 -6 11 -6 8 25 28 14 -5 12 -6 7 24 28 16 -5 13 -6 7 23 29 16 -5 14 -6 6 22 29 18 -5 15 -6 5 21 29 19 -4 Method 2: In one or more of the described modalities, it is proposed to directly derive the 6-tap affine downsampling filters from the SHM filters, which are derived based on the cosine function of the Windowed sinc filter. Specifically, in this method, the affine downsampling filter is derived based on the following equation: L - 1 L - 1 Fílt, / AnJ = MnJ wi.'ii,, n = —.............. Ί T' (3) where L is the filter length and Mn) is the low-pass frequency response, which is calculated as: hóií = s · £ · sincÍs £ M.Ϊ. n = — .....-Hk (4) f, is a cutoff frequency and s is the scaling index. wGü is a cosine window function, which is defined as: I ?i '. i - 1 i - 1 W'..Ί J = eos) ·;---= ——7—.....—7— (5) In one example, assuming / is 0.9 and L = 6, Table 13 and Table 14 illustrate the derived 6-shot brightness downsampling when the spatial scaling index is 1.5X (i.e., s = 1.5) and 2X (s = 2). TABLE 13 6-shot downsampling filters, scaling ratio equal to or greater than 1.5:1 Fractional Sample Filter Coefficients p0 P1 P2 P3 p4 p5 0 -5 18 38 18 -5 0 1 -5 16 39 20 -4 -2 2 -5 14 39 22 -4 -2 3 -5 13 38 24 -3 -3 4 -5 11 37 26 -2 -3 5 -5 9 36 28 -1 -3 6 -5 8 35 30 -1 -3 7 -4 6 34 31 1 -4 8 -4 3 33 33 3 -4 9 -4 1 31 34 6 -4 10 -3 -1 30 35 8 -5 11 -3 -1 28 36 9 -5 12 -3 -2 26 37 11 -5 13 -3 -3 24 38 13 -5 14 -2 -4 22 39 14 -5 15 -2 -4 20 39 16 -5 TABLE 14 6-shot downsampling filters for brightness when the scaling index is MA / IZ / ¿U¿¿ / UOOin O equal to or greater than 2:1 Fractional Sample Position p Filter Coefficients pO P1 P2 P3 p4 p5 0 0 19 26 19 0 0 1 -1 18 27 20 2 -2 2 -1 16 27 21 3 -2 3 -2 15 27 22 5 -3 4 -2 14 26 23 6 -3 5 -2 13 26 24 6 -3 6 -3 12 25 25 8 -3 7 -3 11 25 25 9 -3 8 -3 10 25 25 10 -3 9 -3 9 25 25 11 -3 10 -3 8 25 25 12 -3 11 -3 6 24 26 13 -2 12 -3 6 23 26 14 -2 13 -3 5 22 27 15 -2 14 -2 3 21 27 16 -1 15 -2 2 20 27 18 -1 Note that in Table 13 and Table 14, the filter coefficients are derived in the precision of the 7-bit sign variable, in which the same as the downsampling filters used in the RPR design is maintained. Figure 8 shows a method for decoding a video signal. The method can be applied, for example, to a decoder. In step 810, the decoder can obtain a frequency response of an ideal low-pass filter based on a cutoff frequency and scaling index. In step 812, the decoder can obtain a cosine window function based on a filter length. In step 814, the decoder can obtain the third downsampling filter based on the frequency response and cosine window function. Affine color downsampling filters The following are three methods for downsampling a color reference block when the resolution of the reference image is higher than that of the current image. Method 1: In the first method, it is proposed to reuse the existing 4-tap color downsampling filters for 1.5X (Table 6) and 2X (Table 8) designed for non-affine mode under RPR for downsampling the reference samples for affine mode. Method 2: In the second method, it is proposed to reuse the default 4-take color interpolation filters (Table 3) for downsampling of the affine mode reference samples. Method 3: In the third method, it is proposed to derive 5-shot color downsampling filters based on the cosine windowed synchronization filter function, as illustrated in (3) to (5). Table 15 and Table 16 illustrate the derived 4-shot color downsampling filters for scaling indices of 1.5X and 2X, respectively, when the cutoff frequency of the cosine windowed synchronization function is assumed to be 0.9. TABLE 15 4-shot color downsampling filters, scaling ratio equal to or greater than 1.5:1 Fractional Sample Position p Filter Coefficients PO P1 P2 p3 0 13 38 13 0 1 13 38 14 -1 2 12 38 15 -1 3 11 37 17 -1 4 10 38 18 -2 5 9 38 19 -2 6 8 38 20 -2 7 7 38 21 -2 8 6 37 22 -1 9 6 36 24 -2 10 5 36 25 -2 11 4 35 26 -1 12 4 34 27 -1 13 3 34 28 -1 14 3 33 29 -1 15 2 32 30 0 16 1 31 31 1 17 0 30 32 2 18 -1 29 33 3 19 -1 28 34 3 20 -1 27 34 4 21 -1 26 35 4 22 -2 25 36 5 23 -2 24 36 6 24 -1 22 37 6 25 -2 21 38 7 26 -2 20 38 8 27 -2 19 38 9 28 -2 18 38 10 29 -1 17 37 11 30 -1 15 38 12 31 -1 14 38 13 TABLE 16 4-shot color downsampling filters, scaling ratio equal to or greater than 2:1 Fractional sample position p Filter coefficients p0 P1 P2 p3 0 17 30 17 0 1 17 30 18 -1 2 16 30 18 0 3 16 30 18 0 4 15 30 18 1 5 14 30 18 2 6 13 29 19 3 7 13 29 19 3 8 12 29 20 3 9 11 28 21 4 10 10 28 22 4 11 10 27 22 5 12 9 27 23 5 13 9 26 24 5 14 8 26 24 6 15 7 26 25 6 16 7 25 25 7 17 6 25 26 7 18 6 24 26 8 19 5 24 26 9 20 5 23 TI 9 21 5 22 27 10 22 4 22 28 10 23 4 21 28 11 24 3 20 29 12 25 3 19 29 13 26 3 19 29 13 27 2 18 30 14 28 1 18 30 15 29 0 18 30 16 30 0 18 30 16 31 -1 18 30 17 Block size restricted by RPR mode As discussed in the problem statement section, when downsampling occurs, the existing RPR leads to a significant increase in complexity (e.g., the number of integer samples accessed by move compensation and the number of multiplications required). Specifically, the memory bandwidth and number of multiplications when the reference block needs to be downsampled are 231% and 127% of that of the worst-case bi-prediction. In one or more modes, it is proposed to disable bi-prediction (but still allow uni-prediction) during interprediction for certain block shapes, for example, 4xN, Nx4, and / or 8x8, when the reference image resolution is higher than that of the current image. Table 17 and Table 18 show the corresponding memory bandwidth per sample and the number of multiplications when bi-prediction is disabled for 4xN, Nx4, and 8x8 block sizes during interprediction with RPR. As can be seen, with the proposed restriction, the memory bandwidth and the number of multiplications are reduced to 130% and 107%, respectively, of the worst-case bi-prediction for downsampling to 1.5X, and 116% and 113% for downsampling to 2X. TABLE 17 Memory bandwidth consumption per sample after block size restriction applies for the RPR, for sampling rates down to 1.5X v 2X Ancho Altura RPR IX RPR 1.5X RPR 2X Ancho de banda de Memoria Ancho de banda de Memoria índice a RPR IX Ancho de banda de Memoria índice a RPR IX 4 8 5.16 7.72 98% 10.78 136% 8 4 5.16 7.72 98% 10.78 136% 4 16 7.91 10.25 130% 13.09 166% 16 4 7.91 10.25 130% 13.09 166% 4 32 6.70 8.94 113% 11.67 148% 32 4 6.70 8.94 113% 11.67 148% 4 64 6.10 8.28 105% 10.96 139% 64 4 6.10 8.28 105% 10.96 139% 4 128 5.80 7.95 101% 10.61 134% 128 4 5.80 7.95 101% 10.61 134% 8 8 7.03 9.16 116% 11.78 149% 8 16 5.39 9.20 116% 14.02 177% 16 8 5.39 9.20 116% 14.02 177% 8 32 4.57 8.16 103% 12.76 161% 32 8 4.57 8.16 103% 12.76 161% 8 64 4.16 7.64 97% 12.13 153% 64 8 4.16 7.64 97% 12.13 153% 8 128 3.96 31 7.38 93% 11.81 149% 128 8 3.96 7.38 93% 11.81 149% 16 16 4.13 7.51 95% 11.88 150% 16 32 3.50 6.66 84% 10.82 137% 32 16 3.50 6.66 84% 10.82 137% 16 64 3.19 6.24 79% 10.28 130% 64 16 3.19 6.24 79% 10.28 130% 16 128 3.03 6.02 76% 10.02 127% 128 16 3.03 6.02 76% 10.02 127% 32 32 2.97 5.91 75% 9.85 125% 32 128 2.57 5.34 68% 9.12 115% 128 32 2.57 5.34 68% 9.12 115% 64 64 2.46 5.18 66% 8.90 113% 64 128 2.34 5.00 63% 8.67 110% 128 64 2.34 5.00 63% 8.67 110% 128 128 2.22 4.83 61% 8.44 107% TABLE 18 Number per sample of multiplications when the block size restriction is applied towards the RPR, for sampling indices below 1.5X and 2X Width Height RPR IX RPR 1.5X RPR 2X Mu Mu index for RPR IX Mu index for RPR IX 4 8 23.00 27.00 45% 31.00 52% 8 4 30.00 34.00 57% 38.00 63% 4 16 39.00 43.00 72% 47.00 78% 16 4 60.00 64.00 107% 68.00 113% 4 32 35.50 39.50 66% 43.50 73% 32 4 60.00 64.00 107% 68.00 113% 4 64 33.75 37.75 63% 41.75 70% 64 4 60.00 64.00 107% 68.00 113% 4 128 32.88 36.88 61% 40.88 68% 128 4 60.00 64.00 107% 68.00 113% 8 8 46.00 50.00 83% 54.00 90% 8 16 39.00 47.00 78% 55.00 92% 16 8 46.00 54.00 90% 62.00 103% 8 32 35.50 43.50 73% 51.50 86% 32 8 46.00 54.00 90% 62.00 103% 8 64 33.75 41.75 70% 49.75 83% 64 8 46.00 32 54.00 90% 62.00 103% 8 128 32.88 40.88 68% 48.88 81% 128 8 46.00 54.00 90% 62.00 103% 16 16 39.00 47.00 78% 55.00 92% 16 32 35.50 43.50 73% 51.50 86% 32 16 39.00 47.00 78% 55.00 92% 16 64 33.75 41.75 70% 49.75 83% 64 16 39.00 47.00 78% 55.00 92% 16 128 32.88 40.88 68% 48.88 81% 128 16 39.00 47.00 78% 55.00 92% 32 32 35.50 43.50 73% 51.50 86% 32 128 32.88 40.88 68% 48.88 81% 128 32 35.50 43.50 73% 51.50 86% 64 64 33.75 41.75 70% 49.75 83% 64 128 32.88 40.88 68% 48.88 81% 128 64 33.75 41.75 70% 49.75 83% 128 128 32.88 40.88 68% 48.88 81% Although in the previous example, RPR's bi-prediction mode is only disabled for block sizes of 4xN, Nx4, and 8x8, for people knowledgeable in the state of the art of modern video technologies, the proposed restriction is also applicable to other block sizes and intercoding modes (e.g., uni- / bi-prediction, integration / non-integration mode, among others). Adaptive bit depth change In the existing RPR design, the WC only supports adaptive changes in image resolutions within the same bitstream, while the bit depth used for encoding the video sequence remains constant. However, as discussed earlier, allowing changes in encoding bit depth within the same bitstream can offer greater flexibility for practical encoder / decoder devices and presents various advantages and disadvantages in terms of encoding performance and computational complexity. In this section, an adaptive bit depth change (ABS) approach is proposed to allow changing the internal encoding bit depth without the requirement of introducing an IRAP image such as the Instant Decoder Refresh (IDR) image, etc. Figure 6 illustrates a hypothetical example where the current image 620 and its reference images 610 and 630 are encoded at different internal bit depths. Figure 6 shows reference image 610 RefO with 8-bit encoding, current image 620 with 10-bit encoding, and reference image 630 Refl with 12-bit encoding. Specifically, in what follows, high-level syntax signaling and modifications to the motion compensation process are proposed for the current WC framework to support the proposed ABS capability. High-level ABS signaling For the proposed ABS signaling, in the SPS, a new syntax element sps_max_bit_depth_minus8 is proposed to replace the existing bit depth syntax element bit_depth_minus8, which specifies the maximum internal encoding bit depth used for encoded images referencing the SPS. Therefore, when the encoding bit depth is changed, a new PPS syntax pps_bit_depth_minus8 is sent to specify the different encoding bit depth for images referencing the PPS. There is bitflow agreement that the values of pps_bit_depth_minus8 should not exceed that of sps_max_bit_depth_minus8. Table 19 illustrates the proposed ABS signaling in the SPS and PPS. TABLE 19 The ABS signaling proposed in the SPS v PPS seq_parameter_set_rbsp() { Descriptor sps max bit depth minus8 ue(v) —bit_depth_minus8 uc(v)} foot parameter set rbsp() { Descriptor pps_bit_depth_minus8 ue(v) ...} Prediction sample bit depth adjustment When there is a change in encoding bit depth within an encoded video sequence, the current image can be predicted from a reference image whose reconstructed samples are represented at different bit depth accuracies. In such a case, the prediction samples generated from the motion compensation of the reference image should be adjusted to the encoding bit depth of the current image. Interaction with other coding tools Since ABS applied to the reference image and the current image can be rendered at different accuracies, some existing WC encoding tools that use reference samples to derive certain encoding parameters may not be functioning correctly. For example, in the current WC, Bidirectional Optical Flow (BDOF) and Decoder-Side Motion Vector Refinement (DMVR) are two decoder-side technologies that use time-prediction samples to improve intercoding efficiency. Specifically, the BDOF tool uses the LO and Ll prediction samples to calculate convenient sample refinements to improve the prediction sample quality, while DMVR relies on the LO and Ll prediction samples to refine the motion vector accuracy at the sub-block level.Based on the above consideration, this is proposed to always avoid the BDOF and DMVR processes for an inter-block when either of the two prediction signals is encoded at a different bit depth from the current image. Figure 9 shows a 910 computing environment coupled with a 960 user interface. The 910 computing environment can be part of the data processing server. The 910 computing environment includes the 920 processor, the 940 memory, and the 950 I / O interface. The 920 processor typically controls the general operations of the 910 computing environment, such as those associated with the display, data acquisition, data communications, and image processing. The 920 processor may include one or more processors to execute instructions for performing all or some of the steps in the methods described above. In addition, the 920 processor may include one or more modules that facilitate interaction between the 920 processor and other components. The processor may be a Central Processing Unit (CPU), a microprocessor, a single-chip machine, a GPU, or similar. The 940 memory is configured to store various types of data to support the operation of the 910 computing environment. The 940 memory may include pre-installed software 942. Examples of such data include instructions for any applications or methods operated in the 910 computing environment, video data sets, image data, etc. The 940 memory may be implemented using any type of volatile or non-volatile memory device, or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk. The 950 I / O interface provides an interface between the 920 processor and peripheral interface modules, such as a keyboard, click wheel, buttons, and the like. Buttons may include, but are not limited to, a start button, a scan start button, and a scan stop button. The 950 I / O interface can be coupled with an encoder and a decoder. In one embodiment, a non-transient, computer-readable storage medium is also provided, comprising a plurality of programs, such as those contained in memory 940, executable by the processor 920 in the computing environment 910, to perform the methods described above. For example, the non-transient, computer-readable storage medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, or the like. The non-transient, computer-readable storage medium has stored therein a plurality of programs to be executed by a computing device having one or more processors, wherein the plurality of programs, when executed by the one or more processors, cause the computing device to perform the method described above for motion prediction. In one mode, the 910 computing environment can be implemented with one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), graphics processing units (GPUs), controllers, microcontrollers, microprocessors, or other electronic components, to perform the above methods. The description in this application is for illustrative purposes only and is not intended to be exhaustive or limited to the description provided herein. Many modifications, variations, and alternative implementations will be evident to those skilled in the art, having benefited from the lessons presented in the preceding descriptions and associated drawings. The examples were chosen and described to illustrate the principles of the description and to enable those skilled in the art to understand the description for various implementations, to better utilize the underlying principles, and to demonstrate that several implementations with various modifications are suitable for the intended use. Therefore, it should be understood that the scope of the description is not limited to the specific examples of the implementations described, and that modifications and other implementations are intended to be included within the scope of this description.
Claims
1. A method for decoding a video signal, comprising: obtaining a reference image associated with a video block within the video signal; obtaining reference samples of the video block from the reference image; determining a luma interpolation filter for the encoded video block in an affine motion mode based on a scaling ratio derived from the resolutions of the reference image and the current image; and obtaining inter-luma prediction samples of the video block by applying the luma interpolation filter to the reference samples.
2. The method according to claim 1, further characterized in that determining the luma interpolation filter for the video block encoded in affine motion mode based on the scale ratio comprises: determining a first luma interpolation filter as the luma interpolation filter for the video block encoded in affine motion mode in response to the scale ratio being equal to or greater than a first value, wherein the first luma interpolation filter is different from a second luma interpolation filter for the video block encoded in affine motion mode when the scale ratio is not equal to or greater than the first value.
3. The method according to claim 1, further characterized in that the luma interpolation filter is associated with a third luma interpolation filter used when the video block is encoded in a non-affine motion mode and the resolution scale ratio is equal to or greater than the first value. 4.- The method according to claim 3, further characterized in that one of the filter coefficients of the luma interpolation filter is equal to a sum of the first two filter coefficients or the last two filter coefficients of the third luma interpolation filter.
5. The method according to claim 1, further characterized in that it additionally comprises: determining a chroma interpolation filter for the video block encoded in the affine motion mode based on a comparison between the resolutions of the reference image and the current image; and obtaining an interchroma prediction sample of the video block by applying the chroma interpolation filter to the reference samples.
6. The method according to claim 1, further characterized in that the determination of the luma interpolation filter comprises: obtaining the luma interpolation filter based on applying a windowed sinc cosine function on top of the filters of the scalable HEVC test model (SHM).
7. The method according to claim 6, further characterized in that the determination of the luma interpolation filter comprises: obtaining a frequency response of an ideal low-pass filter based on a cutoff frequency and scale ratio; obtaining a cosine window function based on a filter length; and obtaining the sample-down filter based on the frequency response and the cosine window function.
8. The method according to claim 7, further characterized in that the cutoff frequency is equal to 0. 9, the filter length is equal to 6 and the scale ratio is equal to 1.
5. 10 9.- The method according to claim 7, further characterized in that the cutoff frequency is equal to 0.9, the filter length is equal to 6 and the scale ratio is equal to 2.