Phase aware inter prediction using rpr

Abstract

A video codec is described wherein a motion vector transmitted in the data stream for a predetermined block of a first picture is scaled according to a resolution ratio between the first picture and a second picture to obtain a scaled motion vector, the predetermined block is motion-compensatedly predicted by sampling the second picture at a motion-compensatedly offset position resulting from applying the motion vector to a corresponding position in the second picture, corresponding to a position of the predetermined block in the first picture, wherein the encoder / decoder inserts / derives, as / from side information in the data stream, information on a relative sampling phase offset and determines the corresponding position in the second picture using the information on the relative sampling phase offset. Further, a video codec is described where the encoder / decoder determines the motion vector using the information on the relative sampling phase offset. The described aspects may be combined.

Description

[0001] PHASE AWARE INTER PREDICTION USING RPR

[0002] DESCRIPTION

[0003] TECHNICAL FIELD

[0004] Embodiments of the invention relate to apparatuses for video encoding or decoding supporting inter prediction using reference picture resampling.

[0005] BACKGROUND

[0006] Overview of RPR

[0007] Reference Picture Resampling is a technology used in video coding, which has been included in VVC, that allows for using reference pictures that have a different size / resolution than the currently predicted picture. Referencing a picture with a different size / resolution requires some kind of resampling of the reference picture so that a reference picture that has a different size than the current predicted picture is “transformed” to have the same size / resolution as the current picture.

[0008] A naive approach could involve first using some resampling process that uses resampling filters to convert the reference picture to a picture that has the target size / resolution of the current predicted picture and then apply motion estimation and motion compensation to the “transformed” / ” converted” reference picture. However, this comes with an additional complexity, which is the resampling step.

[0009] A different approach that would not have such a complexity has been used in the case of VVC: namely, reusing the fractional interpolation filters used in motion compensation. As shown in the following tables, the interpolation filters used for “upsampling” reference pictures or when the “downsampling” factor is smaller than or equal to 1.25x, the same interpolation filter is used as for fractional sample interpolation due to MVs pointing to a fractional sample position when there is no RPR. For larger downsampling factors two further set of interpolation filter coefficients are used that provide better results avoiding visual artifacts such as aliasing. In the following the luma interpolation filter coefficients are explained but for chroma a similar approach applies. The following table relates to the case where Upsampling and downsampling <= 1.25x equal to the fractional interpolation filter when there is no RPR:

[0010] The following table relates to the case where the fractional interpolation filter > 1.75x downsampling: The following table relates to the case where the fractional interpolation filter > 1.25x downsampling and >= 1.75x downsampling:

[0011] Thus, basically RPR can be seen as a regular form of motion compensation. Since the fractional interpolation filters have a 1 / 16 sample precision for each sample that is predicted, the relative position of each sample in the reference picture at its 1 / 16th sample precision is derived and the respective interpolation filter coefficients are used. In the following the formula for the horizontal position is shown that performs the described reference 1 / 16 sample position (refXL). refxSbL = ( ( ( xSb - ScalingWinLeftOffsetLuma ) « 4 ) + refMvLXf 0 ] ) * scalingRatiof 0 ] (1) refxL = ( ( Sign( refxSbL ) * ( ( Abs( refxSbL ) + 128 ) » 8 ) +

[0012] XL * ( ( scalingRatiof 0 ] + 8 ) » 4 ) ) + fRefLeftOffset + 32 ) » 6 (2) where xSb is the (horizontal) position of the top4eft luma sample of the current block in luma samples, ScalingWinLeftOffsetLuma is as discussed in the following a scaling window left border of the current picture in luma samples, refMvLXf 0 ] is the horizontal component of the motion vector for the current block in fractional luma samples (e.g., 1 / 16 precision as in VVC - and therefore the «4 for the other variables), scalingRatiof 0 ] is the horizontal scaling ratio between current picture and the reference picture which is given in the example as a ratio between current picture and reference picture shifted by 14 to the left (i.e. « 14), xLfrom 0 ... Blocksize-1 is the horizontal sample position (in luma samples) for each of the samples within the block and fRefLeftOffset in luma samples (of the reference picture) is the scaling window left border of the reference picture.

[0013] One aspect to mention that can be seen in the above formula is the fact that RPR allows defining Scaling Window Offsets, which basically define a scaling window in the current picture and in the reference picture. Without using such a scaling window the whole output or decoded pictures (current and reference) would be used to compute the scaling ratio and reference samples. However, Actually, it is the size of the defined Scaling Window Offsets of the pictures (current and reference), which defines the scaling ratio between two different pictures and borders of those scaling windows are used to “align” reference samples. This is done to be able to support more flexible use-cases such as Rol scaling (zoom in) or extending the visual field of a picture (zoom out). For instance, the relative distance of a sample of the current picture to the left border of the scaling window is applied (scaled) to the left border of the scaling window of the reference picture to find the reference sample as shown in the equation above and as illustrated in figure 1.

[0014] There are some constraints that have been added to RPR in VVC that aim at avoiding complexity issues related to the memory bandwidth (memory fetched by a current block from the reference picture), which is particularly important when the reference picture is larger than the current picture, and also issues related to quality of the interpolation. Therefore, VVC imposes a constraint that says that a reference picture shall not be larger than 2x the current picture or smaller than 1 / 8x the current picture (note that those ratios apply to the relation of scaling window as explained before).

[0015] RPR can be already performed at the encoder side (and therefore also necessarily at decoder side), if the pictures that are encoded have different resolution or can happen only at the decoder side as a result of a bitstream switch to a different resolution when open GOP switching is carried out (each of the different resolution bitstreams available are generated by an encoder with a single resolution - therefore no RPR used in original bitstream). An issue may arise at switching points for which a couple of pictures may use RPR, when the resampling of pictures to generate the different resolutions use a particular phase in their resampling filters, as explained in the following.

[0016] Typically, in order to generate several resolutions of a particular video the server takes the highest resolution content version and downsamples it to several other resolution versions as shown in figure 2. In order to generate such downsampled versions, typically a well-designed filter is used that avoids artifacts such as aliasing, which consist of a low-pass filter before decimation. One of such could be a Lanczos filter. When applying a downsampling filter different phases might be chosen. For instance, in figure 3 two different phases used for downsampling in horizontal and vertical direction are shown. On the left side the position of the top-left sample of the high-resolution and low-resolution pictures coincide, while on the right side the top-left sample of the low resolution corresponds to a sample shifted by 0.5 sample to the right and to the bottom with respect to the top-left sample.

[0017] When RPR is applied as part of the encoding process, the motion-estimation / motion compensation process may be able to select the best fractional position that is aligned to the used resampling phase and might be able to “correct” any difference by adding the residual. It is important to point out that although, contrary what will be explained in the following, in this case there is no artefacts that appear from such a process, ignoring the phase of the current and reference picture may lead to a non-efficient result as explained later. However, when RPR is only applied at the decoder (e.g., as a result of using open GOP switching in a streaming scenario), the decoder would currently use the same sample as reference irrespective of the used resampling phase, which could lead to unpleasant visual artefact (content shifting / moving / flickering). See an illustration of the drift in the decoding process.

[0018] As mentioned, the phase difference when open GOP resolution switching happens may lead to visual artefacts that would prevent a switch from being seamless. But also, when RPR is applied at the encoder ignoring the phase of each of the picture may lead to inefficient cases. For instance, when carrying out motion compensation the used motion vector, which is result of predicting a motion vector from candidates (spatial neighbouring or temporal co-located block) and adding a motion vector difference (MVD) to the prediction, may point to a reference position corresponding to a correct phase; i.e., the lack of information of the phase used during resampling can be compensated by performing a more in-depth search at the motion compensation stage, i.e. searching for the right reference by adding MVD. This may lead to a larger motion vector differences to compensate for the missing phase information that need to be added to each motion vector prediction of each block instead of allowing a per picture signalling of the phase that would by default add a phase-dependent motion vector difference to the search. In other words, when performing motion vector prediction from spatial / temporal candidates from spatial- neighbour / temporal-colocated blocks ignoring the phase may lead to a wrong / inaccurate prediction. In fact, the reference picture of a spatial-neighbour / temporal-colocated blocks may have a different phase than the phase of the reference picture of the current block. Thus, the wrong / inaccurate prediction requires additional syntax or larger MVDs to correct the predicted motion vector to account for the different phase and thus requires more data to be included into the bitstream. As explained in the following considering that phase during the decoding process can enhance the coding efficiency.

[0019] SUMMARY

[0020] It is the object of the present invention to provide concepts for video codecs supporting motion- compensated inter prediction involving reference picture re-sampling and motion vector scaling which are improved in terms of coding efficiency such as compression rate.

[0021] This is achieved by the subject matter of the independent claims.

[0022] In accordance with a first aspect of the present invention, a video codec which involves reference picture resampling and motion vector scaling is improved in terms of coding efficiency such as compression rate by utilizing comparative resolution information, such as a resolution ratio, associated with a first picture and a second picture to scale a motion vector, which is available in a data stream, for a predetermined block in the first picture, and deriving information on a relative sampling phase offset between respective samples of the two pictures, and taking benefit of this information for performing a motion compensation prediction of the predetermined block in the first picture. In the course of the said motion compensation prediction, the video codec samples the second picture at a motion-compensatedly offset position, this motion-compensatedly offset position obtained by applying the scaled motion vector to a corresponding position in the second picture which is equivalent to a position of the predetermined block in the first picture. The video codec determines the corresponding position based on the information of the relative sampling phase offset, thus making the sampling of the second picture aware of the relative sampling phase offset. By this measure, the video codec takes cognizance of the information of the relative sampling phase, or offset, and achieves an improved prediction in terms of an enhanced tradeoff between coding accuracy and coding performance.

[0023] Accordingly, in accordance with the first aspect of the present invention, a video decoder / encoder decodes / encodes a video from / into a data stream using motion compensation prediction based on motion vectors transmitted in the data stream by scaling a motion vector transmitted in the data stream for a predetermined block of a first picture according to a resolution ratio between the first picture and a second picture to obtain a scaled motion vector, and motion-compensatedly predicting the predetermined block of the first picture by sampling the second picture at a motion- compensatedly offset position resulting from applying the scaled motion vector to a corresponding position in the second picture, corresponding to a position of the predetermined block in the first picture. The video decoder / encoder is configured to derive / insert, from / as side information in the data stream, information on a relative sampling phase offset between first samples of the first picture and second samples of the second picture, and to determine the corresponding position in the second picture using the information on the relative sampling phase offset.

[0024] In accordance with a second aspect of the present invention, a video codec which involves reference picture resampling and motion vector scaling is improved in terms of coding efficiency such as compression rate by utilizing comparative resolution information, such as a resolution ratio, associated with a first picture and a second picture to scale a motion vector, which is available in a data stream, for a predetermined block in the first picture, and deriving information on a relative sampling phase offset between respective samples of the two pictures, and taking benefit of this information for performing a motion compensation prediction of the predetermined block in the first picture. In the course of the said motion compensation prediction, the video codec samples the second picture at a motion-compensatedly offset position, this motion- compensatedly offset position obtained by applying the scaled motion vector to a corresponding position in the second picture which is equivalent to a position of the predetermined block in the first picture. The video codec determines the motion vector based on the information of the relative sampling phase offset, thus making the sampling of the second picture aware of the relative sampling phase offset. By this measure, the video codec takes cognizance of the information of the relative sampling phase, or offset, and achieves an improved prediction in terms of an enhanced trade-off between coding accuracy and coding performance.

[0025] Accordingly, in accordance with the second aspect of the present invention, a video decoder / encoder decodes / encodes a video from / into a data stream using motion compensation prediction based on motion vectors transmitted in the data stream by scaling a motion vector transmitted in the data stream for a predetermined block of a first picture according to a resolution ratio between the first picture and a second picture to obtain a scaled motion vector, and motion- compensatedly predicting the predetermined block of the first picture by sampling the second picture at a motion-compensatedly offset position resulting from applying the scaled motion vector to a corresponding position in the second picture, corresponding to a position of the predetermined block in the first picture. The video decoder / encoder is configured to derive / insert, from / as side information in the data stream, information on a relative sampling phase offset between first samples of the first picture and second samples of the second picture, and to determine the motion vector using the information on the relative sampling phase offset.

[0026] Further embodiments according to the invention are defined by the subject matter of the dependent claims of the present application.

[0027] BRIEF DESCRIPTION OF THE FIGURES

[0028] Embodiments of the present disclosure are described in more detail below with respect to the figures, in which:

[0029] Fig. 1-4 shows representations of existing approaches in the art of intra prediction and reference picture resampling,

[0030] Fig. 5 shows an encoder according to embodiments,

[0031] Fig. 6 shows a decoder according to embodiments,

[0032] Fig. 7 shows a block partitioning according to embodiments,

[0033] Fig. 8 shows an example of the relative sampling phase offset between the first picture and the second picture according to embodiments,

[0034] Fig. 9 shows another example of the relative sampling phase offset between the first picture and the second picture according to embodiments,

[0035] Fig. 10 shows a further example of the relative sampling phase offset between the first picture and the second picture according to embodiments,

[0036] Fig. 11 shows an example of the relative sampling phase offset between multiple resolution versions of a video segment according to embodiments, Fig. 12 shows an example of a motion vector along with its associated first and second picture and further motion vectors along with their reference and current pictures according to embodiments.

[0037] DETAILED DESCRIPTION OF THE FIGURES

[0038] In the above sections, specific embodiments have been described. In the following, further embodiments are described which are based on the above thoughts and ideas, but are broadened. Before, however, a general codec framework is described, into which all embodiments for decoder and encoder described herein could be built into, or with which all these embodiments might be combined.

[0039] Note that in the drawings the same or similar elements or elements that have the same or similar functionality have the same reference signs assigned or are identified with the same name. In the following description, a plurality of details is set forth to provide a thorough explanation of embodiments of the disclosure. However, it will be apparent to one skilled in the art that other embodiments may be implemented without these specific details. In addition, features of the different embodiments described herein may be combined with each other, unless specifically noted otherwise.

[0040] The following description preliminarily switches to a description of the broader or abstracted embodiments. Thereinafter, the description is resumed with the presentation of the more specific embodiments. However, it should be mentioned that any detail(s) described with respect to one embodiment may individually or in combination be used to form a new embodiment when transferred onto a different embodiment. The description starts with a presentation of a description of an encoder and a decoder of a block-based predictive codec for coding pictures of a video in order to form an example for a coding framework into which embodiments of the present invention may be built in. The respective encoder and decoder are described with respect to Fig. 5, Fig. 6, and Fig. 7. Thereinafter the description of the broadened embodiments of the concept of the present invention is presented. While being, as said, combinable with Fig. 5 - 7, the embodiments described before and afterwards may, however, also be used to form encoders and decoders not operating according to the coding framework underlying the encoder and decoder of Fig. 5, and Fig. 6. Fig. 5 shows an apparatus for predictively coding a picture A12 into a data stream A14 exemplarily using transform-based residual coding. The apparatus, or encoder, is indicated using reference sign A10. Fig. 2 shows a corresponding decoder A20, i.e. an apparatus A20 configured to predictively decode the picture 12’ from the data stream A14 also using transform-based residual decoding, wherein the apostrophe has been used to indicate that the picture A12’ as reconstructed by the decoder A20 deviates from picture A12 originally encoded by apparatus A10 in terms of coding loss introduced by a quantization of the prediction residual signal.

[0041] The encoder A10 is configured to subject the prediction residual signal to spatial-to-spectral transformation and to encode the prediction residual signal, thus obtained, into the data stream A14. Likewise, the decoder A20 is configured to decode the prediction residual signal from the data stream A14 and subject the prediction residual signal thus obtained to spectral-to-spatial transformation.

[0042] Internally, the encoder A10 may comprise a prediction residual signal former A22 which generates a prediction residual A24 so as to measure a deviation of a prediction signal A26 from the original signal, i.e. from the picture A12. The prediction residual signal former A22 may, for instance, be a subtractor which subtracts the prediction signal from the original signal, i.e. from the picture A12. The encoder A10 then further comprises a transformer A28 which subjects the prediction residual signal A24 to a spatial-to-spectral transformation to obtain a spectral-domain prediction residual signal A24’ which is then subject to quantization by a quantizer A32, also comprised by the encoder A10. The thus quantized prediction residual signal A24” is coded into bitstream A14. To this end, encoder A10 may optionally comprise an entropy coder A34 which entropy codes the prediction residual signal as transformed and quantized into data stream A14. The prediction signal A26 is generated by a prediction stage A36 of encoder A10 on the basis of the prediction residual signal A24” encoded into, and decodable from, data stream A14. To this end, the prediction stage A36 may internally, as is shown in Fig. 5, comprise a dequantizer A38 which dequantizes prediction residual signal A24” so as to gain spectral-domain prediction residual signal A24’”, which corresponds to signal A24’ except for quantization loss, followed by an inverse transformer A40 which subjects the latter prediction residual signal A24’” to an inverse transformation, i.e. a spectral-to-spatial transformation, to obtain prediction residual signal A24””, which corresponds to the original prediction residual signal A24 except for quantization loss. A combiner A42 of the prediction stage A36 then recombines, such as by addition, the prediction signal A26 and the prediction residual signal A24”” so as to obtain a reconstructed signal A46, i.e. a reconstruction of the original signal A12. Reconstructed signal A46 may correspond to signal A12’. A prediction module A44 of prediction stage A36 then generates the prediction signal A26 on the basis of signal A46 by using, for instance, spatial prediction, i.e. intra-picture prediction, and / or temporal prediction, i.e. inter-picture prediction.

[0043] Likewise, decoder A20, as shown in Fig. 6, may be internally composed of components corresponding to, and interconnected in a manner corresponding to, prediction stage A36. In particular, entropy decoder A50 of decoder A20 may entropy decode the quantized spectral- domain prediction residual signal A24” from the data stream, whereupon dequantizer A52, inverse transformer A54, combiner A56 and prediction module A58, interconnected and cooperating in the manner described above with respect to the modules of prediction stage A36, recover the reconstructed signal on the basis of prediction residual signal A24” so that, as shown in Fig. 6, the output of combiner A56 results in the reconstructed signal, namely picture A12’.

[0044] Although not specifically described above, it is readily clear that the encoder A10 may set some coding parameters including, for instance, prediction modes, motion parameters and the like, according to some optimization scheme such as, for instance, in a manner optimizing some rate and distortion related criterion, i.e. coding cost. For example, encoder A10 and decoder A20 and the corresponding modules A44, A58, respectively, may support different prediction modes such as intra-coding modes and inter-coding modes. The granularity at which encoder and decoder switch between these prediction mode types may correspond to a subdivision of picture A12 and A12’, respectively, into coding segments or coding blocks. In units of these coding segments, for instance, the picture may be subdivided into blocks being intra-coded and blocks being intercoded. Intra-coded blocks are predicted on the basis of a spatial, already coded / decoded neighborhood of the respective block as is outlined in more detail below. Several intra-coding modes may exist and be selected for a respective intra-coded segment including directional or angular intra-coding modes according to which the respective segment is filled by extrapolating the sample values of the neighborhood along a certain direction which is specific for the respective directional intra-coding mode, into the respective intra-coded segment. The intra-coding modes may, for instance, also comprise one or more further modes such as a DC coding mode, according to which the prediction for the respective intra-coded block assigns a DC value to all samples within the respective intra-coded segment, and / or a planar intra-coding mode according to which the prediction of the respective block is approximated or determined to be a spatial distribution of sample values described by a two-dimensional linear function over the sample positions of the respective intra-coded block with driving tilt and offset of the plane defined by the two-dimensional linear function on the basis of the neighboring samples. Compared thereto, inter-coded blocks may be predicted, for instance, temporally. For inter-coded blocks, motion vectors may be signaled within the data stream, the motion vectors indicating the spatial displacement of the portion of a previously coded picture of the video to which picture A12 belongs, at which the previously coded / decoded picture is sampled in order to obtain the prediction signal for the respective inter-coded block. This means, in addition to the residual signal coding comprised by data stream A14, such as the entropy-coded transform coefficient levels representing the quantized spectral-domain prediction residual signal A24”, data stream A14 may have encoded thereinto coding mode parameters for assigning the coding modes to the various blocks, prediction parameters for some of the blocks, such as motion parameters for inter-coded segments, and optional further parameters such as parameters for controlling and signaling the subdivision of picture A12 and A12’, respectively, into the segments. The decoder A20 uses these parameters to subdivide the picture in the same manner as the encoder did, to assign the same prediction modes to the segments, and to perform the same prediction to result in the same prediction signal.

[0045] Fig. 7 illustrates the relationship between the reconstructed signal, i.e. the reconstructed picture A12’, on the one hand, and the combination of the prediction residual signal A24”” as signaled in the data stream A14, and the prediction signal A26, on the other hand. As already denoted above, the combination may be an addition. The prediction signal A26 is illustrated in Fig. 7 as a subdivision of the picture area into intra-coded blocks which are illustratively indicated using hatching, and inter-coded blocks which are illustratively indicated not-hatched. The subdivision may be any subdivision, such as a regular subdivision of the picture area into rows and columns of square blocks or non-square blocks, or a multi-tree subdivision of picture A12 from a tree root block into a plurality of leaf blocks of varying size, such as a quadtree subdivision or the like, wherein a mixture thereof is illustrated in Fig. 7 in which the picture area is first subdivided into rows and columns of tree root blocks which are then further subdivided in accordance with a recursive multi-tree subdivisioning into one or more leaf blocks.

[0046] Again, data stream A14 may have an intra-coding mode coded thereinto for intra-coded blocks A80, which assigns one of several supported intra-coding modes to the respective intra-coded block A80. For inter-coded blocks A82, the data stream A14 may have one or more motion parameters coded thereinto. Generally speaking, inter-coded blocks A82 are not restricted to being temporally coded. Alternatively, inter-coded blocks A82 may be any block predicted from previously coded portions beyond the current picture A12 itself, such as previously coded pictures of a video to which picture A12 belongs, or picture of another view or an hierarchically lower layer in the case of encoder and decoder being scalable encoders and decoders, respectively.

[0047] The prediction residual signal A24”” in Fig. 7 is also illustrated as a subdivision of the picture area into blocks A84. These blocks might be called transform blocks in order to distinguish same from the coding blocks A80 and A82. In effect, Fig. 7 illustrates that encoder A10 and decoder A20 may use two different subdivisions of picture A12 and picture A12’, respectively, into blocks, namely one subdivisioning into coding blocks A80 and A82, respectively, and another subdivision into transform blocks A84. Both subdivisions might be the same, i.e. each coding block A80 and A82, may concurrently form a transform block A84, but Fig. 7 illustrates the case where, for instance, a subdivision into transform blocks A84 forms an extension of the subdivision into coding blocks A80, A82 so that any border between two blocks of blocks A80 and A82 overlays a border between two blocks A84, or alternatively speaking each block A80, A82 either coincides with one of the transform blocks A84 or coincides with a cluster of transform blocks A84. However, the subdivisions may also be determined or selected independent from each other so that transform blocks A84 could alternatively cross block borders between blocks A80, A82. As far as the subdivision into transform blocks A84 is concerned, similar statements are thus true as those brought forward with respect to the subdivision into blocks A80, A82, i.e. the blocks A84 may be the result of a regular subdivision of picture area into blocks (with or without arrangement into rows and columns), the result of a recursive multi-tree subdivisioning of the picture area, or a combination thereof or any other sort of blockation. Just as an aside, it is noted that blocks A80, A82 and A84 are not restricted to being of quadratic, rectangular or any other shape.

[0048] Fig. 7 further illustrates that the combination of the prediction signal A26 and the prediction residual signal A24”” directly results in the reconstructed signal A12’. However, it should be noted that more than one prediction signal A26 may be combined with the prediction residual signal A24”” to result into picture A12’ in accordance with alternative embodiments.

[0049] In Fig. 7, the transform blocks A84 shall have the following significance. Transformer A28 and inverse transformer A54 perform their transformations in units of these transform blocks A84. For instance, many codecs use some sort of DST or DCT for all transform blocks A84. Some codecs allow for skipping the transformation so that, for some of the transform blocks A84, the prediction residual signal is coded in the spatial domain directly. However, in accordance with embodiments described below, encoder A10 and decoder A20 are configured in such a manner that they support several transforms. For example, the transforms supported by encoder A10 and decoder A20 could comprise: o DCT-II (or DCT-III), where DCT stands for Discrete Cosine Transform o DST-IV, where DST stands for Discrete Sine Transform o DCT-IV o DST-VII o Identity Transformation (IT)

[0050] Naturally, while transformer A28 would support all of the forward transform versions of these transforms, the decoder A20 or inverse transformer A54 would support the corresponding backward or inverse versions thereof: o Inverse DCT-II (or inverse DCT-III) o Inverse DST-IV o Inverse DCT-IV o Inverse DST-VII o Identity Transformation (IT)

[0051] The subsequent description provides more details on which transforms could be supported by encoder A10 and decoder A20. In any case, it should be noted that the set of supported transforms may comprise merely one transform such as one spectral-to-spatial or spatial-to- spectral transform.

[0052] As already outlined above, Fig. 5, Fig. 6 and Fig. 7 have been presented as an example where the inventive concept described further below may be implemented in order to form specific examples for encoders and decoders according to the present application. Insofar, the encoder and decoder of Fig. 5, and Fig. 6, respectively, may represent possible implementations of the encoders and decoders described herein below. Fig. 5, and Fig. 6 are, however, only examples. An encoder according to embodiments of the present application may, however, perform encoding of a picture 12 using the concept outlined in more detail below and being different from the encoder of Fig. 5 such as, for instance, in that same is no video encoder, but a still picture encoder, in that same does not support inter-prediction, or in that the sub-division into blocks A80 is performed in a manner different than exemplified in Fig. 7. Likewise, decoders according to embodiments of the present application may perform block-based decoding of picture A12’ from data stream A14 using the coding concept further outlined below, but may differ, for instance, from the decoder A20 of Fig. 6 in that same is no video decoder, but a still picture decoder, in that same does not support intra-prediction, or in that same sub-divides picture A12’ into blocks in a manner different than described with respect to Fig. 7, for instance.

[0053] As illustrated in Fig. 6, decoder A20 may further comprise a filtering module A62, which filters the reconstructed signal A12’, the prediction A58 being performed based on the filtered reconstructed signal A12’. Similarly, encoder A10 of Fig. 5 may comprise a filtering module A62, which may perform the same filtering as filtering module A62 of decoder A20, in the prediction stage A36 to filter the reconstructed signal A46. As the filtering is performed in the prediction loop provided by prediction stage A36 (e.g., in combination with operator A22, transformer A28, and quantizer A32), the filtering by filtering module A62 and / or filtering module A62’ may be referred to as inloop filtering. Accordingly, embodiments of the invention may optionally be implemented as described with respect to Fig. 5, 6, and 7.

[0054] In the following, embodiments of the invention are described, which may optionally be implemented as described with respect to Fig. 5, Fig. 6, and / or Fig. 7, wherein the features described above may be combined with the embodiments described below individually or in combination with each other. For instance, in-loop filtering A62 might be missing. Further, ... might not be used, or, vice versa, intra-prediction.

[0055] Fig. 8 shows a first picture and a second picture in accordance with embodiments of the present invention. The first picture and the second picture belong to a video fitting a video codec in accordance with embodiments of the invention. For instance, the first picture may be referred to as a current picture and the second picture may be referred to as the reference picture, wherein current implies that the first picture may be a picture relevant for, or in consideration of, decoding within a decoding order whereas reference implies that the the second picture may serve as a basis for prediction of parts or a whole of the first picture.

[0056] The first samples 1061 of the first picture denoted by smaller circles and the second samples 1062 of the second picture denoted by larger circles are shown in Fig. 8. In Fig. 8, the first picture and the second picture are shown overlaid on each other. They are shown in am manner so that they registered to each other in terms of picture content meaning that, if no scene change took place between the pictures, the picture content would be the same. As can be seen in the Fig. 8, the samples IO61, IO62 (shown as rectangular grids) clearly differ in their positions. In particular, in Fig. 8, the samples IO61, IO62 differ in their respective sample pitch as indicated by different offsets between ones of the first samples IO61 and ones of the second samples IO62. In general terms, the first picture has a first spatial resolution and the second picture has a second spatial resolution, wherein the first spatial resolution and the second spatial resolution are different from each other. This means that the first and the second pictures sample the scene at different resolution, manifesting itself, for instance, in different sizes of the pictures or in their differently resolving details in the scene. The encoder might have chosen the resolution change between the pictures, or the resolution change has been selected by an entity feeding the encoder.

[0057] As shown, the first samples 1061 have a first sample pitch, i.e. a first sample unit, which is denoted by Si. In other words, a spacing between adjacent samples is same everywhere for the first picture and is denoted by Si. Accordingly, although not explicitly shown, the second samples IO62 have a second sample pitch, i.e. a second sample unit, which can be denoted by s2. Note that, according to an alternative, the pitch may vary, however, laterally.

[0058] A predetermined block 102 of the first picture depicted by solid lines between a rectangular grid of the first samples IO61 is shown. For example, the video codec may perform the coding in a block-wise or block-based manner. That is, the codec may subdivide a picture (which could be the first picture and / or the second picture) into one or more blocks, in units of which the codec may code the picture into / from the data stream. Generally, the subdivision may end up into one or more blocks of constant size such as an array of blocks arranged in rows and columns or into one or more blocks of different block sizes such as derived by use of a hierarchical multi-tree subdivisioning applied to the whole picture area of the picture, or derived from a pre-partitioning of picture into an array of tree blocks which are then subjected to multi-tree subdivisioning. The hierarchical subdivision information may be signalled in the data stream. Further reference is made to the above described examples of Fig. 5 to 7. It is noted that these examples shall not be considered as excluding other possible approaches of subdivisioning the picture into one or more blocks. Thus, the predetermined block 102 which the video codes may be formed from the picture using such or similar approaches.

[0059] Note that the predetermined block 102 may by a quadratic block or a rectangular or non-square block, while any other shape may be used as well.

[0060] The video decoder decodes the video from the data stream by making use of motion vectors transmitted in the data stream to perform motion compensation prediction. A motion vector 100, for the predetermined block 102 is shown as an arrow pointing from a position 104 of the predetermined block 102 in the first picture. The video decoder may select the motion vector 100 among the motion vectors transmitted in the data stream by decoding motion vector selection information, such as one or more indices thereof, signalled in the data stream. Alternatively, the motion vector 100 is signalled in the data stream for the block 102 explicitly, i.e. without motion vector prediction.

[0061] As can be seen, the arrow for the motion vector 100 points from a top left corner position 104 of the predetermined block 102 in the first picture. That is, its foot is positioned at that position 104. The video decoder performs a scaling of the motion vector 100 according to a resolution ratio (e.g. a size ratio) between the first picture and the second picture in order to obtain a scaled motion vector 110. The scaling may be performed so that the scaled motion vector may be larger or smaller in magnitude than the motion vector 100 depending on the resolution ratio. Essentially, the scaling may relate to any linear transformation enlarging or diminishing the motion vector 100 resulting in the scaled motion vector 110. In even other words, for instance, the scaled motion vector 110 is a result of transforming the motion vector 100 in proportion of the spatial resolutions of the second and the first picture.

[0062] For instance, the resolution ratio between the first picture and the second picture may be related to their respective sizes or spatial resolutions or visual details. The resolution ratio may, possibly, be determined using a ratio of the first spatial resolution and the second spatial resolution.

[0063] The video decoder predicts the predetermined block 102 of the first picture in a motion compensated manner by sampling the second picture in a motion-compensatedly offset position, the motion compensatedly offset position resulting from applying the scaled motion vector 110 to a corresponding position 108 in the second picture corresponding to the position 104 in the first picture. In other words, the motion-compensatedly offset position is obtained by way of indication by the scaled motion vector 110 when used at the corresponding position 108 in the second picture.

[0064] For instance, the video decoder may motion-compensatedly predict a sample filling of the predetermined block 102, wherein the sample filling comprises a portion (such as a sub sample filling) or a whole of the predetermined block 102.

[0065] Although the motion vector 100 already indicates per se a translatory displacement of the predetermined block 102 at which the second picture is to be sampled in order to yield the motion- compensatedly offset position, it only does so correctly when the first picture and the second picture have the same size or spatial resolution; therefore, when the first picture and the second picture have different sizes or spatial resolution (or visual detail), the motion vector 100 is scaled 100 to obtain the scaled motion vector 110 which subsequently indicates the motion compensatedly offset position in the second picture at which it is to be sampled by the video decoder.

[0066] It is to be noted that the scaled motion vector 110 and the motion vector 100 (i.e. the motion vector prior to and post scaling) are depicted using the same arrow in Fig. 8 implying that both the motion vectors 100, 110 may have different respective vector component values, because they relate to differently sampled pictures, but they are equally long and point into the same direction in Fig. 8 because of the chosen overlay, registered in terms of scene content. Note that the scaling ratio between the pictures might differ in x and y, or equivalently in horizontal and vertical directions (or components), as described further below, so that the motion vector representations of vectors 100 and 110 might differ in ratio between the two vector components. It is also to be noted that Fig. 8 shows the position 104 in the first picture and the corresponding position 108 in the second picture are being located at exactly the same position of the predetermined block 102, thereby indicating that both positions refer, or relate, to the same positions within the predetermined block 102 in their respective pictures. In the case shown in Fig. 8, this same position of the block 102 corresponds to a top left corner position. This means that, for the first picture, the motion vector 100 points from, or is applied to, the position 104 (e.g. a top-left corner) of the predetermined block 102, while for the second picture the scaled motion vector 110, obtained by the video decoder, points from, or is applied to, the corresponding position 108 (e.g. a top-left corner) of the predetermined block 102. It is emphasised that the apparent lack of a difference between these positions 104, 108 as shown in Fig. 8 is a consequence of the first and the second pictures being overlaid on each other and, further, is a simplification of the possibility that, due to some granularity or coarseness of the subsequently discussed relative sampling phase offset information, the positions 104 / 108 might, in fact, be slightly offset to each other.

[0067] The video decoder further derives, from the side information provided in the data stream, information on a relative sampling phase offset 120 between the first samples 1061 of the first picture and the second samples 1062of the second picture. For instance, this information on the relative sampling phase offset 120 may be derived at a phase offset unit which is finer than, or as fine as, a minimum of the first sample pitch Si and the second sample pitch s2. That is, the information associated with the relative sampling phase offset 120 is derived with a granularity which is smaller than or equal to that of one associated with smaller among the first sample pitch Si and the second sample pitch S2.

[0068] The video decoder further determines the corresponding position 108 in the second picture, to which the scaled motion vector 110 is applied yielding the motion-compensatedly offset position for sampling, by making use of the information on the relative sampling phase offset 120. By this measure, sampling the second picture at the motion-compensatedly offset position, pointed to by the scaled motion vector 110, allows the motion compensated prediction of the predetermined block 120 to take into account the information on the relative sampling phase offset arising due to the different sizes or spatial resolutions of the first and the second pictures.

[0069] The information associated with the relative sampling phase offset 120 can be specified in a number of different ways, such specific instantiations are provided using different embodiments further in the present application. Additionally, fig. 8 exemplarily shows the relative sampling phase offset 120 as an arrow between a first sample and a second sample at an upper left picture corner 126, details of this particular embodiment associated with the relative sampling phase offset are provided further in this disclosure.

[0070] Figure 8 also shows a first scaling window 112i in the first picture depicted using a solid rectangle and a second scaling window 1122 in the second picture depicted using a dashed rectangle. In accordance with embodiments, the video decoder may: decode scaling window information from the data stream, wherein the scaling window information, provided to the decoder by signalization in the data stream, defines a position of the first scaling window 112i in the first picture and a position of the second scaling window 1122 in the second picture, derive the resolution ratio between the first and the second picture based on a size ratio between the first scaling window and the second scaling window, and determine the corresponding position 108 so that the corresponding position 108 is offset from a co-located position 116 which is, relative to the second scaling window 1122, positioned just as the position 104 of the predetermined block 102 is located relative to the first scaling window 112i, according to the information on the relative sampling phase offset 120. For instance, the amount of offset between the co-located position 116 and the corresponding position 108 may be determined on the basis of the information on the relative sampling phase offset 120.

[0071] For example, the size ratio, denoted by ratiox, can be defined as ratiox= hSW2 / hSwi with hSW2 measuring a height of the second scaling window 1122 in samples IO62 (or the second sample pitch of samples IO62) and hswimeasuring a height of the first scaling window 112i in units of samples IO61 (or the first sample pitch of samples IO61), as shown in Fig. 8, so that mv2 = ratiox* mvi wherein mvi is the motion vector 100 given in units of samples IO61 and mv2 is the scaled motion vector 110 given in units samples IO62. Thus, the scaled motion vector 110, mv2, may be obtained by scaling the motion vector 100, mvi, with the resolution ratio expressed as the size ratio, ratiox, of the scaling windows.

[0072] For example, as shown in Fig. 8, Ay2measures a vertical distance of the co-located position 116 from an upper border of the second scaling window 1122and can be derived using Ayi / hswi= Ay2 / hSW2 with Ayi measuring the vertical distance of position 104 in units of the first samples IO61 (or the first sample pitch of the first samples IO61) from an upper border of window 112i. Although the distances of the co-located position 116 and the position 104 have exemplarily been defined here using the upper border of their respective scaling windows, other choices for defining Ay2 and Ayi are entirely feasible, such as measuring these distances from the lower border of their respective scaling windows. Additionally, or alternatively, these distances may be defined horizontally.

[0073] In accordance with embodiments, the video decoder may derive, from the side information in the data stream, the information on the relative sampling phase offset 120 in a form decomposed horizontally and vertically. It is to be noted that the terms horizontal and vertical may be defined along dimensions of the pictures (the first and / or the second), or of the predetermined block, or of scaling windows, or of cropping windows.

[0074] It suffices to state here that the borders of scaling windows (or the cropping windows) might provide an anchor, a fixed reference, using which positions of the predetermined block and other artefacts of the coding process located within the respective scaling windows (or the cropping windows) could be measured relative to.

[0075] In accordance with embodiments, the video decoder may: decode, from the side information signalled or provided in the data stream, one or more of a first phase information and a second phase information, and determine the corresponding position 108 in the second picture using the one or more of the first phase information and the second phase information, wherein the first phase and the second phase are indicative of the relative sampling phase offset 120. This means that either of the first phase and the second phase or both of the phases may allow the computation of the relative sampling phase offset 120. In other words, the first and the second phase may be involved in a computation of the relative sampling phase offset 120. In particular, the first phase information may indicate a first phase at which the first samples 1061 of the first picture sample the first picture, and the second phase information indicate a second phase at which the second samples IO62 of the second picture sample the second picture.

[0076] For example, assuming there exists a fixed predetermined phase and picture parameter set, PPS, for a certain picture comprising phase information only in case of deviating from the fixed predetermined phase; the absence of phase information in the PPS could, for instance, be deduced from the fact that the picture of the PPS has a “full” (highest resolution) size. This is equally applicable to either the first picture or the second picture. That is, only one among the first and the second phase information may be needed, thus, describing the case in the previously described embodiment making use of only one of the first phase information and the second phase information.

[0077] For example, the first phase, denoted by qn, may be measured relative to, for instance, the upper / left border of the picture content, illustrated in Fig. 8 as an eye. It is to be noted that due to camera / eye movement, light change and noise, the picture content of the two pictures, i.e. the first picture and the second picture, might slightly differ from each other, possibly contributing to the relative sampling phase offset 120. It is also to be noted that <pi may be used to indicate the first phase as a fraction between 0 and 1 of the first sample pitch. For example, the second phase is denoted by q>2. In regard to the embodiments according to Fig. 8, for instance, the first phase <pi is non-zero and the second phase q>2 is zero, i.e. q>2 = 0.

[0078] In accordance with embodiments, in addition to details of the previously described embodiment, the video decoder may decode the one or more of the first phase information and the second phase information from the data stream in addition to the scaling window information associated with the first scaling window 112i and the second scaling window 1122, as shown in Fig. 8 and described previously in the present application. The video decoder may then derive the resolution ratio between the first and second pictures based on a size ratio between the first scaling window and the second scaling window. The video decoder may further derive information on, or even directly derive, the relative sampling phase offset 120, as indicated by the first and the second phase and additionally determine the corresponding position 108 so that it is offset from the colocated position 116 according to the derived information on, or directly derived, the relative sampling phase offset 120. In accordance with embodiments, in the course of decoding the scaling window information, the video decoder may decode the position of the first scaling window 112i in the first picture represented in units of the first samples 1061, or equivalently the first sample pitch, and the position of the second scaling window 1122 in the second picture represented in units of the second samples IO62, or equivalently the second sample pitch IO62. In other words, the positions of the scaling windows 112i, 1122 in the respective pictures may be expressed in multiples of, or in increments / decrements of, the respective samples (e.g. having the respective sample pitches).

[0079] In accordance with embodiments, the video decoder may decode, from the side information in the data stream, as a measure of the first phase, a first phase offset length, or as a measure of the second phase, a second phase offset length. An example of the first phase offset length related to the first phase of the first picture is depicted in Fig. 8. The first phase offset length, as shown in Fig. 8 between two adjacent samples near the upper left corner is merely a fraction of the first sample pitch Si and equals <pi ■ Si , wherein Si is the sample unit, or the first sample pitch, of the first samples 1061. It is to be noted that the phase offset lengths defined here lack the word “relative” since the first (or the second) phase offset length determines the (first) phase (or phase offset or displacement) absolutely relative to some picture content.

[0080] In accordance with embodiments, and in accordance with Fig. 8, the first phase offset length or the second phase offset length may be measured relative to an upper left picture corner, an upper left scaling window or an upper left cropping window corner. Alternatively, the first phase offset length or the second phase offset length may be measured relative to other positions of the picture, or of the scaling window, or of the cropping window, such as any of the other corners or edges or borders thereof.

[0081] In accordance with embodiments, the video decoder may decode the first phase offset length or the second phase offset length from the data stream in predetermined reference units. In other words, in embodiments, each picture is associated with a phase relative to a reference resolution; although it could be simply a phase relative to an arbitrary reference. For example, the predetermined reference unit, indicative of or associated with the reference resolution, may be provided or made available or signaled in the data stream. It is emphasized here that the term reference resolution refers to a resolution relative to which the phase lengths or phase information may be defined, it should not be confused with a reference picture (a picture, and blocks thereof, used to form predictions for blocks of the current picture) or a resolution thereof. In accordance with embodiments, the video decoder may support a plurality of sample resolutions, with each picture of the video being of one of the plurality of sample resolutions, in the motion-compensatedly prediction of the predetermined block 102 of the first picture, perform the sampling of the second picture at the motion-compensatedly offset position resulting from applying the scaled motion vector 110 to a corresponding position 108 in the second picture, corresponding to a position 104 of the predetermined block 102 in the first picture, at a resolution which is x times finer than a sample resolution of the second picture, wherein the predetermined reference units correspond to a resolution which is between being as fine as, and being x times finer than a finest sample resolution among the plurality of sample resolutions. In other words: length, in units of the sample pitch, of the highest-resolution pictures down to 1 / xthof this sample pitch, with x denoting the maximum motion vector accuracy (e.g. as given by motion compensation interpolation filter tap pitch). This means that each picture is associated with a phase relative to a reference resolution, wherein the reference resolution is one finest or most granular or biggest of the (available) resolutions of the pictures. For example, the video decoder may support the plurality of sample resolutions, wherein information associated with the plurality of sample resolutions could be signaled in the data stream. For example, a list of the plurality of sample resolutions supported by the decoder could be signaled in the sequence parameter set, SPS, and one or more indices related to the resolutions could be signaled in the picture parameter set, PPS, specifying the resolutions corresponding to the pictures.

[0082] In accordance with embodiments, the video decoder may support the plurality of sample resolutions, with each picture of the video being of one of the plurality of sample resolutions, wherein the predetermined reference units correspond to a resolution which is between being as fine as, and being a finest sample resolution among the plurality of sample resolutions. That is, for example, the predetermined reference units, associated with the reference resolution, may corresponds to a range of resolutions spanned by a resolution which is as granular, or sharp or detailed, as the finest resolution among the plurality of sample resolutions and a resolutions which is even more granular, or even shaper, or even more detailed, than the finest resolution among the plurality of sample resolutions.

[0083] In accordance with embodiments, the video decode may decode, from the side information in the data stream, as a measure of the first phase, a first numerator and / or a first denominator of a first fraction representing the first phase, or as a measure of the second phase, a second numerator and a second denominator of a second fraction representing the second phase. That is, the first phase may be expressed as a fraction with a first numerator and / or a first denominator and the second phase may be expressed as a fraction with a first numerator and / or a first denominator, both sets of numerator and denominators may be signaled in the data stream. For example, it is also feasible that a predetermined numerator (or predetermined denominator) is agreed between the encoder and the decoder, and signaled in the data stream, as a result of which only the first and the second denominators (or the first and the second numerators) may additionally be needed to express / represent the respective fractions associated with the first and the second phases.

[0084] For example, if <pi ■ si is the first phase offset length measuring, for instance, the first phase against the upper left corner of the picture, as shown in Fig. 8, with Si being the sample unit (or the first sample pitch) of the first samples 106i, then <pi = numerator / denominator. For example, the first and the second phase may exemplarily represented for the horizontal direction (a horizontal component of the relative sampling phase offset, or equivalently the relative sampling phase offset in the horizontal direction) and under the assumption that the signalled phase numerator and denominator, i.e. the first and second numerator, and the first and the second denominator, are in units of full samples in the “resolution” or fidelity of the current picture. This means, in accordance with some embodiments, that the numerators and denominators are allowed in increments / decrements of the first sample pitch, or first samples 1061 in the first picture, but not any smaller than the first sample pitch.

[0085] In accordance with embodiments, the first numerator and / or the first denominator are measured in units of first samples (cpi = first numerator [si] I first denominator [s-i]) and / or the second numerator and the second denominator are measured in units of second samples (cp2 = second numerator [S2] I second denominator [S2]). For example, as shown in Fig. 8, the first phase may be indicated in the horizontal direction as a phase shift to the right of pps_phase_offset_hor_num I (pps phase offset hor den minusl +1) samples of the top-left sample / corner 126 of the first picture, wherein ‘pps_phase_offset_hor_num’ is the first numerator and ‘pps_phase_offset_hor_den_minusl’ is the first denominator associated with a fraction representation of the first phase.

[0086] In accordance with embodiments, the first numerator and / or the first denominator are measured in units of predetermined reference units and / or the second numerator and the second denominator are measured in predetermined reference units. That is, the predetermined reference units in terms of which the two sets of numerators and denominators may be represented, or expressed may be chosen arbitrarily. Further, different arbitrary predetermined reference units could be chosen for the first numerator and the first denominator, and for the second numerator and the second denominator.

[0087] In accordance with embodiments, in addition to the previously described embodiment, the video decoder may support a plurality of sample resolutions, with each picture of the video being of one of the plurality of sample resolutions, in the motion-compensatedly predict the predetermined block 102 of the first picture, perform the sampling the second picture at the motion- compensatedly offset position resulting from applying the scaled motion vector 110 to a corresponding position 108 in the second picture, corresponding to a position 104 of the predetermined block 102 in the first picture, at a resolution which is x times finer than a sample resolution of the second picture, wherein the predetermined reference units correspond to a resolution which is between being as fine as, and being x times finer than a finest sample resolution among the plurality of sample resolutions.

[0088] That is, for example, the predetermined reference units, associated with the reference resolution wherein the two numerators and denominators are measured in units of the predetermined reference units, may correspond to a range of resolutions spanned by a resolution which is as granular, or sharp or detailed, as the finest resolution among the plurality of sample resolutions and a resolution which is even more granular, or even shaper, or even more detailed, than the finest resolution among the plurality of sample resolutions. In other words, in accordance with this embodiment, each picture is associated with a phase relative to the (arbitrary) reference resolution (typically the highest resolution present / original resolution) expressed in terms of the predetermined reference units wherein these units could be at least as detailed as a most detailed resolution among the available sample resolutions or could at most be x times more detailed than the most detailed resolution.

[0089] Further, in accordance with embodiments, and especially in addition to the previous embodiment, the predetermined reference units correspond to a resolution which is between being as fine as, and being, a finest sample resolution among the plurality of sample resolutions.

[0090] In accordance with embodiments, the first numerator and / or the first denominator are signaled in the data stream at a predetermined representation accuracy and / or the second numerator and / or the second denominator are signaled in the data stream at a predetermined representation accuracy. For example, the first phase may given by <pi = first numerator / first denominator with the first numerator and the first denominator being signaled, for instance, in x bit-resolution wherein x might be equal for numerator and denominator or different between both.

[0091] In accordance with embodiments, the predetermined representation accuracy may be signaled in the data stream.

[0092] In accordance with embodiments, the video decoder may decode, from the side information in the data stream, a common denominator, as a measure of the first phase, a first numerator forming, along with the common denominator, a first fraction representing the first phase, or as a measure of the second phase, a second numerator forming, along with the common denominator), a second fraction representing the second phase. That is, the two fractions representing different phases (the first phase and the second phase) may share a common denominator (i.e. the denominators all share a common value).

[0093] For example, a common value for the phase offset denominators (denominators associated with measures of the phase information) could be signalled at bitstream or sequence level that suffices for all resolution versions, for example when the decoder supports a plurality of sample resolutions and each picture in the video being one of the plurality of sample resolutions.

[0094] In accordance with embodiments, the video decoder may decode, from the side information in the data stream, the common denominator, at a scope covering the whole data stream or a sequence of pictures including the first and second pictures. In other words, a common value for the phase offset denominators (the first denominator of the first fraction representing the first phase and the second denominator of the second fraction representing the second phase) could be signalled at bitstream or sequence level that suffices for all resolution versions. That is, the common denominator may be used for the entirety of the data stream and for the plurality of sample resolutions for the sequence of pictures, including the first picture and the second picture. For example, vps / sps_phase_offset_hor / ver_den_minus1 may be the common denominator and may be signalled in the VPS or SPS. Alternatively, for example, a sufficiently high constant value denominator could be used in the derivation without signalling, e.g. using 232as constant denominator. That is, a predetermined value of the common denominator could be used without signalling, wherein the predetermined value of the common denominator is a sufficiently high constant value and is agreed between the encoder and the decoder.

[0095] Alternatively, sets of such horizontal and vertical phase information (i.e. the phase information represented as fractions with numerators and denominator decomposed in horizontal and vertical directions) could be signalled at bitstream or sequence level, e.g. VPS or SPS, in an indexable fashion.

[0096] In accordance with embodiments, the video decoder may decode, from the side information in the data stream, the first phase information, and the second phase information, and in determining the corresponding position 108 in the second picture using the information on the relative sampling phase offset 120, check whether the first phase equals the second phase, if the first phase is unequal to the second phase, determining the corresponding position 108 in the second picture by using the first phase and the second phase, and if the first phase equals the second phase, determining the corresponding position 108 in the second picture without using the first phase and without using the second phase. Thus, for instance, at the decoder side, as part of the embodiment, the phases of the first picture (the current picture) and the second picture (the reference picture) are compared and the relative phase is computed (e.g., a phase difference), which then is used to determine the sample position in fractional accuracy that leads to the proper reference, to avoid the problem illustrated in figure 9. By this measure, as per this embodiment, for instance, the video decoder decides to utilize both the first and the second phase information when the first and the second phases are equal and to not utilize any of the phases when they are equal to each other and therefore, the decoder can determine the corresponding position 108 in the second picture by exploiting the information on the relative sampling phase offset 120.

[0097] In accordance with embodiments, the video decoder may decode, as part of each phase information of the one or more of the first phase information and the second phase information, a zeroness flag indicative of the respective phase information being indicative of a zero phase, followed by, if the zeroness flag does not indicate a zero phase, one or more syntax elements indicative of the respective phase. By this measure, the video decoder may be equipped to ascertain a vanishing, or zero, value of either of the respective phase offset lengths or respective phases associated with the respective phase information depending on a value of the respective zeroness flag and perform a decoding of the relevant one or more syntax elements if required. For example, the first phase information and the second phase information may have an equal number, count, of the syntax elements associated to them. This number, or count, may vary depending on a precision, or accuracy, required for determining the respective phase information.

[0098] For instance, the presence of such a phase (e.g. the first phase and / or the second phase) can also be conditioned to a gating flag (i.e. the zeroness flag) so that it is only signalled for the case that the phase is not 0, as shown in the following instantiation of the embodiment. Table 1 shows an example of how the zeroness flag could be implemented in a syntax structure such as PPS.

[0099] In particular, table 1 shows the PPS syntax accompanied by the respective semantics.

[0100] Table 1

[0101] The zeroness flag, for instance denoted by pps_phase_offset_signalling_flag herein, in the case it is equal to 1 specifies that the one or more syntax elements such as pps_phase_offset_hor_num, pps phase offset hor den minusl, pps phase offset ver num and pps_phase_offset_ver_den_minusl indicate the information on the relative sampling phase offset 120 such as the first phase information and the second phase information decomposed in horizontal and vertical directions and wherein each of the phase information is represented by a fraction using a respective numerator and a respective denominator. It is shown in Table 1 that the zeroness flag, e.g. pps_phase_offset_signalling_flag, coded as u(1), is parsed as an unsigned integer using 1 , or single, bit whereas the one or more syntax elements, coded as ue(v), are parsed as an unsigned integer O-th order Exp-Golomb coded syntax element using a variable number of bits, wherein the number of bits may vary in a manner dependent on values of other syntax elements within the same syntax structure, or belonging to different syntax structures.

[0102] Additionally, or alternatively, in accordance with embodiments, the side information may be contained in picture parameter sets, PPS, a picture header or a slice header contained in the data stream for the first and / or second picture. For example, the first phase information may be contained in the PPS (or picture or slice header) of the first picture and the second phase information in the PPS (or picture or slice header) of the second picture. It should be noted that the instantiation of the previous embodiment as described by the PPS syntax structure in Table 1 , instead of being included into the PPS it could be included in the bitstream, or the data stream, at any other syntax structure such as a Picture header, or slice header for instance.

[0103] According to embodiments, the first phase and the second phase may be measured relative to a upper left picture corner or an upper left scaling window corner or an upper left cropping window corner, in the cases where a scaling window or a cropping window is defined. For example, the first and / or the second phase may be measured relative to a top-left sample of the picture or of the scaling window or of the cropping window and relative to what would mean a phase 0. For instance, the reference sample, or a sample in the second picture, would require computing the difference shifting of the phase (the first phase and / or the second phase; or generally, the information on the relative sampling phase offset 120) as follows:

[0104] Phase OffsetHorcurr= pps_phase_offset_hor_numCUrr / (pps_phase_offset_hor_den_minusl curr +1) (3)

[0105] Phase OffsetHorRef= pps_phase_offset_hor_numref / (pps_phase_offset_hor_den_minuslref +1) (4)

[0106] PhaseOffsetHordiff = Phase OffsetHorcurr* scalingRatioHor - Phase OffsetHorRef (5) with scalingRatioHor being a resampling ratio (for instance, being different to ScalingRatio[0] which has a precision of 214wherein ScalingRatio[0] may be the resolution ratio between the first and the second picture decomposed in the horizontal direction), PhaseOffsetHorCurr being the phase offset of the current block (current picture) i.e. the first phase associated with the first phase information of the predetermined block 102 in the first picture, PhaseOffsetHorRefbeing the phase offset of the reference block (reference picture) i.e. the second phase associated with the second phase information of the corresponding predetermined block in the seond picture and where PhaseOffsetHordiff is subsequently used to determine the fractional position of the reference sample in the reference picture i.e. a fractional part of the corresponding position 108 in the second picture.

[0107] Thus, in accordance with embodiments and as exemplarily expressed by equation (5), the video decoder in determining the relative sampling phase offset 120, subtracts from the first phase, scaled according to the resolution ratio, the second phase. Thus, for example, the relative sampling phase offset 120 may be (e.g. reduced to) the phase difference of the first and the second phase.

[0108] For example, the shift, (i.e. the phase difference PhaseOffsetHordiff), or the relative sampling phase offset 120 would be converted to the same factional resolution as the interpolation filter (e.g., 1 / 16 sample in VVC) and added to the fractional position determined by the MVs. Thus, in accordance with embodiments, the video decoder may determine the relative sampling phase offset 120, by using the first phase, scaled according to the resolution ratio, and the second phase, unsealed, or using the first phase according to a first sampling pitch of the first samples 1061 and the second phase scaled according to a second sampling pitch of the second samples IO62.

[0109] Alternatively, in accordance with embodiments, the current picture, i.e. the first picture, could directly indicate the relative phase offset (i.e. the relative sampling phase offset 120) with respect to the reference picture, i.e. the second picture, without requiring the additional signalling of the phase offset of each picture (current and ref, or equivalently first and second respectively).

[0110] Alternatively (but equivalently), this i.e. the relative sampling phase offset 120 for the first picture with respect to the second picture, could be added to the equations (1) and (2) with the following changes as shown below given in equations (6) and (7): refxSbi, = ( ( ( xSb + PhaseOffsetHorcurr>( ScalingWinLeftOffsetLuma ) ) « 4 ) + refMvLXf 0 ] ) * scalingRatiof 0 ] (6) refxL= ( ( Sign( refxSbi, ) * ( ( Abs( refxSbi, ) + 128 ) » 8 ) +

[0111] XL * ( ( scalingRatiof 0 ] + 8 ) » 4 ) ) + fRefLeftOffset - PhaseOffsetHorRef + 32 ) » 6 (7)

[0112] As mentioned above, the phase information (i.e. the information on the relative sampling phase offset 120) can be given for each picture in units of samples anchored to the top-left sample of that respective picture or, in other cases anchored to the top-left sample of the scaling window of that respective picture.

[0113] When resampled pictures, i.e. pictures resulting from sampling of the second picture at the motion-compensatedly offset position, are not cropped out, the simple solution relating to indicating the phase information (i.e. the first phase or the first phase information or generally, the information on the relative sampling phase offset 120) for the top-left sample is enough as any further sample position or relation among sample positions of different pictures (i.e. respective equivalents in the different pictures of the corresponding position 108 in the second picture) can be simply computed using the relative sampling phase offset 120 (derived from the difference phase information for each resolution picture) and the distance of each sample to the top left sample as shown in figure 9. However, when part of the pictures , i.e. the first picture or the second picture are cropped out as shown in figure 9, a different phase would be indicated based on the new top-left picture (cropped out samples are not there or discarded and probably unknown or not available to the decoder). This approach is similar to scaling ratios (that is, for instance, the size ratio between the first scaling window and the second scaling window) being computed based on the scaling window, for instance depicted in Fig. 4 as 112i, 1122 instead of being based on picture dimensions as done for zooming in (for example, resample a part of the reference image, i.e. the second picture, by cropping the rest) and zooming out. Scaling windows could define a common anchor for indicating the information related to the relative sampling phase offset 120 (e.g. the first phase information, the second phase information). That is, the information related to the relative sampling phase offset 120 would be indicated for the top-left sample of the scaling window, as aforementioned in one of the embodiments.

[0114] Further, sets of the phase information (i.e. a set of the first phase information and a set of the second phase information) decomposed into horizontal and vertical direction could be signalled in the bitstream or sequence level in an indexable fashion and be subsequently referenced from PPS or PH level. For instance, the described sets could be indexed with an explicit identifier or the applicable resolution (i.e. a predetermined resolution) or dimensions of the cropped picture or uncropped picture (i.e. dimensions of the cropped window and the cropped out parts of the picture) or scaling windows. Additionally, for instance, the described sets could be referenced from PPS or PH level through signalling an explicit pps / ph_phase-offset_info_set_id or through the existing signalling or resolution or dimensions of the cropped or uncropped picture or scaling window for a given coded picture. Thus, in accordance with embodiments, the video decoder may decode the one or more of the first phase information and the second phase information from the data stream by decoding an index into a list of predetermined phases transmitted in the data stream at a scope covering the whole data stream or a sequence of pictures including the first and the second pictures.

[0115] Besides, for instance, instead of defining the horizontal, or vertical, phase by numerators or denominators on scale of each of the resolutions (i.e. the plurality of sample resolutions supported by the decoder described earlier in the application), a reference resolution (e.g. a predetermined resolution) could be signaled to which the phase offsets (e.g. the first phase and the second phase) apply. The difference among both cases is shown in figure 10. For example, in the just described case, the reference resolution (e.g. the predetermined resolution) would need to be signaled in the bitstream, e.g., ref pic width in samples and ref pic height in samples. In other words, the first and the second phase represented in terms of numerators and denominators could be signaled in the data stream at a predetermined representation accuracy, which is itself signaled in the data stream.

[0116] Thus, according to embodiments, the video decoder may decode scaling window information from the data stream, the scaling window information defining the position of the first scaling window (shown as 112i in Fig. 8) in the first picture in units finer than the first samples (e.g. in units of the first sample pitch) and / or defining the position of a second scaling window (shown as 1122in Fig. 8) in the second picture in units finer than the second samples (e.g. in units of the second sample pitch), wherein a sub-sample part of the scaling window information [e.g. the fractional part of the position of the first scaling window 112i when represented in units of the first samples and / or the fractional part of the position of the second scaling window 1122when represented in units of the first samples] is indicative of the relative sampling phase offset 120, the video decoder derives the resolution ratio between the first and second pictures based on a size ratio between the first scaling window 112i and the second scaling window 1122, and determines the corresponding position 108 so that the corresponding position 108 is, relative to the second scaling window, positioned just as the position 104 of the predetermined block 102 is located relative to the first scaling window 112i, according to the information on the relative sampling phase offset 120.

[0117] Further, as explained earlier, in accordance with embodiments, the video decoder may: decode cropping window information from the data stream, the cropping window information defining a position of a first cropping window 112i in the first picture and a position of a second cropping window 1122in the second picture, wherein the cropping window information defines the position of the first cropping window 112i in the first picture in units finer than the second samples, wherein a sub-sample part of the cropping window information is indicative of the relative sampling phase offset 120, and derive the resolution ratio between the first and the second pictures based on a size ratio between the first cropping window 112i and the second cropping window 1122, and determine the corresponding position 108 so that the corresponding position 108 is, relative to the second cropping window 1122, positioned just as the position 104 of the predetermined block 102 is located relative to the first cropping window 112i, according to the information on the relative sampling phase offset 120.

[0118] For example, the size ratio denoted by ratioxcan be defined as ratiox= hSW2 / hSwi with t measuring a height of the cropping window 1122in samples 1062(or the second sample pitch of samples 1062) and hswimeasuring a height of the cropping window 112i in units of samples 1061 (or the first sample pitch of samples 1 O61), as shown in Fig. 8, so that mv2 = ratiox* mvi wherein mvi is the motion vector 100 given in units of samples 1061 and mv2 is the scaled motion vector 110 given in units samples IO62.

[0119] Alternatively, among further variations of the just described embodiment, the information on the relative sampling phase offset 120 could be provided for a top-left sample of a virtual picture before being cropped out (could be considered an original picture). If so, the information on the relative sampling phase offset 120 would be anchored at the same position, or measured relative to a same position within each picture, for each picture. In this case, information would be required to be provided to each picture related to the cropped out dimensions of each of the pictures so that the relative position of samples of a current picture (a respective first picture) and its reference picture(s) (i.e. its respective second picture(s)) can be still properly computed. Therefore, in the just described case, the information on the relative sampling phase offset 120 may comprise also the information provided to each picture relating a set of cropped out dimensions so that respective corresponding position(s), analogous or equivalent to the corresponding position 108 in the second picture, in their respective second picture(s) can still be determined.

[0120] It is emphasised that although the cropping windows and the scaling windows have been shown in accordance with Fig. 8 with the same reference signs, they refer to different windows differing in their functionalities. While the scaling window may be used to zoom in or out as desired for a given picture without discarding the rest of the given picture, the cropping window may simply crop out a specific region of the given picture, essentially not using or discarding the cropped out portion of the given picture. Therefore, for instance, the first scaling window 112i and the first cropping window 112i are depicted by the solid rectangular sample of the picture simply coincide in their dimensions in this example. Further examples, where their respective sizes do not coincide are naturally feasible. It is possible that in embodiments, both the scaling windows and the cropping windows may be used, it is then understandable to persons skilled in the art that in such embodiments, the scaling windows and the cropping windows may be assigned differing reference signs.

[0121] For example, alternatively, when scaling window signalling is absent in a bitstream, i.e. the cropping window coordinates also serve as scaling window coordinates. Hence, an alternative embodiment of the present invention is to signal sub-pel offsets relative to the cropping window. Even further, the boundaries of the whole decoded picture may serve as anchor point for additional sub-pel offsets.

[0122] In the described embodiments, while scaling windows boundaries may be located at sub-pel positions, scaling window dimensions remain full-pel. Alternatively, for example, it could be allowed to use sub-pel scaling window dimension to enable further scaling factor granularity and / or more accurately match the image content of scaling window area across resolution versions by signalling separate sub-pel offsets for the left, right, top and bottom boundary of scaling windows, cropping windows or decoded picture boundaries.

[0123] In particular, embodiments (including described variation and other details) comprising scaling windows described in this disclosure are readily transferable onto embodiments comprising cropping windows so as to be configured to apply concepts related to scaling windows onto cropping windows.

[0124] Further, in accordance with embodiments, details associated with the first and the second phase offset lengths as measures of the first and the second phase such as their resolutions and signalization thereof in the data stream, and details associated with the first phase and the second phase represented as fractions having their respective numerators and denominators and signalization thereof in the data stream - both sets of details having been described earlier in the disclosure, are readily transferable for signalling the position of the scaling windows and / or the cropping windows. Such details are not repeated here for the sake of conciseness and brevity of the disclosure.

[0125] Alternatively, as a further aspect of the embodiment, the phase information could be provided for a top-left sample of a virtual picture before being cropped out (could be considered an original picture). If such, the phase information would be anchored at the same position for each picture. In this case, information would be required to be provided to each picture related to the cropped out dimensions of each of the pictures so that the relative position of samples of a current picture and its reference picture(s) can be still properly computed.

[0126] As a further alternative to some of the already described embodiments, the decoder may compensate for the relative sampling phase offset, or shift, by allowing sub-pel positions of the scaling window boundaries. This could be achieved by signalling the scaling window coordinates using pps scaling win left offset, pps scaling win right offset, pps scaling win top offset, and pps scaling win bottom offset with sub-pel precision or by signalling, in addition to the existing full-pel coordinates, a further sub-pel precision offset, e.g. adding pps_scaling_win_hor_sub_pel_offset and pps_scaling_win_ver_sub_pel_offset, in the PPS at a certain maximum sub-pel precision. An instantiation of such an approach could also involve signalling that the precision of the indicated sub-pel offsets is different from the maximum precision to save signalling bits, e.g. by additionally signalling sub_pel_hor_precision_minusl and sub pel ver precision minusl that specify the precision of the sub-pel offsets, e.g. as specifying the value of n in 1 / (n+2), or 2A-(n+1) or as an index n into a mapping table [1 / 2, 1 / 4, ...].

[0127] Table 2 exemplarily shows the PPS syntax accompanied by the respective semantics in the case where the scaling window coordinates are explicitly signalled in the data stream. For instance, the presence of explicit signalling of the scaling window coordinates could be achieved by the use of an explicit coordinate flag, namely pps scaling window explicit signalling flag. In the case the explicit coordinate flag is equal to 1 , syntax elements (possibly one or more) are specified. For instance, sub pel hor precision minusl and sub pel ver precision minusl are specified, each coded as u(8) indicating that they are to be parsed as unsigned integer using 8 bits and pps scaling win left offset, pps scaling win right offset, pps scaling win top offset, and pps scaling win bottom offset are specified as the scaling window coordinates in sub-pel positions and coded as ue(v) indicating that they are to be parsed as signed integer O-th order Exp_Golomb coded syntax elements using a variable number of bits, wherein values of such syntax elements may be derived using a predetermined mapping process.

[0128] Table 2 Further, additionally or alternatively, in relation to the previously described embodiment as instantiated by T able 2, the presence of sub-pel precision of the scaling window coordinates could be gated, or indicated, by a presence flag. For example, the presence flag is denoted by pps scaling window precision signalling flag in Table 3. In the case the presence flag equals 1 , the sub-pel precision of the scaling window coordinates are provided, or available in the data stream.

[0129] Table 3

[0130] Alternatively, an additional presence flag could be signalled, wherein the additional presence flag is conditioned to indicate whether the scaling window coordinates are to be specified with sub-pel precision. For instance, in accordance with Table 4, the additional presence flag may be implemented as pps scaling window sub pel signalling flag, wherein in the case this additional presence flag equals 1 , then the further sub-precision offset decomposed in horizontal and vertical directions, such as pps_scaling_win_hor_sub_pel_offset and pps_scaling_win_hor_sub_pel_offset, are specified along with their respective precision indicated by sub_pel_hor_precision_minusl and sub pel ver precision minusl. As described in Table 4, the former are coded as signed se(v) using a variable number of bits while the latter are coded as signed integer with 8 bits, in accordance with details of se(v) and u(8) having already been described earlier in the disclosure.

[0131] Table 4

[0132] For all of the above embodiments, signalling can be reduced through using syntax element inference for vertical phase derivation using the values of syntax elements of the horizontal counterparts and vice versa by signalling gating flags, in accordance with embodiments, such as uniform phase shift flag or uniform sub pel precision flag or uniform_scaling_win_sub_pel_offset_flag, e.g. as shown in Tables 5 and 6.

[0133] Table 5 Table 6

[0134] Thus, in accordance with embodiments and as instantiated in Tables 5 and 6, the video decoder may in deriving, from the side information in the data stream, the information on the relative sampling phase offset 120 in the form decomposed horizontally and vertically, be responsive to a gating flag in the side information so as to derive one of a horizontal component and a vertical component of the sampling phase offset 120 from a signalization in the side information, relating to the other of the horizontal component and the vertical component of the sampling phase offset 120, or derive each of the horizontal component and the vertical component of the sampling phase offset 120 from a separate signalization in the side information relating to the respective of the horizontal component and the vertical component of the sampling phase offset 120.

[0135] That is, in Table 5, the information on the relative sampling phase offset 120 is conditioned to the gating flag, denoted by uniform phase shift flag, so as to derive pps phase offset ver num (i.e. numerator of the horizontal component of the relative sampling phase offset 120) and pps_phase_offset_ver_den_minusl (i.e. denominator of the vertical component of the relative sampling phase offset 120), where the former and the latter are the vertical counterparts to the horizontal components of the relative sampling phase offset 120 pps_phase_offset_hor_num and pps_phase_offset_hor_den_minusl. It is to be noted that the vertical components are made available to the decoder in the data stream in the case the gating flag value equals 0.

[0136] Alternatively, in Table 6, two gating flags, namely uniform_sub_pel_precision_flag and uniform scaling win sub pel offset flag, are responsible for specifying, or further detailing, the information on the relative sampling phase offset 120. The first gating flag, denoted by uniform sub pel precision flag, allows the decoder to derive a vertical component of sub-pel precision information related to the relative sampling phase offset 120 (denoted by sub_pel_hor_precision_minusl), wherein the horizontal component (denoted by sub_pel_ver_precision_minusl) is already available (i.e. existing in the scope outside of the conditioning of the first gating flag) in the scope of pps scaling window sub pel signalling flag syntax header. The second gating flag, denoted by uniform_scaling_win_sub_pel_offset_flag, allows the decoder to derive a vertical component of sub-pel precision offset information related to the relative sampling phase offset 120 (denoted by pps_scaling_win_ver_sub_pel_offset), wherein the horizontal component (denoted by pps_scaling_win_hor_sub_pel_offset) is already available (i.e. existing in the scope outside of the conditioning of the first gating flag) in the scope of pps scaling window sub pel signalling flag syntax header.

[0137] In accordance with embodiments, the video decoder may decode the motion vector from the data stream using motion vector prediction. That is, the motion vectors transmitted in the data stream may themselves be subject to prediction, especially but not limited to the case where the second picture (reference picture) lacks motion information of samples comprised within, such as in the case where the second picture is coded in an intra-prediction mode.

[0138] Further, in accordance with embodiments, the video decoder, in decoding the motion vector using motion vector prediction, forms a list of motion vector candidates based on further motion vectors of previously decoded blocks depending on, for a predetermined further motion vector of the further motion vectors of previously decoded blocks, a relative sampling phase offset 120 between reference and source pictures of the predetermined further motion vector. This means that in the course of decoding the motion vector for the first and the second picture, a previously available motion vector, i.e. the predetermined further motion vector, and the corresponding relative sampling phase offset 120 between the reference and source pictures to which the predetermined further motion vectors belongs is used a basis for the motion vector prediction. For instance, a choice of the predetermined further motion vector among the further motion vectors may be signalled in the data stream. For instance, the previously decoded blocks may spatially neighboring or temporally neighbouring to the predetermined block 102 in the first picture. Further details applicable to this embodiment are presented in Figure 12.

[0139] In accordance with embodiments, the information on the relative sampling phase offset 120 such as the phase difference may not happen within a particular resolution but into a layered data stream (or equivalently, a layered bitstream), wherein, for instance, the layered data stream comprises a plurality of layers and each layer of the plurality of layers forms a coded data stream. For example, the plurality of layers may be coded at different spatial resolutions, or visual qualities (or video fidelities). In other words, the information on the relative sampling phase offset 120 such as the phase difference may be defined between a resolution associated with a first layer and a resolution associated with a second layer. In particular, providing different resolutions into different layers and the prediction from one layer to another would then take into account the phase of each of the layers for applying inter-layer prediction. Thus, as per embodiments, the first picture may belong to the first layer and the second picture may belong to the second layer, wherein the information on the relative sampling phase offset may be derived using the first and the second layers (or more layers) in a layered data stream and wherein the motion vectors are transmitted in the layered data stream. Summarily, it is to be noted that, for instance, the present invention could be implemented in a scalable coding framework, wherein the video is coded using the aforementioned layered data stream.

[0140] Although the description so far has focused on luma samples, same may apply to chroma samples. For example, chroma samples could suffer the same relative phase phase sampling offset as luma samples or a different one if the resampler (or the video encoder or decoder) would decide so. A flag could indicate that the luma and chroma samples suffer the same relative sampling phase offset so that the same is derived for both samples / components. Alternatively, a different relative sampling phase offset could be signalled and included in the data stream (i.e. bitstream), either as itself or as a delta (e.g. a minor shift or offset or difference) to the luma relative sampling phase offset.

[0141] Alternatively, when scaling window signalling is absent in a bitstream, i.e. the cropping window coordinates also serve as scaling window coordinates. Hence, an alternative embodiment of the invention is to signal sub-pel offsets relative to the cropping window. Even further, the boundaries of the whole decoded picture may serve as anchor point for additional sub-pel offsets. In all of the above embodiments, while scaling windows boundaries may be located at sub-pel positions, scaling window dimensions remain full-pel. Alternatively, it could be allowed to use sub- pel scaling window dimension to enable further scaling factor granularity and / or more accurately match the image content of scaling window area across resolution versions by signalling separate sub-pel offsets for the left, right, top and bottom boundary of scaling windows, cropping windows or decoded picture boundaries.

[0142] Figure 11 exemplarily shows two sets of representations 402, 502 in an open GOP scenario.

[0143] An adaptive bitrate streaming technique, for example, DASH, may offer a plurality of video representations for retrieval of a video from the server (depending on dynamic network conditions). For example, the plurality of video representations may comprise bitstreams, or data streams, of different sizes, or bitstreams, or data streams of different resolutions. The downloaded data stream 300 potentially concatenates a plurality of video segments, for instance, GOPs. Therefore, multiple resolution versions of a particular video segment may have been generated, all individually, as illustrated in figure 11.

[0144] A first representation 402 comprises a first GOP 404 and a second GOP 406 sampled at a first resolution 450 whereas a second representation 502 comprises a first GOP 504 and a second GOP 506 sampled at a second resolution 550, wherein the first resolution 450 is lower than the second resolution 550. Both resolutions 450, 550 may vary in temporally, with the one kind coarser than the other (on average). For instance, a resolution attributed to a GOP may be defined as a minimum resolution for a picture among the resolution of each comprised picture in the said GOP.

[0145] As exemplarily illustrated in both representations 402, 502, each of the second GOPs 406, 506, illustrated on the right, comprise a B-picture 440, 540, two P-pictures 420, 430, 520, 530 and an l-picture 410, 510 wherein the comprised P-picture 420, 520 references an outside picture 408, 508 comprising within the representation 402, 502, located outside the second GOP 406, 506, comprised in the first GOP 404, 504 (as illustrated on the left), thus resulting in an open GOP scenario. For instance, each picture herein may be encoded at different resolutions. In a streaming scenario, an open GOP switching may be required, possibly due to, for example, attributing to dynamic network conditions. For example, an open GOP switching may imply, for the illustrated figure 11 , a P picture 520 comprised in the second GOP 506 of the second representation 502, having a second resolution 550, which is higher than the first resolution 450 of comprised GOPs 404, 406 of the first representation 402, may reference an outside picture 408, comprised in the first GOP 404 of the first representation 402.

[0146] An open GOP switching may be referred to as a bitstream, or data stream, switch to a different resolution at one or more of open GOP switching points. This open GOP switching, for instance, may result in a phase offset (or the relative sampling phase offset) 120 arising from the sampling between pictures with differing resolutions 450, 550. For example, the phase offset 120 may be referred to as a phase difference or phase mismatch. For instance, the different resolutions here could also refer to different sizes, different aspect ratios, or different phase information.

[0147] The decoder may then decode the GOPs 404, 506 without using the phase offset 120 (e.g., any information derived from or depending on the phase offset) or possibly, with an incorrect or misaligned information derived from the phase offset 120, resulting in a drift in the decoding process, which can lead to unpleasant visible visual artefacts in the output video. For instance, the unpleasant visible visual artefacts could comprise misalignment of pictures at the output side, aliasing, flickering and edge effects.

[0148] In accordance with embodiments, the video decoder may derive, from the side information in the data stream, the information on the relative sampling phase offset 120 by decoding the relative sampling phase offset 120 from a portion of the data stream relating to the first picture in a form measuring the relative sampling phase offset 120 for a predetermined position of the first picture. This means that the relative sampling phase 120 may be signaled directly in the PPS of the current picture (i.e. the first picture), but note that this example requires the data stream to remain as it is without being the result of a splicing between different representations coded in open GOP format, because, then, the reference picture (i.e. the second picture) might change and so the relative sampling phase offset 120 might do as well; rather, this embodiment fits to the case of taking advantage of the improved / harmonized reservoir of motion vectors and the corresponding improved coding efficiency.

[0149] Figure 12 exemplarily shows an embodiment in which the motion vector 100 associated with the predetermined block 102 in the first picture 200 and the second picture 200’ is determined using the information on the relative sampling phase offset 120. As shown in Fig. 12, a plurality of blocks, each of which has a further motion vector FMV#, is in a spatial neighborhood of the predetermined block 102, wherein the plurality of blocks are previously decoded in comparison to the predetermined block 102. Although the plurality of blocks are shown here to be spatially neighboring the predetermined block 102, variations wherein some or even all of the plurality of blocks may be temporally neighboring the predetermined block 102 form feasible embodiments.

[0150] The respective motion vector FMV# is associated with two pictures, namely a respective source picture 200# and a respective reference picture 200’#. The source pictures 200i- and the reference pictures 200’ 1- may each have a different relative sampling phase offset 120 between them when compared to the relative sampling phase offset 120 between the first 200 and the second 200’ picture. An aggregation of the further motion vectors FMV1.10 might be available as a list 180, or collection, of motion vectors candidate in the data stream after an ordering operation, as indicated by the arrow 170 in Fig. 12 . In particular, towards determining the motion vector 100, the ordering 170 of the list 180 of motion vector candidates may be performed by the video decoder on a basis of the respective relative sampling phase offset 120 between respective reference 200’# and source 200# of the respective further motion vector.

[0151] For instance, such an ordering 170 may relate to ordering motion vectors candidates formed by (further) motion vectors of previously decoded blocks (e.g. spatially or temporally neighboring the predetermined block) of a different relative sampling phase offset between referenced picture 200#’ and source (own) picture 200# than compared to the relative sampling phase offset 120 between the first 200 and the second picture 200’ to be subsequent to motion vector candidates of the same relative phase offset 120 between referenced picture 200’# and source (own) picture 200# than the relative sampling phase offset 120 between first 200 and second picture 200’. Further, an indexing into this list of mv candidates transmitted in the data stream may be obtained, for instance, using a variable length code whose code length increases (e.g. monotonically) increases with list rank, and using the indexed motion vector candidate for motion vector prediction. An opposite ordering of the just described is naturally also feasible.

[0152] Further in the course of determining the motion vector 100, the video decoder may determine 210 (depicted as an arrow 210) the motion vector 100 based on a further motion vector 220 of a previously decoded block (for instance, in the scope of this example, from the previously decoded blocks spatially neighboring the predetermined block 120) depending on a relative sampling phase offset 120 between reference and source picture of the further motion vector 220 and the relative sampling phase offset 120 between the first 200 and the second picture 200’. For example, determining the motion vector 100 may comprise: correcting the motion vector (i.e. ones of the further motion vectors) of a previously decoded block which is of a different relative sampling phase offset 120 between referenced picture 200#’ and source (own) picture 200# so as to compensate for the difference in the relative sampling phase offset 120; and for example, add the corrected motion vector to a list of MV candidates for the predetermined block 120.

[0153] In accordance with embodiments, when predicting a motion vector (e.g. the motion vector 100) from a motion vector candidate (either spatial or temporal), or from the list 180, the information on the relative sampling phase difference, such as the phase difference, between the current block (e.g. the predetermined block 102 in the first picture 200) and reference block (e.g. the predetermined block in the second picture 200’) and the neighbouring block (e.g. either spatial or temporal neighbour, already decoded block in one of the current pictures 200#) and its correspondent reference block (e.g. the respective block in the reference picture 200#’ respective to the one of the current pictures 200#) could be used for carrying out the motion vector prediction. In other words, the MV candidates, i.e. the further motion vectors, (denoted by FMV# in Fig. 12) of neighbouring or temporal candidates may have a different relative sampling phase offset 120 between the blocks (i.e. the blocks in the current pictures 200#) and their respective reference blocks (i.e. the blocks in the reference pictures 200#) than that of the current block (e.g. the block in the picture 200) and its reference block (e.g. the block in the picture 200’). This case, applies when the encoder has control of the RPR process (i.e., the decoder will receive only the bitstream that the encoder produces and reference pictures are not substitute as in the case of Open GOP streaming). As a further aspect of this embodiment, given that its use case relates to the encoder having full control on the relative sampling phase offsets (i.e. it knows the current pictures and reference pictures), instead of providing the information on the relative sampling phase offset for each picture and letting the decoder derive the information on the relative sampling phase offset for each block (depending on the information on the relative sampling phase offset between current picture and reference picture), the encoder could provide the information on the relative sampling phase offset directly for each picture (or more than one picture, in case there is more than one reference picture) so that the decoder only needs to parse the relative sampling phase offset and apply it. For instance, in such a case the units in which the information on the relative sampling phase offset 120 is given may be in the reference picture resolution or reference picture fractional resolution (e.g. 1 / 16ththereof assuming that MVs can point to 1 / 16thsample positions). Details described so far in regard to the figure 12 are equally applicable to not only all described embodiments but especially to the second aspect of the present disclosure. Hitherto the present disclosure has described the first aspect wherein the video decoder derives the information on the relative sampling phase offset 120 and uses it to determine the corresponding position 108 in the second picture. The present disclosure also relates to the second aspect, additionally or alternatively, wherein in contrast to the first aspect, in the second aspect the video decoder makes use of the derived information on the relative sampling phase offset to determine the motion vector 100 associated with the predetermined block 102. It is emphasized that details in regard to figure 12 and other details, alternatives and embodiments described elsewhere herein are promptly combinable with the decoder which determines the motion vector 100 using the information on the relative sampling phase offset 120.

[0154] The present application now proceeds further with description of encoders fitting the decoders already described so far.

[0155] In accordance with embodiments, a video encoder encodes a video into a data stream using motion compensation prediction based on motion vectors transmitted, or available, in the data stream. This encoding of the video is performed by: scaling a motion vector 100 transmitted, or available, in the data stream (the same data stream as the motion vector) according to a resolution ratio between the first picture (current picture, samples of which are to be encoded) and a second picture (reference picture, sample of which might be used for prediction) to obtain a scaled motion vector. The encoding of the video further involves motion-compensatedly predicting the predetermined block 102 of the first picture by sampling the second picture at a motion- compensatedly offset position 108 in the second picture, corresponding to a position 104 of the predetermined block 102 in the first picture. For instance, the motion compensation prediction may predict a sample filling of the predetermined block wherein the sample filling may comprise a portion or whole of the predetermined block 102. The video encoder inserts, as side information into the data stream, information on a relative sampling phase offset 120 between first samples 1061 of the first picture and the second samples IO62 of the second picture. This information, for example, might be inserted at a relative sampling phase offset unit which is finer than, or as fine as, a minimum of a first sample pitch and a second sample pitch.

[0156] In accordance with a first aspect, the video encoder determines the corresponding position 108 in the second picture using the information on the relative sampling phase offset 120. In accordance with a second aspect, additionally or alternatively to the first aspect, the video encoder determines the motion vector 100 using the information on the relative sampling phase offset 120.

[0157] Details previously described pertaining to deriving this information on the relative sampling phase offset 120 are applicable to the encoder of both aspects taken individually or in combination.

[0158] It is to be noted that although the encoder could disregard the phase offset 120 in motion vector compensation due to the described compensation by way of the motion vector determination on the encoder side, the decoder and encoder could form an improved video codec by mutually harmonizing motion vectors transmitted in the data stream in that same relates to motion compensation without phase offset.

[0159] Further, in accordance with Fig. 12, in determining the motion vector 100 using the information on the relative sampling phase offset 120, the video encoder may order 170 a list 180 of motion vector candidates based on further motion vectors of previously decoded blocks. This ordering 170 may depend on a relative sampling phase offset 120 between reference 200#’ and source picture 200# of the respective further motion vector. The video encoder may determine the motion vector 100 based on a further motion vector 220 of a previously decoded block depending on a relative sampling phase offset 120 between reference and source picture of the further motion vector 220 and the relative sampling phase offset 120 between the first 200 and second picture 200’.

[0160] For example, the encoder may order 170 motion vectors candidates formed by motion vectors of previously decoded blocks (e.g. spatially or temporally neighboring the predetermined block) of a different relative phase offset between referenced picture and source (own) picture then compared to the relative phase offset between first and second picture to by subsequent to motion vector candidates of the same relative phase offset between referenced picture and source (own) picture than the relative phase offset between first and second picture; and use an index into this list of mv candidates transmitted in the data stream using a variable length code whose code length (e.g. monotonically) increases with list rank, and using the indexed motion vector candidate for motion vector prediction.

[0161] For example, the encoder may correct the motion vector of a previously decoded block which is of a different relative phase offset between referenced picture and source (own) picture so as to compensate for the difference in relative sampling phase offset; and, for instance, add the corrected motion vector to a list of MV candidates for the predetermined block 102.

[0162] In the following, further implementation alternatives are described, referring to all of the embodiments described above.

[0163] Although some aspects have been described as features in the context of an apparatus it is clear that such a description may also be regarded as a description of corresponding features of a method. Although some aspects have been described as features in the context of a method, it is clear that such a description may also be regarded as a description of corresponding features concerning the functionality of an apparatus.

[0164] In particular, it is noted that Fig. 8, 11 and 12 may also be regarded as illustration of a method for decoding a video and / or for encoding a video, where the blocks, modules and stages may be regarded as steps of methods.

[0165] Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, one or more of the most important method steps may be executed by such an apparatus.

[0166] The inventive encoded image signal can be stored on a digital storage medium or can be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet. In other words, further embodiments provide a video bitstream product including the video bitstream according to any of the herein described embodiments, e.g. a digital storage medium having stored thereon the video bitstream.

[0167] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software or at least partially in hardware or at least partially in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable. Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0168] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.

[0169] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0170] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0171] A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and / or non-transitory.

[0172] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.

[0173] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0174] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0175] A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

[0176] In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

[0177] The apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0178] The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0179] In the foregoing Detailed Description, it can be seen that various features are grouped together in examples for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed examples require more features than are expressly recited in each claim. Rather, as the following claims reflect, subject matter may lie in less than all features of a single disclosed example. Thus the following claims are hereby incorporated into the Detailed Description, where each claim may stand on its own as a separate example. While each claim may stand on its own as a separate example, it is to be noted that, although a dependent claim may refer in the claims to a specific combination with one or more other claims, other examples may also include a combination of the dependent claim with the subject matter of each other dependent claim or a combination of each feature with other dependent or independent claims. Such combinations are proposed herein unless it is stated that a specific combination is not intended. Furthermore, it is intended to include also features of a claim to any other independent claim even if this claim is not directly made dependent to the independent claim.

[0180] The above described embodiments are merely illustrative for the principles of the present disclosure. It is understood that modifications and variations of the arrangements and the details described herein will be apparent to others skilled in the art. It is the intent, therefore, to be limited only by the scope of the pending patent claims and not by the specific details presented by way of description and explanation of the embodiments herein.

Claims

CLAIMS1. Video decoder configured to decode a video from a data stream using motion compensation prediction based on motion vectors transmitted in the data stream by scaling a motion vector (100) transmitted in the data stream for a predetermined block (102) of a first picture according to a resolution ratio between the first picture and a second picture to obtain a scaled motion vector, motion-compensatedly predict the predetermined block (102) of the first picture by sampling the second picture at a motion-compensatedly offset position resulting from applying the scaled motion vector (110) to a corresponding position (108) in the second picture, corresponding to a position (104) of the predetermined block (102) in the first picture, wherein the decoder is configured to derive, from side information in the data stream, information on a relative sampling phase offset (120) between first samples (1061) of the first picture and the second samples (1062) of the second picture, and determine the corresponding position (108) in the second picture using the information on the relative sampling phase offset (120).

2. Video decoder of claim 1 , configured to decode scaling window information from the data stream, the scaling window information defining a position of a first scaling window (112i) in the first picture and of a second scaling window (1122) in the second picture, derive the resolution ratio between the first and second pictures based on a size ratio between the first scaling window (112i) and the second scaling window (1122), and determine the corresponding position (108) so that the corresponding position (108) is offset from a co-located position (116) which is, relative to the second scaling window(1122), positioned just as the position (104) of the predetermined block (102) is located relative to the first scaling window (112i) according to the information on the relative sampling phase offset (120).

3. Video decoder of claim 1 or 2, configured to decode, from the side information in the data stream, and determine the corresponding position (108) in the second picture using, one or more of first phase information, indicating a first phase at which the samples (106i) of the first picture sample the first picture, and second phase information, indicating a second phase at which the second samples of the second picture sample the second picture, wherein the first phase and the second phase are indicative of the relative sampling phase offset (120) and the decoder is configured to determine the corresponding position (108) in the second picture using the one or more of the first phase information and the second phase information.

4. Video decoder of claim 3, configured to decode the one or more of the first phase information and the second phase information from the data stream in addition to scaling window information, the scaling window information defining a position of a first scaling window (112i) in the first picture and of a second scaling window (1122) in the second picture, derive the resolution ratio between the first and second pictures based on a size ratio between the first scaling window (112i) and the second scaling window (1122), and determine the corresponding position (108) so that the corresponding position (108) is offset from a co-located position (116) which is, relative to the second scaling window (1122), positioned just as the position (104) of the predetermined block (102) is located relative to the first scaling window (112i) according to the information on the relative sampling phase offset (120).

5. Video decoder of claim 4, configured to decode the scaling window information so that the position of the first scaling window (112i) in the first picture is represented in units of thefirst samples and the second scaling window (1122) in the second picture in units of the second samples.

6. Video decoder of any of previous claims 3 to 5, configured to decode, from the side information in the data stream, as a measure of the first phase, a first phase offset length, or as a measure of the second phase, a second phase offset length.

7. Video decoder of claim 6, wherein the first phase offset length, or the second phase offset length is measured relative to a upper left picture corner (126), an upper left scaling window corner or an upper left cropping window corner.

8. Video decoder of any of previous claims 6 or 7, configured to decode the first phase offset length, or the second phase offset length from the data stream in predetermined reference units.

9. Video decoder of claim 8, configured to support a plurality of sample resolutions, with each picture of the video being of one of the plurality of sample resolutions, in the motion-compensatedly prediction of the predetermined block (102) of the first picture, perform the sampling of the second picture at the motion-compensatedly offset position resulting from applying the scaled motion vector (110) to a corresponding position (108) in the second picture, corresponding to a position (104) of the predetermined block (102) in the first picture, at a resolution which is x times finer than a sample resolution of the second picture, wherein the predetermined reference units correspond to a resolution which is between being as fine as, and being x times finer than a finest sample resolution among the plurality of sample resolutions.

10. Video decoder of claim 8, configured tosupport a plurality of sample resolutions, with each picture of the video being of one of the plurality of sample resolutions, wherein the predetermined reference units correspond to a resolution which is between being as fine as, and being a finest sample resolution among the plurality of sample resolutions.

11. Video decoder of any of previous claims 3 to 10, configured to decode, from the side information in the data stream, as a measure of the first phase, a first numerator and / or a first denominator of a first fraction representing the first phase, or as a measure of the second phase, a second numerator and a second denominator of a second fraction representing the second phase.

12. Video decoder of claim 11 , wherein the first numerator and / or the first denominator are measured in units of first samples and / or the second numerator and the second denominator are measured in units of second samples.

13. Video decoder of any of previous claims 11 or 12, wherein the first numerator and / or the first denominator are measured in units of predetermined reference units and / or the second numerator and the second denominator are measured in predetermined reference units.

14. Video decoder of claim 13, configured to support a plurality of sample resolutions, with each picture of the video being of one of the plurality of sample resolutions, in the motion-compensatedly predict the predetermined block (102) of the first picture, perform the sampling the second picture at the motion-compensatedly offset position resulting from applying the scaled motion vector (110) to a corresponding position (108) in the second picture, corresponding to a position (104) of the predetermined block (102) in the first picture, at a resolution which is x times finer than a sample resolution of the second picture,wherein the predetermined reference units correspond to a resolution which is between being as fine as, and being x times finer than a finest sample resolution among the plurality of sample resolutions.

15. Video decoder of claim 9, configured to support a plurality of sample resolutions, with each picture of the video being of one of the plurality of sample resolutions, wherein the predetermined reference units correspond to a resolution which is between being as fine as, and being, a finest sample resolution among the plurality of sample resolutions.

16. Video decoder of any of previous claims 11 to 15, wherein the first numerator and / or the first denominator are signaled in the data stream at a predetermined representation accuracy and / or the second numerator and / or the second denominator are signaled in the data stream at a predetermined representation accuracy.

17. Video decoder of any of previous claim 16, wherein the predetermined representation accuracy is signaled in the data stream.

18. Video decoder of any of previous claims 11 to 17, configured to decode, from the side information in the data stream, a common denominator. as a measure of the first phase, a first numerator forming, along with the common denominator, a first fraction representing the first phase, or as a measure of the second phase, a second numerator forming, along with the common denominator, a second fraction representing the second phase.

19. Video decoder of claim 18, configured to decode, from the side information in the data stream, the common denominator,at a scope covering the whole data stream or a sequence of pictures including the first and second pictures.

20. Video decoder of any previous claim 3 to 19, configured to, decode, from the side information in the data stream, the first phase information, and the second phase information, in determining the corresponding position (108) in the second picture using the information on the relative sampling phase offset (120),Check whether the first phase equals the second phase,If the first phase is unequal to the second phase, determining the corresponding position (108) in the second picture by using the first phase and the second phase, andIf the first phase equals the second phase, determining the corresponding position (108) in the second picture without using the first phase and without using the second phase.

21. Video decoder of any of previous claims 3 to 20, configured to decode, as part of each phase information of the one or more of the first phase information and the second phase information, a zeroness flag indicative of the respective phase information being indicative of a zero phase, followed by, if the zeroness flag does not indicate a zero phase, one or more syntax elements indicative of the respective phase.

22. Video decoder of any of previous claims 3 to 21 , wherein the side information is contained in picture parameter sets, a picture header or a slice header contained in the data stream for the first and / or second picture.

23. Video decoder of any previous claim 3 to 22, wherein the first phase and the second phase are measured relative to a upper left picture corner an upper left scaling window corner or an upper left cropping window corner.

24. Video decoder of any previous claim 3 to 23, configured to in determining the relative sampling phase offset (120), subtract from the first phase, scaled according to according to the resolution ratio, the second phase.

25. Video decoder of any previous claim 3 to 24, configured to determining the relative sampling phase offset (120), by using the first phase, scaled according to the resolution ratio, and the second phase, unsealed, or using the first phase, scaled according to according to a sampling pitch of the first samples and the second phase scaled according to a sampling pitch of the second samples.

26. Video decoder of any of claims 3 to 25, configured to decode the one or more of the first phase information and the second phase information from the data stream by decoding an index into a list of predetermined phases transmitted in the data stream at a scope covering the whole data stream or a sequence of pictures including the first and second pictures.

27. Video decoder of any previous claim 1 to 26, configured to decode scaling window information from the data stream, the scaling window information defining a position of a first scaling window (112i) in the first picture and of a second scaling window (1122) in the second picture, wherein the scaling window information defines the position of the first scaling window (112i) in the first picture in units finer than the first samples and / or defines the position of a second scaling window (1122) in the second picture in units finer than the second samples, wherein a sub-sample part of the scaling window information is indicative of the relative sampling phase offset (120), derive the resolution ratio between the first and second pictures based on a size ratio between the first scaling window and the second scaling window (1122) , anddetermine the corresponding position (108) so that the corresponding position (108) is, relative to the second scaling window, positioned just as the position (104) of the predetermined block (102) is located relative to the first scaling window (112i) , according to the information on the relative sampling phase offset (120).

28. Video decoder of any previous claim 1 to 27, configured to decode cropping window information from the data stream, the cropping window information defining a position of a first cropping window (112i) in the first picture and of a second cropping window (1122) in the second picture, wherein the cropping window information defines the position of the first cropping window (112i) in the first picture in units finer than the first samples and / or defines the position of a second cropping window (1122) in the second picture in units finer than the second samples, wherein a sub-sample part of the cropping window information is indicative of the relative sampling phase offset (120), derive the resolution ratio between the first and second pictures based on a size ratio between the first cropping window and the second cropping window (1122), and determine the corresponding position (108) so that the corresponding position (108) is, relative to the second cropping window, positioned just as the position (104) of the predetermined block (102) is located relative to the first cropping window (112i), according to the information on the relative sampling phase offset (120).

29. The specifics of claims 6 to 10 and claims 11 to 20 might be used for signaling the position of the scaling / cropping windows as well.

30. Video decoder of any previous claim 1 to 29, configured to derive, from the side information in the data stream, the information on the relative sampling phase offset (120) in a form decomposed horizontally and vertically.

31. Video decoder of claim 30, configured toIn deriving, from the side information in the data stream, the information on the relative sampling phase offset (120) in the form decomposed horizontally and vertically, be responsive to a gating flag in the side information so as to derive one of a horizontal component and a vertical component of the sampling phase offset (120) from a signalization in the side information, relating to the other of the horizontal component and the vertical component of the sampling phase offset (120), or derive each of the horizontal component and the vertical component of the sampling phase offset (120) from a separate signalization in the side information relating to the respective of the horizontal component and the vertical component of the sampling phase offset (120).

32. Video decoder of any previous claim 1 to 31 , configured to derive, from the side information in the data stream, the information on the relative sampling phase offset (120) by decoding the relative sampling phase offset (120) from a portion of the data stream relating to the first picture in a form measuring the relative sampling phase offset (120) for a predetermined position of the first picture.

33. Video decoder of any previous claim 1 to 32, configured to decode the motion vector from the data stream using motion vector prediction.

34. Video decoder of claim 33, configured toIn decode the motion vector from the data stream using motion vector prediction, form a list of motion vector candidates based on further motion vectors of previously decoded blocks depending on, for a predetermined further motion vector of the further motion vectors of previously decoded blocks, a relative sampling phase offset (120) between reference and source picture of the predetermined further motion vector.

35. Video decoder configured to decode a video from a data stream using motion compensation prediction based on motion vectors transmitted in the data stream byscaling a motion vector (100) transmitted in the data stream for a predetermined block (102) of a first picture according to a resolution ratio between the first picture and a second picture to obtain a scaled motion vector, motion-compensatedly predict the predetermined block (102) of the first picture by sampling the second picture at a motion-compensatedly offset position resulting from applying the scaled motion vector (110) to a corresponding position (108) in the second picture, corresponding to a position (104) of the predetermined block (102) in the first picture, wherein the decoder is configured to derive, from side information in the data stream, information on a relative sampling phase offset (120) between first samples (1061) of the first picture and the second samples (1062) of the second picture, and determine the motion vector (100) using the information on the relative sampling phase offset (120).

36. Video decoder of claim 35, configured to in determining the motion vector (100) using the information on the relative sampling phase offset (120), order a list of motion vector candidates based on further motion vectors of previously decoded blocks depending on, for each of the further motion vectors of previously decoded blocks, a relative sampling phase offset (120) between reference and source picture of the respective further motion vector, determine the motion vector based on a further motion vector of a previously decoded block depending on a relative sampling phase offset (120) between reference and source picture of the further motion vector and the relative sampling phase offset (120) between the first and second picture.

37. Video encoder configured to encode a video into a data stream using motion compensation prediction based on motion vectors transmitted in the data stream by scaling a motion vector (100) transmitted in the data stream for a predetermined block (102) of a first picture according to a resolution ratio between the first picture and a second picture to obtain a scaled motion vector, motion-compensatedly predict the predetermined block (102) of the first picture by sampling the second picture at a motion-compensatedly offset position resulting from applying the scaled motion vector (110) to a corresponding position (108) in the second picture, corresponding to a position (104) of the predetermined block (102) in the first picture, wherein the encoder is configured to insert, as side information in the data stream, information on a relative sampling phase offset (120) between first samples (1061) of the first picture and the second samples (1062) of the second picture, and determine the corresponding position (108) in the second picture using the information on the relative sampling phase offset (120).

38. Video encoder configured to encode a video into a data stream using motion compensation prediction based on motion vectors transmitted in the data stream by scaling a motion vector (100) transmitted in the data stream for a predetermined block (102) of a first picture according to a resolution ratio between the first picture and a second picture to obtain a scaled motion vector, motion-compensatedly predict the predetermined block (102) of the first picture by sampling the second picture at a motion-compensatedly offset position resulting from applying the scaled motion vector (110) to a corresponding position (108) in the secondpicture, corresponding to a position (104) of the predetermined block (102) in the first picture, wherein the encoder is configured to insert, as side information in the data stream, information on a relative sampling phase offset (120) between first samples (1061) of the first picture and the second samples (1062) of the second picture, and determine the motion vector (100) using the information on the relative sampling phase offset (120).

39. Video encoder of claim 38, configured toIn determining the motion vector (100) using the information on the relative sampling phase offset (120), order a list of motion vector candidates based on further motion vectors of previously decoded blocks depending on, for each of the further motion vectors of previously decoded blocks, a relative sampling phase offset (120) between reference and source picture of the respective further motion vector, determine the motion vector based on a further motion vector of a previously decoded block depending on a relative sampling phase offset (120) between reference and source picture of the further motion vector and the relative sampling phase offset (120) between the first and second picture.

40. Method for video decoding, wherein the method comprises decoding a video from a data stream using motion compensation prediction based on motion vectors transmitted in the data stream by scaling a motion vector (100) transmitted in the data stream for a predetermined block (102) of a first picture according to a resolution ratio between the first picture and a second picture to obtain a scaled motion vector,motion-compensatedly predicting the predetermined block (102) of the first picture by sampling the second picture at a motion-compensatedly offset position resulting from applying the scaled motion vector (110) to a corresponding position (108) in the second picture, corresponding to a position (104) of the predetermined block (102) in the first picture, wherein the method comprises deriving, from side information in the data stream, information on a relative sampling phase offset (120) between first samples (1061) of the first picture and the second samples (1062) of the second picture, and determining the corresponding position (108) in the second picture using the information on the relative sampling phase offset (120).41 . Method for video decoding, wherein the method comprises decoding a video from a data stream using motion compensation prediction based on motion vectors transmitted in the data stream by scaling a motion vector (100) transmitted in the data stream for a predetermined block (102) of a first picture according to a resolution ratio between the first picture and a second picture to obtain a scaled motion vector, motion-compensatedly predicting the predetermined block (102) of the first picture by sampling the second picture at a motion-compensatedly offset position resulting from applying the scaled motion vector (110) to a corresponding position (108) in the second picture, corresponding to a position (104) of the predetermined block (102) in the first picture, wherein the method further comprises deriving, from side information in the data stream, information on a relative sampling phase offset (120) between first samples (1061) of the first picture and the second samples (1062) of the second picture, anddetermining the motion vector (100) using the information on the relative sampling phase offset (120).

42. Method for video encoding, wherein the method comprises encoding a video into a data stream using motion compensation prediction based on motion vectors transmitted in the data stream by scaling a motion vector (100) transmitted in the data stream for a predetermined block (102) of a first picture according to a resolution ratio between the first picture and a second picture to obtain a scaled motion vector, motion-compensatedly predicting the predetermined block (102) of the first picture by sampling the second picture at a motion-compensatedly offset position resulting from applying the scaled motion vector (110) to a corresponding position (108) in the second picture, corresponding to a position (104) of the predetermined block (102) in the first picture, wherein the method comprises inserting, as side information in the data stream, information on a relative sampling phase offset (120) between first samples (1061) of the first picture and the second samples (1062) of the second picture, and determining the corresponding position (108) in the second picture using the information on the relative sampling phase offset (120).

43. Method for video encoding, wherein the method comprises encoding a video into a data stream using motion compensation prediction based on motion vectors transmitted in the data stream by scaling a motion vector (100) transmitted in the data stream for a predetermined block (102) of a first picture according to a resolution ratio between the first picture and a second picture to obtain a scaled motion vector,motion-compensatedly predicting the predetermined block (102) of the first picture by sampling the second picture at a motion-compensatedly offset position resulting from applying the scaled motion vector (110) to a corresponding position (108) in the second picture, corresponding to a position (104) of the predetermined block (102) in the first picture, wherein the method comprises inserting, as side information in the data stream, information on a relative sampling phase offset (120) between first samples (1061) of the first picture and the second samples (1062) of the second picture, and determining the motion vector (100) using the information on the relative sampling phase offset (120).

44. Computer program for performing the method according to any of claims 40 to 43, when the computer program runs on a computer.

45. Data stream comprising an encoded representation of a video, encoded using the method according to any of claims 42 to 43.

Images