Subsample interpolation for reference image resampling

Abstract

To provide a method for encoding video into a data stream and decoding a data stream having video encoded therein, using motion compensated prediction between images of equal resolution and images of different resolution on the basis of motion vectors at half sample resolution and motion vectors at different resolutions using an interpolation filter to obtain sub-sample values in a reference sample array.SOLUTION: In a video decoder 410, an interpolation filter is selected from two interpolation filter versions 431, 432 that differ in higher edge-preserving properties, and the selection is made depending on whether the current image has equal image resolution in the horizontal and / or vertical dimensions to the reference sample array and / or constraint information 425 in a data stream 420.SELECTED DRAWING: Figure 4c

Description

[Technical Field]

[0001] This application relates to video coding concepts, and in particular to interpolation filters for motion compensation.

[0002] Current video coding standards, such as Varied Video Coding (VVC), allow switching interpolation filters for motion compensation depending on the resolution of motion vectors (MVs), which can be signaled at the block level. If the MV (or MV-difference MVD) is coded at a particular resolution, e.g., half-sample precision, then a different interpolation filter can be used to interpolate particular sub-sample positions.

[0003] Another new feature is reference picture resampling, which allows previously coded pictures to be referenced in motion compensated inter-picture prediction with a different resolution / size than the current picture. To do so, the reference picture region is resampled to a block of the same size as the current block. This can lead to the case where some sub-positions are obtained using different phases of the interpolation filter.

[0004] For example, if a 16x16 block references an image that is 1 / 4 the size in each dimension, the corresponding 4x4 block of the referenced image needs to be upsampled to 16x16, which can involve different interpolation filters for specific sub-positions / phases. For example, if the MV is signaled with a precision associated with a smoothing interpolation filter, this filter can be applied to the phase associated with the upsampling of the reference image, while a sharpening interpolation filter can be applied to the other phase.

[0005] This mix-up can produce visible artifacts and therefore result in a worse motion-compensated inter predictor, which in turn increases the prediction error and the bit rate required to encode the prediction residual to achieve equal quality.

[0006] The present application aims to bring about a more efficient video coding concept that supports reference image resampling.

[0007] This object is achieved by the subject matter of the independent claims.

[0008] Preferred embodiments of the present application are described below with reference to the drawings. [Brief explanation of the drawings]

[0009] [Figure 1] 1 shows an apparatus for predictively encoding an image into a data stream. [Figure 2] 1 shows an apparatus for predictively decoding an image from a data stream. [Figure 3] 1 illustrates the relationship between the reconstructed signal and the combination of the prediction residual signal and the prediction signal. [Figure 4a] A spatially overlapping sample of the reference and current images is shown. [Figure 4b] It shows samples of the reference and current images that have the same resolution and for which the motion vectors are defined at half-pel resolution. [Figure 4c] 1 illustrates a video decoder according to an embodiment of the present invention; [Figure 4d]The selection of interpolation filters for different types of samples of the reference image, i.e., samples with different phases, is illustrated here using the table of FIG. 9 as an example. The selection for half-pel resolution positions has two options, which embodiments advantageously select. Furthermore, the application of the selected interpolation filter is described for one exemplary sample, here a quarter-pel sample. The filter selection among the two half-pel interpolation filters may be performed separately for the horizontal and vertical directions, such that the reference image is vertically interpolated for half-pel samples using one half-pel interpolation filter, while another half-pel interpolation filter may be used horizontally for interpolation at half-pel positions. Alternatively, the selection may be performed globally for both directions, depending on whether the resolutions of the current and reference images match vertically and horizontally, such that one or the other half-pel interpolation filter is used horizontally and vertically. [Figure 5] 1 illustrates a video encoder according to an embodiment of the present invention; [Figure 6] Demonstrates reference image resampling bitrate adaptation for video conferencing with varying throughput [Figure 7] "Standard for DASH and Open GOP Resolution Switching" shows an example of image resampling bitrate adaptation. [Figure 8] 10 shows an example of a third image that performs a RoI zoom-in of a portion of a second image. [Figure 9] An example of a smoothing filter for adaptive motion vector resolution when motion vector differentials are at half sample resolution is shown, and the figure also includes a table of additional interpolation filters for use with samples at phases other than half-pel phase. [Figure 10] An example of signaling in a bitstream using syntax where amvr_flag is equal to 1 and amvr_precision_idx is equal to 0 is shown below.

[0010] The following description of the figures begins with presenting a description of an encoder and decoder of a block-based predictive codec for encoding images of video to form an example for an encoding framework in which embodiments of the present invention may be incorporated. Each encoder and decoder is described with reference to Figures 1-3. Below, a description of embodiments of the inventive concepts is presented along with an explanation of how such concepts may be incorporated into the encoder and decoder of Figures 1 and 2, respectively, although the embodiments described with reference to Figures 4 et seq. below may also be used for encoders and decoders that do not operate according to the encoding framework underlying the encoder and decoder of Figures 1 and 2, such as without intra-coded blocks competing with inter-coded blocks within an image and / or without transform-based residual coding.

[0011] FIG. 1 illustratively illustrates an apparatus for predictively encoding an image 12 into a data stream 14 using transform-based residual coding. The apparatus or encoder is indicated using the reference symbol 10. FIG. 2 illustrates a corresponding decoder 20, i.e., an apparatus 20 configured to predictively decode an image 12′ from the data stream 14, also using transform-based residual decoding, where an apostrophe has been used to indicate that the image 12′ reconstructed by the decoder 20 deviates from the image 12 originally encoded by the apparatus 10 in terms of coding loss introduced by quantization of the prediction residual signal. While FIGS. 1 and 2 illustratively use transform-based predictive residual coding, embodiments of the present application are not limited to this type of predictive residual coding. This also applies to other details described with respect to FIGS. 1 and 2, as outlined below.

[0012] The encoder 10 is configured to perform a spatial-to-spectral transformation on the prediction residual signal and to encode the prediction residual signal thus obtained into a data stream 14. Similarly, the decoder 20 is configured to decode the prediction residual signal from the data stream 14 and to perform a spectral-to-spatial transformation on the prediction residual signal thus obtained.

[0013] Internally, the encoder 10 may include a prediction residual signal former 22 that generates a prediction residual 24 to measure the deviation of a prediction signal 26 from the original signal, i.e., from the image 12. The prediction residual signal former 22 may, for example, be a subtractor that subtracts the prediction signal from the original signal, i.e., from the image 12. The encoder 10 then includes a transformer 28 that performs a spatial-to-spectral transformation on the prediction residual signal 24 to obtain a spectral-domain prediction residual signal 24′, which is then quantized by a quantizer 32, also included in the encoder 10. The quantized prediction residual signal 24″ is then coded into the bitstream 14. For this purpose, the encoder 10 may optionally include an entropy coder 34 that entropy codes the transformed and quantized prediction residual signal into the data stream 14. The prediction signal 26 is generated by a prediction stage 36 of the encoder 10 based on the prediction residual signal 24″ that is coded into and decodable from the data stream 14. For this purpose, the prediction stage 36 may comprise, as shown in FIG. 1, a dequantizer 38 that dequantizes the prediction residual signal 24″ to obtain a spectral-domain prediction residual signal 24″ that corresponds to the signal 24′ except for quantization losses, followed by an inverse transformer 40 that subjects the latter prediction residual signal 24″ to an inverse transform, i.e., a spectral-to-spatial transformation, to obtain a prediction residual signal 24″″ that corresponds to the original prediction residual signal 24 except for quantization losses. A combiner 42 of the prediction stage 36 then recombines, e.g., by addition, the prediction signal 26 and the prediction residual signal 24″″ to obtain a reconstructed signal 46, i.e., a reconstruction of the original signal 12. The reconstructed signal 46 may correspond to the signal 12′. A prediction module 44 of the prediction stage 36 then generates a prediction signal 26 based on the signal 46, for example by using spatial prediction, i.e., intra-picture prediction, and / or temporal prediction, i.e., inter-picture prediction.

[0014] Similarly, decoder 20 may be internally configured from components corresponding to prediction stage 36 and interconnected in a manner corresponding to prediction stage 36, as shown in Figure 2. Specifically, entropy decoder 50 of decoder 20 may entropy decode a quantized spectral domain prediction residual signal 24'' from the data stream, whereby dequantizer 52, inverse transformer 54, combiner 56 and prediction module 58, interconnected and cooperating in the manner described above with respect to the modules of prediction stage 36, provide a reconstructed signal based on prediction residual signal 24'', such that the output of combiner 56 provides a reconstructed signal, i.e., image 12', as shown in Figure 2.

[0015] Although not specifically described above, it is readily apparent that the encoder 10 may set several coding parameters, including, for example, prediction modes, motion parameters, etc., according to several optimization schemes, such as, for example, several rate-distortion criteria, i.e., methods that optimize coding cost. For example, the encoder 10 and decoder 20 and corresponding modules 44, 58 may each support different prediction modes, such as intra-coding and inter-coding modes. The precision with which the encoder and decoder switch between these prediction mode types may correspond to the subdivision of the image 12 and 12′ into coding segments or coding blocks, respectively. In these coding segments, for example, the image may be subdivided into intra-coded and inter-coded blocks. The intra-coded blocks are predicted based on their spatial, already coded / decoded neighbors, as outlined in more detail below. Several intra-coding modes may be selected for each intra-coding segment, including directional or angular intra-coding modes, according to which each segment is filled by extrapolating sample values ​​from neighboring segments along a direction specific to each directional intra-coding mode. Intra-coding modes may also include one or more additional modes, such as a DC coding mode in which the prediction of each intra-coded block assigns a DC value to all samples in the respective intra-coded segment, and / or a planar intra-coding mode in which the prediction of each block approximates or determines a spatial distribution of sample values ​​described by a two-dimensional linear function over the sample positions of the respective intra-coded block, with the planar driving slope and offset defined by the two-dimensional linear function based on neighboring samples. In contrast, inter-coded blocks may be predicted temporally, for example. For inter-coded blocks, a motion vector may be signaled in the data stream, indicating the spatial displacement of a portion of a previously coded image of the video to which image 12 belongs, where the previously coded / decoded image was sampled to obtain a prediction signal for the respective inter-coded block.This means that in addition to the residual signal coding constituted by data stream 14, such as entropy-coded transform coefficient levels representing the quantized spectral domain predicted residual signal 24", data stream 14 may have any further parameters encoded therein, such as coding mode parameters for assigning coding modes to various blocks, prediction parameters for parts of the blocks, e.g., motion parameters for inter-coded segments, and parameters controlling and signaling the respective subdivision of pictures 12 and 12' into segments. The decoder 20 uses these parameters to subdivide the pictures in the same way as the encoder did, assign the same prediction modes to the segments, and perform the same predictions resulting in the same predicted signal.

[0016] FIG. 3 illustrates the relationship between the combination of the reconstructed signal, i.e., the reconstructed image 12′, on the one hand, and the predicted residual signal 24′″ signaled in the data stream 14, on the other hand, and the prediction signal 26. As already indicated above, the combination may be additive. The prediction signal 26 is illustrated in FIG. 3 as a subdivision of the image region into intra-coded blocks, exemplarily shown using hatching, and inter-coded blocks, exemplarily shown without hatching. The subdivision may be any subdivision, such as a regular subdivision of the image region into rows and columns of square or non-square blocks, or a multi-tree subdivision of the image 12 from a tree root block into multiple leaf blocks of various sizes. A mixture thereof is illustrated in FIG. 3, such as a quad-tree subdivision, in which the image region is first subdivided into rows and columns of tree root blocks, which are then further subdivided according to a recursive multi-tree subdivision into one or more leaf blocks.

[0017] Again, data stream 14 may have intra-coding modes coded therein for intra-coded blocks 80, which assigns each intra-coded block 80 one of several supported intra-coding modes. For inter-coded blocks 82, data stream 14 may have one or more motion parameters coded therein. In general, inter-coded blocks 82 are not restricted to being temporally coded. Alternatively, inter-coded block 82 may be any block predicted from a previously coded portion beyond current image 12 itself, such as a previously coded image of the video to which image 12 belongs, or an image of another view, or a hierarchically lower layer if the encoder and decoder are scalable encoder and decoder, respectively.

[0018] The prediction residual signal 24"" in FIG. 3 is also illustrated as a subdivision of the image region into blocks 84. These blocks are sometimes referred to as transform blocks to distinguish them from the coding blocks 80 and 82. In effect, FIG. 3 illustrates that the encoder 10 and decoder 20 may use two different subdivisions of the image 12 and the image 12', respectively, into blocks: one subdivision into coding blocks 80 and 82, respectively, and another subdivision into transform blocks 84. While both subdivisions may be the same, i.e., each coding block 80, 82 may simultaneously form a transform block 84, FIG. 3 also illustrates the case where, for example, the subdivision into transform blocks 84 forms an extension of the subdivision into coding blocks 80, 82 such that any boundary between the two blocks 80, 82 overlaps the boundary between the two blocks 84, or alternatively, each block 80, 82 coincides with one of the transform blocks 84 or with a group of transform blocks 84. However, the subdivisions may also be determined or selected independently of one another, such that the transformation blocks 84 may alternately cross the block boundaries between the blocks 80, 82. Thus, as far as the subdivision into transformation blocks 84 is concerned, similar statements apply as those made with respect to the subdivision into blocks 80, 82. That is, the blocks 84 may be the result of a regular subdivision of the image region into blocks (with or without arrangement into rows and columns), a recursive multi-tree subdivision of the image region, or a combination thereof or any other type of blocking. As a mere aside, it should be noted that the blocks 80, 82, 84 are not limited to being quadratic, rectangular, or other shapes.

[0019] 3 further illustrates that the combination of the prediction signal 26 and the prediction residual signal 24'''' directly results in the reconstructed signal 12'. However, it should be noted that, according to alternative embodiments, multiple prediction signals 26 can be combined with the prediction residual signal 24'''' to result in the image 12'.

[0020] In FIG. 3, the conversion block 84 has the following meaning. The transformer 28 and the inverse transformer 54 perform their transformations in units of these transform blocks 84. For example, many codecs use some kind of DST or DCT for all transform blocks 84. Some codecs allow to skip the transform, so that for some transform blocks 84, the prediction residual signal is coded directly in the spatial domain. However, according to the embodiments described below, the encoder 10 and the decoder 20 are configured so that they support several transforms. For example, the transforms supported by the encoder 10 and the decoder 20 may include:

[0021] DCT-II (or DCT-III), where DCT stands for Discrete Cosine Transform DST-IV, where DST stands for Discrete Sine Transform DCT-IV DST-VII Identity Transformation (IT)

[0022] Of course, the transformer 28 supports all of the forward transform versions of these transforms, while the decoder 20 or inverse transformer 54 supports the corresponding inverse transforms or inverse versions thereof.

[0023] Inverse DCT-II (or Inverse DCT-III) Reverse DST-IV ·Inverse DCT-IV Reverse DST-VII Identity Transformation (IT)

[0024] Note that the set of supported transforms may consist of just one transform, such as one spectral-to-spatial transform or one spatial-to-spectral transform.

[0025] As already outlined above, Figures 1-3 are presented as examples in which the inventive concepts further described below can be implemented to form specific examples of encoders and decoders according to the present application. To that extent, the encoders and decoders of Figures 1 and 2 may represent possible implementations of the encoders and decoders described below in this specification. However, Figures 1 and 2 are merely exemplary. However, an encoder according to embodiments of the present application may perform block-based encoding of image 12 using concepts outlined in more detail below and may differ from the encoder of Figure 1 in that, for example, it does not support intra-prediction or the subdivision into blocks 80 is performed in a different way than illustrated in Figure 3. Similarly, a decoder according to an embodiment of the present application may perform block-based decoding of image 12′ from data stream 14 using the coding concepts further outlined below, but may differ from decoder 20 of FIG. 2 in that, for example, it does not support intra-prediction, or it subdivides image 12′ into blocks in a different way than described with respect to FIG. 3, and / or it derives prediction residuals from data stream 14 in, for example, the spatial domain rather than the transform domain.

[0026] There are several applications that utilize resolution adaptation for several purposes, for example, bitrate adaptation for throughput variations or bitrate adaptation for region of interest (RoI) use cases.

[0027] The current VVC draft specifies a process usually called reference picture resampling, which allows having varying picture sizes within a video sequence for the RoI encoding process, as in the examples shown in Figures 6 to 8. To this end, the VVC specification draft includes a maximum picture size in the sequence parameter set (SPS), an actual picture size in the picture parameter set (PPS), and a scaling window offset in the PPS (e.g., the red margin in Figure 8), which allows deriving the scaling ratio that needs to be used between the current picture and the reference picture.

[0028] After describing a possible implementation of an encoder and decoder framework that can incorporate embodiments of the present application, the description preliminarily refers back to current VVC developments to motivate the details of the embodiments outlined later.

[0029] In VVC, the scaling ratio is derived using the width of the image given the scaling window defined in PPS for the current image (PicOutputWidthL) and the reference image (fRefWidth) as follows:

[0030] RefPicScale[i][j][0]= ((fRefWidth<<14)+(PicOutputWidthL>>1)) / PicOutputWidthL RefPicScale[i][j][1]= ((fRefHeight<<14)+(PicOutputHeightL>>1)) / PicOutputHeightL

[0031] PicOutputWidth and PicOutputHeight may also be referred to below as CurrPicScalWinWidth and CurrPicScalWinHeight.

[0032] A scale ratio<1, i.e., a RefPicScale value<(1<<14), means that the current image is larger than the reference image, and a ratio>1, i.e., a RefPicScale value>(1<<14), means that the current image is smaller than the reference image.

[0033] The current VVC draft specifies four interpolation filters for motion compensation down to 1 / 16 sample with fractional sample interpolation. The first one is designed for the case of normal motion compensation without reference picture resampling, RRR, and not affine mode. The second filter is designed for the case when affine mode is used. The remaining two filters are used for downsampling by factors 1.5 and 2. Scaling ratios are allowed from 1 / 8 (8x upsampling) to 2 (2x downsampling).

[0034] Depending on whether affine mode is used and the scaling ratio, one of four filters is used, subject to the following conditions:

[0035] Use affine mode => Affine interpolation filter Scaling ratio > 1.75 => 2x downsampling interpolation filter is used 1.25<scaling ratio<=1.75=>1.5x downsampling interpolation filter is used Scaling ratio <= 1.25 => Normal interpolation filter when there is no RPR

[0036] For resolution changes where the current image is larger than the reference image, or where the ratio value is very small (scaling ratio <= 1.25x downsampling factor) where the current image is smaller than the reference image, the regular interpolation filter is used.

[0037] If affine mode is not used, there is no RPR (scaling ratio = 1), the normal interpolation filter used when the scaling ratio is 1.25 or less, a specific smoothing filter can be applied.

[0038] The 1 / 16 sample normal interpolation filter is defined in VVC as an 8-tap filter. However, the VVC specification defines a special 6-tap smoothing filter to be used in the following cases:

[0039] No Intra-Block Copy (IBC) mode The motion vector difference resolution is 1 / 2 luminance sample resolution.

[0040] This 6-tap smoothing filter is used when adaptive motion vector resolution is used and the motion vector differentials are at half sample resolution. The filter is reproduced in Figure 9.

[0041] Considering the 1 / 16 fractional sample precision in VVC, the fractional sample position p=8 corresponds to a half-sample position (8 / 16=1 / 2). The variable hpelIfIdx equal to 1 indicates whether a 6-tap smoothing filter (highlighted) is used for the half-sample positions. hpelIfIdx is set to 1 when AmvrShift equals 3, indicating the half-sample MVD resolution when not using IBC mode. In this case, it is signaled in the bitstream with the syntax amvr_flag=1, amvr_precision_idx=0. See also Figure 10.

[0042] Without RPR, each sample in the reference block references the same fractional (half-sample) interpolated location, so in the above case a smoothing filter is used. However, when using RPR, each sample can refer to a different partial interpolation location.

[0043] In the following text, the n-sample difference of the current block (x'' L -x' L or y'' L -y' L Note that ) is sensitive to the scaling ratio.

[0044] - each luminance sample position (x L =0...sbWidth-1+brdExtSize,y L =0...sbHeight-1+brdExtSize), the corresponding predicted luminance sample values ​​predSamplesLX[x L ][y L ] is derived as follows:

[0045] -(refxSb L ,refySb L ) and (refx L ,refy L ) is the luminance position indicated by the motion vector (refMvLX[0], refMvLX[1]) given in 1 / 16 sample units. L , refx L , refySb L , refy L is derived as follows:

[0046] refxSb L =(((xSb-scaling_win_left_offset)<<4)+refMvLX[0]) * scalingRatio[0] refx L =((Sign(refxSb L ) * ((Abs(refxSb L +128)>>8)+x L * ((scalingRatio[0]+8)>>4))+fRefLeftOffset+32)>>6 refySb L =(((ySb-scaling_win_top_offset)<<4)+refMvLX[1]) * scalingRatio[1] refy L =((Sign(refySb L ) * ((Abs(refySb L)+128)>>8)+yL * ((scalingRatio[1]+8)>>4))+fRefTopOffset+32)>>6

[0047] Here, scaling_win_left_offset can be calculated as SubWidthCxpps_scaling_win_left_offset, and scaling_win_top_offset can be calculated as SubHeightCxpps_scaling_win_top_offset.

[0048] For example, suppose the current image is twice as large as the reference image. This is illustrated in Figure 4a, where the reference and current image samples are shown spatially overlapping, with crosses indicating samples from the reference (d) image and block circles indicating samples from the current image. The current 4x4 block in the current image is displayed. The motion vector for that block is indicated by an arrow. It is defined at half-pel resolution relative to the pel grid of the current image samples. The location of the block sample in the reference image, to which the reference image should be interpolated to obtain a predictor for the block in the current image, is indicated by a white circle. The motion vector is L = 0 (top left sample of 4x4 block) points to the half sample position, and sample x' L = 1 (the sample to its right) is exemplarily chosen to refer to integer or full sample positions, indicating that both positions are half-pel or full-pel relative to the reference image. In this way, the situation leading to different pixel positions, or in other words, pixels associated with different phases, does not occur in reference and current images that have the same resolution and whose motion vectors are defined at half-pel resolution, as shown in Figure 4b. Similarly, if the scaling is 4x instead of 2x and the sample x' L =0 refers to the half sample position, and sample x' L =1 refers to the quarter sample position, and sample x' L =2 refers to integer sample positions.

[0049] This means that within one block, some samples will use the smoothing filter and some will not, which will result in unpleasant visual effects and visible artifacts.

[0050] In one embodiment, the derivation of the variable hpelIfIdx in the motion compensation step is modified to include the scaling ratio between the current and reference images as follows:

[0051] If AmvrShift is equal to 3 and scaling ratio==1, i.e. RefPicScale is equal to 16384, then hpelIfIdx=1.

[0052] 4c shows a video decoder according to the present invention. The video decoder 410 uses motion compensated prediction to decode video 450 from the data stream 420. The motion prediction may be performed in a motion prediction section 440 and is based on a first motion vector 423 and a second motion vector 424 transmitted in the data stream 420.

[0053] A first motion vector 423 is transmitted in the data stream 420 at a first resolution, which is half-sample resolution, and a second motion vector 423 is transmitted in the data stream 420 at a second resolution, which is different from the first resolution.

[0054] Motion compensation is performed between a first image 421 of equal image resolution and a second image 422 of different image resolution, i.e., RPR is supported, or in other words, a motion vector can point from the current image to a reference image of the same resolution. Both current images form the first image, and a motion vector points from the current image to a reference image of a different resolution than the current image, and then both form the second image. Thus, the size and / or resolution of the images may vary, as signaled in the data stream. Motion compensation is performed using interpolation filters 431 and 432 to obtain sub-sample values ​​in the reference image, i.e., in the reference sample array.

[0055] Video decoder 410, for example, in selection portion 430, selects an interpolation filter for a given first motion vector from first interpolation filter version 431 and second interpolation filter version 432. Second interpolation filter version 432 has better edge-preserving properties than first interpolation filter version 431. As will be shown in more detail below, this selection may be specific to a particular phase of a sample, or, stated differently, to a particular sub-pel position, such as a half-pel position.

[0056] The selection of an interpolation filter depends on whether the current image to which the given first motion vector belongs has an image resolution equal to that of the reference sample array to which the given first motion vector relates. The selection and equality check can be performed separately for each dimension, i.e., in the horizontal and / or vertical dimension. Additionally or alternatively, the selection can also depend on constraint information 425 transmitted in the data stream, as outlined in more detail below.

[0057] The dependencies of the constraint information 425 are not shown in FIG. 4c.

[0058] Furthermore, the encoder and decoder can obtain full sample values ​​in the reference sample array for a given first motion vector without using an interpolation filter. In Figures 4a and 4b, when transmitting a data stream at half-pel resolution, for example, using AmvrShift=3, the motion vector shown there is the "first motion vector." The positions to which the samples of the predicted block are shifted according to each motion vector are indicated by circles in Figures 4 and 4b. The circles corresponding to the crosses are "full sample values." These can be determined directly from the positioned samples of the reference image (cross) by interpolation. That is, the sample values ​​of the samples of the reference image to which the shifted positions of the samples of the inter-predicted block directly correspond are directly used as predictors for the samples of the inter-predicted block to which the shifted positions correspond. Naturally, the same can be applied to the shifted sample positions of the inter-predicted block having a second motion vector, i.e., a motion vector transmitted in a data stream at a resolution other than half-pel.

[0059] The decoder can also use an additional interpolation filter to obtain non-half-sample subsample values, for example, a filter with better edge-preserving properties than the first interpolation filter version. This is shown again in Figures 4a and 4b. A "non-half-sample subsample value" refers to a circle that does not correspond to any reference image sample or to the middle of two horizontally, vertically, or diagonally adjacent samples of the reference image, i.e., does not correspond to any cross or to the middle of two horizontally, vertically, or diagonally adjacent crosses. Similarly, an interpolation filter with better edge-preserving properties is used. See Figure 4d. This is shown for a quarter-pel position. The shifted position of the top left and second samples of the block is the quarter-pel position. This is the sample position 12 / 16th of a full sample pitch to the right from the reference image sample (the top left cross in Figure 4d). This is the sample position 12 / 16th of a full sample pitch to the right from the reference image sample (the top left cross in Figure 4d). It is an FIR filter. This table consists of filter coefficients applied to the reference image samples where the interpolated sample locations are located. Figure 4d highlights the entries that define the interpolation filter and shows how some of the reference image samples are weighted by this filter to obtain the interpolated quarter-pel samples. Note that horizontal interpolation may be applied first to obtain sample values ​​at sub-pel locations between reference image samples, and then vertical interpolation may be performed using these interpolated intermediate samples to obtain the actual sub-pel samples required. The required sub-pel samples are offset from the reference image samples vertically and horizontally, or vice versa, i.e., first vertically and then horizontally, with sub-sample accuracy. Again, the choice of the two half-pel sample location interpolation filter versions can be made separately in the horizontal and vertical directions, or globally in both directions, depending on the image resolution equivalence in both dimensions.

[0060] As mentioned above, the selection can be performed separately for horizontal and vertical interpolation. This is illustrated in Figure 4d by two entries for half-pel position 8 / 16: which filter to use depends on hpelIfIdx. The latter variable is set, for example, depending on whether the resolutions of the reference and current images are the same. The latter equality check can be performed separately for a and y as shown below using the terms hpelHorIfIdx and hpelVerIfIdx. In particular, if the horizontal image resolutions of the current image and the reference sample array are unequal, a second interpolation filter (a filter with more edge-preserving properties) may be selected for horizontal interpolation. This is the filter defined in the table row where hpelIfIdx=0. Similarly, if the horizontal image resolutions of the current image and the reference sample array are unequal, for example, a second interpolation filter (a filter with more edge-preserving properties) may be selected for horizontal interpolation. Also, if the horizontal and vertical image resolutions of the current image and the reference sample array are unequal, a second interpolation filter (a filter with more edge-preserving properties) may be selected for horizontal and vertical interpolation. For any direction where the second interpolation filter is not used, the first interpolation filter is used, ie the filter in the row of the table where hpelIfIdx=1.

[0061] The choice of which of the two half-pel position interpolation filters to use can be interpreted as being made for all motion vectors, not just half-pel motion vectors. In a broader sense, the choice between the two also depends on whether the motion vectors are at half-sample resolution. If so, the selection is made as described above depending on the resolution equivalence between the reference image and the current image. If not, the selection necessarily goes to the second interpolation filter with better edge-preserving properties.

[0062] As is clear from the above description, a decoder can use the alphabet of one or more syntax elements in the data stream to determine the resolution at which a particular motion vector is transmitted in the data stream. For example, adaptive motion vector resolution is indicated by amvr_flag, which, when set, accommodates deviations from a certain default motion vector resolution, and the resolution precision of the adaptive motion vector can be indicated by the index amvr_precision_idx. This syntax is decoded by the decoder and used to derive the resolution at which the motion vector of a particular 'prediction block' is transmitted in the data stream, and this syntax is correspondingly coded by the encoder to indicate the resolution of the motion vector.

[0063] Decoders and encoders may exclude half-pel resolution from the set of signalable settings for motion vector resolution. They may map the alphabet of one or more syntax elements onto a first set of vector resolutions that does not include half-sample resolutions if one of the following conditions is met (while otherwise the mapping is done onto a second set of vector resolutions that does include half-sample resolutions):

[0064] The constraint information indicates, for example, that for the current image, the filter version with the lower edge preserving property is disabled. This may be indicated, for example, in the image or slice header and indicated, for example, by a ph_disable_hpel_smoothing_filter or sh_disable_hpel_smoothing_filter constraint equal to 1. indicates that the current image to which the given first motion vector belongs and the reference sample sequence to which the given first motion vector is associated differ in at least one dimension of the image resolution. The constraint information indicates whether resampling of the reference sample array is enabled. This can be indicated, for example, at the sequence level in the sequence parameter set (SPS). An example of such an indication is sps_ref_picture_resample_enable_flag equal to 1, which causes a filter version with higher edge-preserving properties to be used when resampling of the reference sample array is enabled.

[0065] If none of the above conditions are met, the decoder maps the alphabet to a second set of vector resolutions, which includes half-sample resolutions.

[0066] It should also be noted that the data stream may include information as to whether temporally successive images have the same or different horizontal and / or vertical image resolution dimensions.

[0067] Furthermore, as mentioned above, the current image may specifically have an image resolution equal to the reference sample array in the horizontal and vertical dimensions.

[0068] The reference sample array can also be a region, a sub-image, or an image.

[0069] The decoder may also derive the constraint information from the data stream either on a per-sequence-of-pictures, per-picture, or per-slice basis.

[0070] FIG. 5 shows a video encoder according to the present invention. The same principles apply there as in the decoder. In summary, a video encoder 510 encodes a video 550 into a data stream 520 using motion compensated prediction. The motion prediction can be performed in a motion prediction section 540. The encoder 510 indicates the in by sending a first motion vector 523 and a second motion vector 524 in the data stream 520.

[0071] A first motion vector 523 is transmitted in the data stream 520 at a first resolution that is half sample resolution, and a second motion vector 523 is transmitted in the data stream 520 at a second resolution that is different from the first resolution.

[0072] Motion compensation is performed between a first image 521 with the same image resolution and a second image 522 with a different image resolution using interpolation filters 531 and 532 to obtain sub-sample values ​​in the reference image, i.e., in the reference array.

[0073] For example, in selection portion 530, video encoder 510 selects an interpolation filter for a given first motion vector from first interpolation filter version 531 and second interpolation filter version 532. Second interpolation filter version 532 has better edge-preserving properties than first interpolation filter version 531.

[0074] The selection of the interpolation filter depends on whether the current image to which the given first motion vector belongs has an image resolution equal in the horizontal and / or vertical dimensions to the reference sample array to which the given first motion vector relates. Additionally or alternatively, the selection may also depend on constraint information 525 transmitted in the data stream.

[0075] As mentioned earlier, the same principles that can be implemented by a decoder can also be implemented by an encoder.

[0076] Therefore, the encoder may also obtain the full sample values ​​in the reference sample array for a given first motion vector without using an interpolation filter.

[0077] The encoder may also use an additional interpolation filter to obtain non-half-sampled sub-sampled values, for example, using a filter with better edge-preserving properties than the first interpolation filter version.

[0078] As mentioned above, selection can be performed separately for horizontal and vertical interpolation.

[0079] Specifically, if the horizontal image resolutions of the current image and the reference sample array are not equal, a second interpolation filter (one with better edge preserving properties) may be selected for horizontal interpolation.

[0080] Similarly, for example, if the horizontal image resolutions of the current image and the reference sample array are not equal, a second interpolation filter (one with better edge preserving properties) may be selected for horizontal interpolation.

[0081] Also, if the current image and the reference sample array are not equal in horizontal and vertical image resolution, a second interpolation filter that is a more edge-preserving filter can be selected for horizontal and vertical interpolation. A further selection can be performed depending on whether a given first motion vector is at half-sample resolution.

[0082] Additionally, to select the resolution of the motion vectors, the encoder may avoid using half-sample resolution for one or more vectors if the current image has an image resolution equal to the reference sample array in the horizontal and / or vertical dimensions.

[0083] For selection, the encoder can map the alphabet of one or more syntax elements in the data stream that indicate the resolution of a given first motion vector. For example, the resolution of the adaptive motion vector can be indicated by amvr_flag, and the resolution precision of the adaptive motion vector can be indicated by amvr_precision_idx.

[0084] The encoder may map the alphabet to a first set of vector resolutions that do not constitute half-sample resolutions if any of the following conditions are met:

[0085] The constraint information indicates, for example, that for the current image, the filter version with the lower edge preserving property is disabled. This may be indicated, for example, in the image or slice header and indicated, for example, by a ph_disable_hpel_smoothing_filter or sh_disable_hpel_smoothing_filter constraint equal to 1. indicates that the current image to which the given first motion vector belongs and the reference sample sequence to which the given first motion vector is associated differ in at least one dimension of the image resolution. The constraint information indicates whether resampling of the reference sample array is enabled. This can be indicated, for example, at the sequence level in the sequence parameter set (SPS). An example of such an indication is sps_ref_picture_resample_enable_flag equal to 1, which causes a filter version with higher edge-preserving properties to be used when resampling of the reference sample array is enabled.

[0086] If none of the above conditions are met, the encoder can map the alphabet to a second set of vector resolutions, which includes half-sample resolutions.

[0087] It should also be noted that the data stream may include information as to whether temporally successive images have the same or different horizontal and / or vertical image resolution dimensions.

[0088] Furthermore, as mentioned above, the current image may specifically have an image resolution equal to the reference sample array in the horizontal and vertical dimensions.

[0089] The reference sample array can also be a region, a sub-image, or an image.

[0090] The encoder can also derive constraint information from the data stream either on a per-sequence-of-pictures, per-picture, or per-slice basis.

[0091] Finally, the above principles may also be embodied in a computer program product including a program having software code portions for employing the above principles when the program is executed on a processing device, and further, the computer program product may also be embodied as a computer-readable medium having the software code portions stored thereon.

[0092] The principles presented above and below can also be embodied in a data stream generated by encoding or by an encoder as described in this document.

[0093] Returning to the description of the embodiment of modifying the current VVC draft, for example, in Figure 5, the dependency of the constraint information is not shown.

[0094] In one embodiment, the derivation of the variable hpelIfIdx in the motion compensation step is modified to incorporate the valid flag of the reference picture signal at the sequence level in the SPS, and the smoothing filter coefficients are used only if reference picture resampling is inhibited as follows:

[0095] If AmvrShift equals 3 and sps_ref_picture_resample_enable_flag=0, then hpelIfIdx=1, i.e. reference picture resampling is disabled.

[0096] In another embodiment, a control syntax flag is added to the image or slice header that indicates whether the smoothing filter is disabled for the current image. hpelIfIdx is then derived as follows:

[0097] If AmvrShift is equal to 3 and a control flag is equal to 0 (for example, ph_disable_hpel_smoothing_filter or sh_disable_hpel_smoothing_filter), then hpelIfIdx = 1.

[0098] In another embodiment, the derivation of the variable AmvrShift is modified to include information about the resampling of the reference image as follows, and to avoid a value equal to 3 if:

[0099] ph_disable_hpel_smoothing_filter or sh_disable_hpel_smoothing_filter is equal to 1, or Scaling ratio !=1, i.e. RefPicScale is not equal to 16384, or ·sps_ref_picture_resample_enable_flag==1

[0100] In another embodiment, if RPR is used for the reference image, i.e., the current image and the reference image have unequal sizes, or if the scaling ratio resulting from the scaling window is not equal to 1, i.e., RefPicScale is not equal to 16384, then it is a bitstream constraint that AmvrShift is not equal to 3.

[0101] Therefore, the horizontal and vertical half-sample interpolation filter indices hpelHorIfIdx and hpelVerIfIdx are derived as follows:

[0102] hpelHorIfIdx=(scalingRatio[0]==16384)?hpelIfIdx:0 hpelVerIfIdx=(scalingRatio[1]==16384)?hpelIfIdx:0

[0103] While some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or device corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware apparatus, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

[0104] The data stream of the present invention can be stored on a digital storage medium or can be transmitted over a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

[0105] Depending on specific implementation requirements, embodiments of the present invention can be implemented in hardware or software. Implementation can be performed using a digital storage medium, such as a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory, on which electronically readable control signals are stored, which cooperate (or can cooperate) with a programmable computer system to execute the respective methods. Thus, the digital storage medium may be computer-readable.

[0106] Some embodiments of the present invention include a data carrier having electronically readable control signals that can cooperate with a programmable computer system to perform one of the methods described herein.

[0107] Generally, embodiments of the present invention can be implemented as a computer program product comprising program code that functions to perform one of the methods when the computer program product is run on a computer, and the program code can be stored, for example, on a machine-readable carrier.

[0108] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0109] In other words, an embodiment of the inventive method is therefore a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer. A further embodiment of the inventive method is therefore a data carrier (or digital storage medium, or computer-readable medium) having recorded thereon a computer program for performing one of the methods described herein. The data carrier, digital storage medium, or recording medium is typically tangible and / or non-transitory.

[0110] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or sequence of signals may be configured to be transferred over a data communication connection, for example over the Internet.

[0111] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0112] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein. Further embodiments according to the invention comprise an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

[0113] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0114] The apparatus described herein can be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer. The apparatus described herein, or any components of the apparatus described herein, can be implemented at least in part in hardware and / or software.

[0115] The methods described herein can be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0116] The methods described herein, or any components of the apparatus described herein, may be performed at least in part by hardware and / or software.

[0117] The above-described embodiments are merely illustrative of the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to others skilled in the art. It is the intention, therefore, to be limited only by the scope of the appended claims, and not by the specific details presented by the description and illustration of the embodiments herein.

Claims

1. 1. A method for decoding an image from a data stream, comprising: selecting a horizontal dimension interpolation filter (431, 432) based on whether the horizontal dimension of the current image (421) and the horizontal dimension of the reference image are the same; if the horizontal dimension of the current image and the horizontal dimension of the reference image are the same, the selected horizontal dimension interpolation filter is a first interpolation filter (431) having six non-zero coefficients: 3, 9, 20, 20, 9, 3; If the horizontal dimension of the current image and the horizontal dimension of the reference image are different, the selected horizontal dimension interpolation filter is a second interpolation filter (432) having eight non-zero coefficients: −1, 4, −11, 40, 40, −11, 4, and −1. selecting the horizontal dimension interpolation filters (431, 432); selecting a vertical dimension interpolation filter (431, 432) based on whether the vertical dimension of the current image and the vertical dimension of the reference image are the same; if the vertical dimension of the current image and the vertical dimension of the reference image are the same, the selected vertical dimension interpolation filter is the first interpolation filter (431); if the vertical dimension of the current image and the vertical dimension of the reference image are different, the selected vertical dimension interpolation filter is the second interpolation filter (432); selecting the vertical dimension interpolation filters (431, 432); predicting the current image based on the horizontal dimension interpolation filter and the vertical dimension interpolation filter; A method comprising:

2. 1. A method of decoding video, comprising decoding video from a data stream using motion compensated prediction, the motion compensated prediction being performed using interpolation filters (431, 432) to obtain sub-sample values ​​in a reference image, the method comprising: selecting an interpolation filter from a first interpolation filter (431) having six non-zero coefficients of 3, 9, 20, 20, 9, 3 and a second interpolation filter (432) having eight non-zero coefficients of -1, 4, -11, 40, 40, -11, 4, -1 for horizontal and vertical interpolation to obtain half-sample values ​​in a reference image for a predetermined first motion vector belonging to the current image and related to said reference image; selecting the first interpolation filter (431) as a horizontal dimension interpolation filter for horizontal interpolation when the horizontal dimension of the current image and the horizontal dimension of the reference image are the same, and selecting the second interpolation filter (432) as the horizontal dimension interpolation filter for horizontal interpolation when the horizontal dimension of the current image and the horizontal dimension of the reference image are different; selecting the first interpolation filter (431) as a vertical dimension interpolation filter for vertical interpolation when the vertical dimension of the current image and the vertical dimension of the reference image are the same, and selecting the second interpolation filter (432) as the vertical dimension interpolation filter for vertical interpolation when the horizontal dimension of the current image and the horizontal dimension of the reference image are different; selecting the interpolation filter by The method further comprises:

3. using motion compensation for decoding an image from a data stream, the motion compensation being performed using interpolation filters (431, 432) to obtain sub-sample values ​​in a reference image, the motion compensation being based on a first motion vector and a second motion vector transmitted in the data stream, the motion compensation being performed between a first image (421) of the same image resolution and a second image (422) of a different image resolution; using said motion compensation; further comprising the first motion vector is transmitted at half-sample resolution and the second motion vector (424) is transmitted at a resolution different from half-sample resolution; The method according to claim 1 or 2.

4. The method of claim 2 , further comprising selecting the second interpolation filter (432) for a second motion vector.

5. if at least one of a set of conditions is satisfied, mapping an alphabet of one or more syntax elements in the data stream that indicate a resolution of the first motion vector onto a first set of vector resolutions that does not include half-sample resolutions; if none of the set of conditions is met, mapping an alphabet of one or more syntax elements in the data stream that indicate a resolution of the first motion vector onto a second set of vector resolutions that includes a half-sample resolution; further comprising The set of conditions is: (i) constraint information indicating that the first interpolation filter is invalid, such that the first interpolation filter is invalid for the current image; (ii) the current image to which the first motion vector belongs is unequal in at least one dimension of image resolution to the reference sample array to which the first motion vector relates; (iii) constraint information indicating that resampling of the reference sample array is enabled, such that the second interpolation filter is used when reference sample array resampling is enabled; and Including, The method of claim 4.

6. The method of claim 1 or 2, further comprising using a further interpolation filter to obtain non-half-sampled sub-sampled values.

7. determining, from one or more syntax elements in the data stream, a resolution at which motion vectors are transmitted in the data stream.

3. The method according to claim 1 or 2.

8. The method of claim 1 or 2, further comprising obtaining full sample values ​​in the reference image without using an interpolation filter.

9. 9. A method for detecting a plurality of stimuli, comprising: at least one processor configured to perform the method according to any one of claims 1 to 8; A video decoder (410) for decoding images from the data stream.

10. A computer-readable medium having a computer program stored thereon, the computer program comprising software code portions configured to perform the steps of the method according to any one of claims 1 to 8 when the computer program is run on a processing device. Computer-readable medium.

11. 1. A method for encoding an image into a data stream, comprising: selecting a horizontal dimension interpolation filter (431, 432) based on whether the horizontal dimension of the current image (421) and the horizontal dimension of the reference image are the same; if the horizontal dimension of the current image and the horizontal dimension of the reference image are the same, the selected horizontal dimension interpolation filter is a first interpolation filter (531) having six non-zero coefficients: 3, 9, 20, 20, 9, 3; If the horizontal dimension of the current image and the horizontal dimension of the reference image are different, the selected horizontal dimension interpolation filter is a second interpolation filter (532) having eight non-zero coefficients: −1, 4, −11, 40, 40, −11, 4, and −1. selecting the horizontal dimension interpolation filters (431, 432); selecting a vertical dimension interpolation filter (531, 532) based on whether the vertical dimension of the current image and the vertical dimension of the reference image are the same; if the vertical dimension of the current image and the vertical dimension of the reference image are the same, the selected vertical dimension interpolation filter is the first interpolation filter (531); if the vertical dimension of the current image and the vertical dimension of the reference image are different, the selected vertical dimension interpolation filter is the second interpolation filter (532); selecting the vertical dimension interpolation filters (531, 532); predicting the current image based on the horizontal dimension interpolation filter and the vertical dimension interpolation filter; A method comprising:

12. 1. A method of encoding video, comprising encoding video (550) into a data stream (520) using motion compensated prediction, the motion compensated prediction being performed using interpolation filters (531, 532) to obtain sub-sample values ​​in a reference image, the method comprising: selecting an interpolation filter from a first interpolation filter (531) having six non-zero coefficients of 3, 9, 20, 20, 9, 3 and a second interpolation filter (532) having eight non-zero coefficients of -1, 4, -11, 40, 40, -11, 4, -1 for horizontal and vertical interpolation to obtain half-sample values ​​in a reference image for a predetermined first motion vector belonging to the current image and related to said reference image; selecting the first interpolation filter (531) as a horizontal dimension interpolation filter for horizontal interpolation when the horizontal dimension of the current image and the horizontal dimension of the reference image are the same, and selecting the second interpolation filter (532) as the horizontal dimension interpolation filter for horizontal interpolation when the horizontal dimension of the current image and the horizontal dimension of the reference image are different; selecting the first interpolation filter (531) as a vertical dimension interpolation filter for vertical interpolation when the vertical dimension of the current image and the vertical dimension of the reference image are the same, and selecting the second interpolation filter (532) as the vertical dimension interpolation filter when the horizontal dimension of the current image and the horizontal dimension of the reference image are different; selecting the interpolation filter by The method further comprises:

13. using motion compensation to decode an image from a data stream, the motion compensation being performed using interpolation filters (531, 532) to obtain sub-sample values ​​in a reference image, the motion compensation being based on a first motion vector and a second motion vector transmitted in the data stream, the motion compensation being performed between a first image (421) of the same image resolution and a second image (422) of a different image resolution; using said motion compensation; further comprising the first motion vector is transmitted at half-sample resolution and the second motion vector (424) is transmitted at a resolution different from half-sample resolution; 13. The method of claim 11 or 12.

14. The method of claim 13, further comprising selecting the second interpolation filter (432) for a second motion vector.

15. if at least one of a set of conditions is satisfied, mapping an alphabet of one or more syntax elements in the data stream that indicate a resolution of the first motion vector onto a first set of vector resolutions that does not include half-sample resolutions; if none of the set of conditions is met, mapping an alphabet of one or more syntax elements in the data stream that indicate a resolution of the first motion vector onto a second set of vector resolutions that includes a half-sample resolution; further comprising The set of conditions is: (i) constraint information indicating that the first interpolation filter is invalid, such that the first interpolation filter is invalid for the current image; (ii) the current image to which the first motion vector belongs is unequal in at least one dimension of image resolution to the reference sample array to which the first motion vector relates; (iii) constraint information indicating that resampling of the reference sample array is enabled, such that the second interpolation filter is used when reference sample array resampling is enabled; and Including, 15. The method of claim 14.

16. 13. The method of claim 11 or 12, further comprising using a further interpolation filter to obtain the non-half-sampled sub-sampled values.

17. 13. The method of claim 11 or 12, further comprising determining the resolution at which motion vectors are transmitted in the data stream from one or more syntax elements in the data stream.

18. The method of claim 11 or 12, further comprising obtaining full sample values ​​in the reference image without using an interpolation filter.

19. A processor configured to perform the method of any one of claims 11 to 18, A video encoder (410) for encoding images into a data stream.

20. A computer-readable medium having a computer program stored thereon, the computer program comprising software code portions configured to perform the steps of the method according to any one of claims 11 to 18 when the computer program is run on a processing device. Computer-readable medium.

Images