Composite forecasting and restricted merging

Abstract

To provide a video codec that improves coding efficiency by using a composed prediction signal to predict a predetermined block of a picture.SOLUTION: A video decoder comprises a processor or an electronic circuit, or a programmed computer configured to derive a first prediction value p1 for the current block 106 by using an inter-prediction mode 104, and derive a second prediction value p2 for the current block by using an intra-prediction mode 102, in order to decode a video from a data stream by using block-based predictive decoding, and further determine a scalar weighting parameter α based on prediction parameter information of a block adjacent to the current block, and define the weighted sum of the first prediction value p1 and the second prediction value p2 as α p1+(1-α) p2 by using the scalar weighting parameter α, thereby determining a composite prediction signal (108)q for the current block.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] This application relates to video coding / decoding. [Background technology]

[0002] All relevant video coding standards, such as AVC / H.264 or HEVC / H.265, are To generate the prediction signal, these standards allow two possible modes: In AVC / H.264, the decision between these two modes can be made at the macroblock (16x16 luma samples) level, and in HEVC / H.265 at the Coding Unit (CU) level, which can be of variable size. In intra prediction, sample values ​​of already reconstructed neighboring blocks of the current block may be used to generate a prediction signal. How this intra prediction signal is formed from the reconstructed neighboring sample values ​​is specified by the intra prediction mode. In inter prediction, already reconstructed frames (in coding order) may be used to generate a prediction signal. For inter prediction, in both AVC / H.264 and HEVC / H.265, either uni-prediction or bi-prediction is used. For uni-prediction, the prediction The signal is a shifted and interpolated region of a so-called reference picture. The reference picture used is specified by a reference index, which determines the (possibly interpolated) position within the reference picture. The location of the (interpolated) region is specified (relative to the current block) by a motion vector. The motion vector itself is predictively coded in conjunction with the motion vector predictor, so that only the motion vector differential needs to be actually coded. In HEVC / H.265, the motion vector predictor is selected by signaling a motion vector predictor index. In both AVC / H.264 and HEVC / H.265, motion vectors are coded with quarter-pel (qpel) accuracy. A process of generating such an (interpolated) prediction signal may be specified. The process of generating such a (interpolated) prediction signal is also called motion compensated prediction. In bi-prediction, two motion compensated prediction signals are linearly superimposed (usually using a factor of 0.5 for both constituent prediction signals). Therefore, in the case of bi-prediction, two reference indexes and a motion vector differential (and in HEVC / H.265, a motion vector predictor index) must be transmitted.

[0003] To simplify the coding of nearby areas with the same motion characteristics, HEVC / H.265 supports so-called merge modes, in which prediction parameters (i.e., reference indices and motion vectors) of blocks that are either locally adjacent or temporally co-located can be reused for the current block. The skip mode in HEVC / H.265 is a particular case of merging, in which the prediction residual is not transmitted.

[0004] Although the available and supported prediction modes of today's video codecs are already quite effective at keeping prediction residuals small with a reasonable amount of prediction side information needed to control the prediction use of these prediction modes, it would be advantageous to further improve the coding efficiency of block-based predictive video codecs. Summary of the Invention [Problem to be solved by the invention]

[0005] It is therefore an object of the present invention to provide a video codec using block-based predictive coding with improved coding efficiency.

[0006] This object is achieved by the subject matter of the independent claims of the present application.

[0007] The basic idea underlying the present invention is that increased coding efficiency can be achieved by predicting certain blocks of a picture using a composite prediction signal. [Means for solving the problem]

[0008] According to one embodiment, the number of primitive predictions that are combined to result in a composite prediction signal is allowed to exceed two, or, in other words, to include a first prediction signal on the basis of which a given block is finally predicted. The number of further prediction signals that contribute to the composite prediction signal may be more than one. The maximum number of contributing prediction signals or primitive predictions may be limited by a default value or by some value signaled in the data stream. The possibility of allowing such a large number of contributing prediction signals or primitive predictions per composite prediction signal for a given block gives the possibility of an inherent noise reduction of the composite prediction signal by exploiting the mutual noise reduction of the independent noise components of the individual prediction contributions.

[0009] According to an embodiment of the present application, the number of contributing primitive predictions or synthesized prediction signals on which a given block is predicted, i.e. the cardinality of the set of primitive predictions on which the composite prediction signal is formed, is subject to local variation at sub-picture granularity. By using spatial and / or temporal prediction to control the fluctuations, with or without explicit information conveyed in the data stream to estimate the signal quality. Signaling overhead can be kept low. In the future, the concept of merging blocks, for example as used in HEVC, can be extended to not only involve the contribution of the first prediction signal or the first primitive prediction to the final assembled prediction signal, but also the definition of further primitive predictions or further prediction signals.For example, the number of contributing primitive predictions or prediction signals and their associated prediction parameters and prediction modes can be adopted from the merging candidates, thereby further reducing the signaling overhead that would otherwise result from the increase in the number of contributing primitive predictions and prediction signals, respectively.

[0010] According to embodiments of the present application, the manner in which the contributing primitive predictions or prediction signals are combined to result in a composite prediction signal is controlled via side information in the data stream. In particular, according to some embodiments of the present application, the individual primitive predictions or prediction signals are successively summed. A second primitive prediction or a first further prediction signal is added to the first primitive prediction or the first further prediction signal to form a first intermediate sum. To control this first addition, the data sequence for a given block is In the summation, the contribution weights are signaled in the stream. This contribution value is used to weight the addend formed by the current primitive prediction or further prediction signal, i.e. the second primitive prediction or the first further prediction signal, respectively, while one minus the contribution weight is used to weight the addend formed by the current primitive prediction or further prediction signal, i.e. the second primitive prediction or the first further prediction signal, respectively. Similarly, the third primitive prediction or the first further prediction signal to the just mentioned intermediate summation is used to weight the addend formed by the current primitive prediction or further prediction signal, i.e. the second primitive prediction or the first further prediction signal, respectively. In order to control the addition of the primitive prediction or the second further prediction signal, A second contribution is sent to the block, and so on. The contribution weights are controlled at a sub-picture granularity, such as the unit of the block itself. In controlling the contributions in this way, the side information overhead for controlling the assembly can be kept low. In particular, according to an embodiment of the present application, the contribution weights are selected by the encoder and are signaled in the data stream using a discrete value domain of a distinct number of values ​​that each contribution weight can take. For example, this number of discrete weight values ​​may be equal for each addition performed in succession, i.e. for all contribution weights, and despite this limitation, this effect can be determined not only by the contribution values ​​of these earlier primitive predictions or further prediction signals, but also by the products of the contribution weights of the primitive predictions and further prediction signals, respectively, that are added subsequently. Through the fact that effective weights are actually formed, a fine setting of the effective weights can be achieved where earlier primitive predictions or earlier further prediction signals contribute to the composite prediction signal. For implementation, the computational overhead for performing successive additions can be kept low by subjecting at least some of the intermediate sums to clipping and / or rounding operations, or some of the successively performed additions to clipping and / or rounding operations. As far as the encoder is concerned, advantageously, since testing of the individual primitive predictions or prediction signals, respectively, is already largely performed in existing implementations of encoders, the increased freedom in constructing the prediction signals translates into a reasonable increase in computational overhead, so that successive additions only result in a reasonable increase in encoder overhead compared to the increase in coding efficiency afforded by the new freedom in constructing the prediction signals.

[0011] According to a further aspect of the present application, used in combination with the above concepts or independently therefrom, merging is allowed to be controllable by syntax in the data stream: merge candidate restriction signaling may activate the restriction of merge candidate set configuration to bi-predictive prediction parameter merge candidates, and if so, a hypothesis selection signaling may be used to select one of the hypotheses of the prediction parameter merge candidates that is finally selected. Alternatively, the merge candidate restriction signaling may activate restricted merging, and if so, a hypothesis selection indication is added to select one of the hypotheses of the prediction parameter merge candidates that is finally selected. Here, the configuration is uni-predictive In this method, both bi-prediction and bi-prediction candidates are put into the set, but if a bi-prediction candidate is selected, only the selected hypothesis is used for the uni-prediction processing of the current block. This method makes the merging concept more effective by simply adding a reasonable amount of side information to adapt the merging procedure.

[0012] Advantageous aspects of the present application are the subject matter of the dependent claims. Preferred embodiments of the present application are described below with reference to the figures. [Brief description of the drawings]

[0013] [Figure 1] 1 is a block diagram of an apparatus for predictively coding video, as an example for a video decoder in which a hybrid prediction concept according to an embodiment of the present application may be implemented. [Diagram 2] 2 is a block diagram of an apparatus for predictively decoding video, which corresponds to the apparatus of FIG. 1 as an example for a video decoder in which a hybrid prediction concept according to an embodiment of the present application can be implemented. [Diagram 3] 1A and 1B are schematic diagrams showing an example for the relationship between a prediction residual signal, a prediction signal, and a reconstructed signal, showing the possibility of setting subdivision for defining a prediction signal, processing a prediction residual signal, etc. [Figure 4] 4A-4D are schematic diagrams illustrating the construction of a prediction signal according to one embodiment using an iterative approach or a recursive relationship, respectively; [Diagram 5] FIG. 1 is a schematic diagram illustrating how individual primitive predictions can be specified through the use of implicit and / or explicit signaling. [Figure 6]FIG. 6 is a schematic diagram illustrating the concept of assembling a prediction signal in an iterative manner as shown in FIG. 5, with one primitive prediction contributing to the composite prediction signal each iteration in a manner that is weighted by a contribution coefficient that is added to an intermediate sum, and then weighted by one minus the contribution coefficient. [Figure 7] FIG. 7 is a schematic diagram illustrating a value domain of signallable discrete values ​​for contribution values ​​for an additional further primitive prediction in FIG. [Figure 8] FIG. 1 is a schematic diagram illustrating the possibility of combining the concept of hybrid prediction with the concept of merging. [Figure 9] FIG. 13 illustrates an example for a prediction unit syntax for signaling further primitive predictions that are necessarily inter mode, with an example showing that some further implicitly defined primitive predictions may be derived from merge candidates. [Figure 10] FIG. 10 is a diagram illustrating an example syntax for a prediction unit syntax that differs from FIG. 9 in that no additional implicitly defined primitive prediction is provided. [Figure 11 (Part 1)] FIG. 11 is a diagram showing an example syntax for a prediction unit syntax in which, in addition to the example of FIG. 10, additional syntax is expended to allow only partial inheritance of hypotheses for the first primitive prediction of B prediction. [Figure 11 (Part 2)] FIG. 11 is a diagram showing an example syntax for a prediction unit syntax in which, in addition to the example of FIG. 10, additional syntax is expended to allow only partial inheritance of hypotheses for the first primitive prediction of B prediction. [Figure 12 (Part 1)] FIG. 13 is a diagram showing an example syntax for coding unit syntax, indicating that the concept of composite prediction may also be applied to blocks where the first prediction is of intra prediction mode, and additionally indicating that the selection result is signaled in the data stream such that the further primitive predictions signaled may be of either intra prediction mode or inter prediction mode. [Figure 12 (Part 2)]FIG. 13 is a diagram showing an example syntax for coding unit syntax, indicating that the concept of composite prediction may also be applied to blocks where the first prediction is of intra prediction mode, and additionally indicating that the selection result is signaled in the data stream such that the further primitive predictions signaled may be of either intra prediction mode or inter prediction mode. [Figure 12 (Part 3)] FIG. 13 is a diagram showing an example syntax for coding unit syntax, indicating that the concept of composite prediction may also be applied to blocks where the first prediction is of intra prediction mode, and additionally indicating that the selection result is signaled in the data stream such that the further primitive predictions signaled may be of either intra prediction mode or inter prediction mode. [Figure 12 (Part 4)] FIG. 13 is a diagram showing an example syntax for coding unit syntax, indicating that the concept of composite prediction may also be applied to blocks where the first prediction is of intra prediction mode, and additionally indicating that the selection result is signaled in the data stream such that the further primitive predictions signaled may be of either intra prediction mode or inter prediction mode. [Figure 13 (Part 1)] FIG. 13 illustrates an example syntax for the prediction unit syntax called by the CU syntax of FIG. 12, showing that even for inter-predicted blocks, additional primitive predictions may be signaled in one of intra-prediction and inter-prediction modes. [Figure 13 (Part 2)] FIG. 13 illustrates an example syntax for the prediction unit syntax called by the CU syntax of FIG. 12, showing that even for inter-predicted blocks, additional primitive predictions may be signaled in one of intra-prediction and inter-prediction modes. [Figure 13 (Part 3)]FIG. 13 illustrates an example syntax for the prediction unit syntax called by the CU syntax of FIG. 12, showing that even for inter-predicted blocks, further primitive predictions may be signaled in one of intra-prediction and inter-prediction modes. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] The following description of the drawings begins with a presentation of a video encoder and a video decoder of a block-based predictive codec for coding pictures of video to form an example for a coding framework within which embodiments for a hybrid predictive codec may be incorporated. The video encoder and the video decoder are described with respect to FIGS. 1-3. Below, a description of an embodiment of the hybrid predictive concept of the present application is provided. These concepts are presented below with an explanation of how they may be incorporated into the video encoder and decoder of FIGS. 1 and 2, respectively, and are illustrated in FIG. 4 below. The described embodiments may also be used to form video encoders and video decoders that do not operate according to the underlying coding framework of the video encoders and decoders of FIGS.

[0015] FIG. 1 shows a video 11 consisting of a sequence of pictures 12, which are predictively encoded into a data stream 14. FIG. 2 shows an apparatus for coding a residual image in a block-wise manner. For this purpose, block-wise predictive coding is used. Furthermore, transform-based residual coding is exemplarily used. The apparatus, i.e. the encoder, is indicated using the reference number 10. FIG. 2 shows a corresponding decoder 20, i.e. Transform-based residual decoding is again used as an example to illustrate an apparatus 20 configured to predictively decode from a data stream 14 a video 11' consisting of pictures 12' in picture blocks, and the pictures 12' and the video 11' as reconstructed by the decoder 20 are respectively represented as The quantization of the prediction residual signal leads to the coding loss. The apostrophe is used to indicate departure from picture 12 originally encoded by device 10. Although Figures 1 and 2 illustratively use transform-based predictive residual coding, embodiments of the present application are not limited to this type of predictive residual coding. This also applies to other details described with respect to Figures 1 and 2, as outlined below.

[0016] The encoder 10 is configured to subject the prediction residual signal to a spatial-to-spectral transformation and to encode the prediction residual signal so obtained into the data stream 14. Similarly, the decoder 20 is configured to decode the prediction residual signal from the data stream 14 and to subject the prediction residual signal so obtained to a spectral-to-spatial transformation.

[0017] Internally, the encoder 10 may comprise a prediction residual signal former 22 which generates a prediction residual 24 so as to measure the deviation of a prediction signal 26 from an original signal, i.e., the video 11 or the current picture 12. The prediction residual signal former 22 may for example be a subtractor which subtracts the prediction signal from the original signal, i.e., the current picture 12. The encoder 10 then subjects the prediction residual signal 24 to a spatial-to-spectral transformation to obtain a spectral domain prediction residual signal 24'. The spectral domain prediction residual signal 24' is then also The prediction residual signal 24'' is then subjected to quantization by a quantizer 32 provided by the encoder 10. The prediction residual signal 24'' thus quantized is coded into the bitstream 14. For this purpose, the encoder 10 may optionally comprise an entropy coder 34 for entropy coding the prediction residual signal as transformed and quantized into the data stream 14. The prediction residual 26 is generated by a prediction stage 36 of the encoder 10 on the basis of the prediction residual signal 24'', which is decoded into and decodable from the data stream 14. For this purpose, the prediction stage 36 internally dequantizes the prediction residual signal 24'' as shown in FIG. 1 so as to obtain a spectral domain prediction residual signal 24'''' which corresponds to the signal 24' but for the quantization losses. A dequantizer 38 is then connected to the latter prediction residual signal 24''' for inverse transformation, i.e., The prediction stage 36 may then comprise an inverse transformer 40, which undergoes a vector-to-spatial transformation to obtain a prediction residual signal 24'''', which corresponds to the original prediction residual signal 24 apart from quantization losses. A combiner 42 in the prediction stage 36 then recombines the prediction signal 26 and the prediction residual signal 24'''', such as by addition, to obtain a reconstructed signal 46, i.e., a reconstruction of the original signal 12. The reconstructed signal 46 may correspond to signal 12'.

[0018] A prediction module 44 of the prediction stage 36 then performs a prediction on the signal 46, for example by using spatial prediction, i.e., intra-prediction, and / or temporal prediction, i.e., inter-prediction. Based on this, a prediction signal 26 is generated, which is described in more detail below.

[0019] Similarly, the decoder 20 may internally consist of components corresponding to the prediction stage 36 and may be interconnected in a manner corresponding to the prediction stage 36. In particular, an entropy decoder 50 of the decoder 20 may entropy decode a quantized spectral domain prediction residual signal 24″ from the data stream, after which an inverse quantizer 52, an inverse transformer 54, a synthesizer 56 and a prediction module 58, which are interconnected and cooperate in the manner described above with respect to the modules of the prediction stage 36, recover a reconstructed signal based on the prediction residual signal 24″, so that the output of the synthesizer 56 results in a reconstructed signal, i.e., the video 11′ or its current picture 12′, as shown in FIG.

[0020] Although not described in detail above, it is understood that the encoder 10 may, for example, optimize the predictive mode according to some optimization scheme, such as a method that optimizes some rate and distortion related criteria, i.e., coding cost, and / or may use some rate control. It is readily apparent that the encoder 10 and the decoder 20 and corresponding modules 44, 58 may set several coding parameters, including the coding mode, motion parameters, etc. As will be explained in more detail below, the encoder 10 and the decoder 20 and corresponding modules 44, 58 each support different prediction modes, such as intra-coding and inter-coding modes, which form a kind of set or pool of primitive prediction modes on the basis of which the predictions of picture blocks are assembled in a manner that will be explained in more detail below. The granularity with which the encoder and the decoder switch between these prediction assemblies may correspond to a subdivision of the pictures 12 and 12′, respectively, into blocks. Some of these blocks may be simply It should be noted that some blocks may be intra-coded blocks, some blocks may be merely inter-coded blocks, and optionally further blocks may be blocks obtained using both intra-coding and inter-coding, as will be described in more detail below. According to the intra-coding mode, the prediction signal for a block is derived from the already coded / decoded data of the respective block. The prediction signal for each block is obtained based on spatial neighborhood. There may be several intra-coding sub-modes, the selection between which represents, in a sense, a kind of intra-prediction parameter. There may be directional or angular intra-coding sub-modes, in which the prediction signal for each block is filled accordingly by extrapolating the neighboring sample values ​​into the respective block along several directions specific to the respective directional intra-coding sub-mode. The intra-coding sub-modes may also include, for example, a DC coding mode, in which the prediction signal for each block assigns DC values ​​to all samples in the respective block accordingly, and / or a prediction signal for each block is filled accordingly. The measured signal is determined by the slope and offset of a plane defined by a two-dimensional linear function based on adjacent samples. Then, the offset is calculated by multiplying the sample positions of each block by 2 and one or more further sub-modes, such as a planar intra-coding mode in which the spatial distribution of sample values ​​is approximated or determined to be represented by a three-dimensional linear function. In comparison, according to the inter prediction modes, a prediction signal for a block may be obtained, for example, by temporally predicting an inner block. For parameterization of the inter prediction modes, motion vectors may be signaled in the data stream, which indicate a previously coded / decoded picture of the video 11, at which previously coded / decoded pictures are sampled to obtain a prediction signal for the respective block. 2, which indicates the spatial displacement of the coded picture portion. This means that in addition to the residual signal coding provided by the data stream 14, such as the entropy coded transform coefficient levels representing the quantized spectral domain prediction residual signal 24", the data stream 14 may have encoded therein prediction related parameters for assigning to the block prediction mode, prediction parameters for the assigned prediction mode, such as motion parameters for inter prediction modes, and optionally further parameters for controlling the assembly of the final prediction signal for the block using the assigned prediction mode and prediction parameters, as outlined in more detail below. The system controls and signals the subdivision of pictures 12 and 12', respectively, into blocks. The decoder 20 uses these parameters to subdivide the picture in the same way as the encoder did, to assign the same prediction modes and parameters to blocks, and to perform the same predictions which should result in the same prediction signals.

[0021] FIG. 3 shows the relationship between the reconstructed signal, i.e. the reconstructed picture 12', on the one hand, and the synthesis of the prediction residual signal 24'''' and the prediction signal 26 as signaled in the data stream, on the other hand. As already indicated above, the synthesis may be additive. The prediction signal 26 is represented in FIG. 3 as a subdivision of the picture area into blocks 80 of varying size. 3, but this is just one example. The subdivision may be any subdivision, such as a regular subdivision of the picture area into rows and columns of blocks, or a multi-tree subdivision of the picture 12 into leaf blocks of varying sizes, such as a quad-tree subdivision, a mixture of which is shown in FIG. 3, where the picture area is first subdivided into rows and columns of tree root blocks, which are then further subdivided according to a recursive multi-tree subdivision resulting in blocks 80.

[0022] The prediction residual signal 24'''' of FIG. 3 is also illustrated as a subdivision of the picture area into blocks 84. These blocks are sometimes called transform blocks to distinguish them from coding blocks 80. In fact, FIG. 3 shows how the encoder 10 and decoder 20 Two different subdivisions of pictures 12 and 12′ into blocks, respectively, one into coding blocks 80 and another into blocks 84 may be used. Although both subdivisions may be the same, i.e., each block 80 may simultaneously form a transform block 84, and vice versa, FIG. 3 illustrates a subdivision into transform blocks 84, for example, such that any boundary between two blocks 80 covers the boundary between two blocks 84. The division forms an extension of the subdivision into blocks 80, or alternatively, each block 80 corresponds to one of the transformation blocks 84 or corresponds to a cluster of transformation blocks 84. 8 shows a case where the subdivisions are either square, rectangular or of any other shape. However, the subdivisions may also be determined or selected independently of each other, so that the transform blocks 84 may alternatively cross a block boundary between the blocks 80. As far as the subdivisions into transform blocks 84 are concerned, similar comments apply as such, since those carried forward with respect to the subdivisions into blocks 80, i.e. blocks 84, may be the result of a normal subdivision of the picture area into blocks arranged in rows and columns, the result of a recursive multi-tree subdivision of the picture area, or a combination thereof, or any other kind of segmentation. Merely as an aside, it should be noted that the blocks 80 and 84 are not limited to being square, rectangular or of any other shape. Moreover, the subdivisions of the current picture 12 into blocks 80, on which the prediction signal is formed, and into blocks 84, on which the prediction residual is coded, may not be the only subdivisions used for coding / decoding. Prediction These subdivisions from the granularity at which signal determination and residual coding are performed, however, firstly, the residual coding may alternatively be performed without subdivision, and secondly, at other granularities than these subdivisions, the encoder and decoder may set certain coding parameters, which may include some of the parameters mentioned above, such as prediction parameters, prediction signal assembly control signals, etc.

[0023] FIG. 3 shows that the combination of the prediction signal 26 and the prediction residual signal 24'''' directly results in the reconstructed signal 12'. However, it may be possible to combine the prediction signal 26 and the prediction residual signal 24'''' from another perspective, or for example, from another perspective with a separate DPB. The prediction loop is used to predict the time series of the input image obtained from the other coding layers that are coded / decoded in the prediction loop. According to alternative embodiments, such as a prediction signal 26, more than one prediction signal 26 may be provided as a prediction residual signal 24''''. Note that the resulting picture 12' may be composited with the above picture 12'.

[0024] In FIG. 3, the transform block 84 has the following meaning: The converter 54 performs their transformations on a per-transform block basis. For example, many codecs use some kind of DST or DCT for all transform blocks 84. Some codecs allow for skipping the transformation for some of the transform blocks 84, such that the prediction residual signal is directly coded in the spatial domain. However, according to the embodiments described below, the encoder 10 and the decoder 20 are configured in such a way that they support several transformations. For example, the transformations supported by the encoder 10 and the decoder 20 may comprise: DCT-II (or DCT-III), where DCT stands for discrete cosine transform DST-IV, where DST stands for discrete sine transform DCT-IV DST-VII ·Identity Transformation (IT)

[0025] Naturally, the transformer 28 will support all of the forward transform versions of these transforms, while the decoder 20 or inverse transformer 54 will support their corresponding backward or inverse versions. Inverse DCT-II (or inverse DCT-III) Reverse DST-IV ·Inverse DCT-IV · Reverse DST-VII Identity Transformation (IT)

[0026] Note that in either case, the set of supported transforms may comprise only one transform, such as one spectral-to-spatial or spatial-to-spectral transform.

[0027] As already outlined above, Figures 1-3 are presented as an example in which the hybrid prediction concept further described below may be implemented to form specific examples for a video encoder and a decoder according to the present application. To that extent, the video encoder and decoder of Figures 1 and 2 respectively represent possible implementations of the video encoder and decoder described below in this specification. As outlined in more detail below, when the subsequently described embodiments for hybrid prediction according to the present application are incorporated into the video encoder and decoder of Figures 1 and 2, the video encoder of Figure 1 and the video decoder of Figure 2 support, as at least one option, processing block 80, or even all blocks of which the current picture 12 consists, in a manner outlined in more detail below. Thus, Thus, the embodiments described below refer inter alia to a video encoder equivalent to encoder 10 of FIG. 1, which handles block 80 in a manner outlined in more detail below. The same applies with respect to the decoder of FIG. 2, which thus represents an example for a video decoder according to one embodiment in which block 80 is treated in the manner described. However, FIGS. 1 and 2 However, a video encoder according to an embodiment of the present application may be implemented differently than illustrated in FIG. Alternatively, this encoder may use concepts outlined in more detail below and may differ from the encoder of FIG. 1 in that it does not use transform prediction residual coding, instead coding direct prediction residuals in the spatial domain. Similarly, a video decoder according to an embodiment of the present application may perform decoding from data stream 14 using the hybrid predictive coding concepts outlined further below, but in that it subdivides picture 12′ into blocks differently than described with respect to FIG. 3 and / or that it processes the data stream in, for example, a spatial domain rather than a transform domain. 2 in that it derives the prediction residual from the stream 14.

[0028] In particular, with respect to the block subdivision into blocks 80, in the manner outlined with respect to FIG. It should be noted that the subdivision into transform blocks, if present, may also be performed as described with respect to FIG. In particular, the subdivision into blocks on the one hand and into other blocks such as transform blocks on the other hand may be performed independently of one another by subdividing picture 12 into these blocks separately, or dependently. A subdivision of may form an extension of another subdivision as described above, or both subdivisions may form an extension of another subdivision of, for example, an array of tree root blocks as described with respect to FIG. It may form a separate extension of a normal primary subdivision, such as a subdivision of a picture; and , such possibilities also apply for other sub-picture granularity mentioned below, such as for the definition of certain prediction parameters, prediction modes, contribution weights, etc. For different ones of these entities, different subdivisions may be used, which may be defined independently of each other, partially independently, or as extensions from each other.

[0029] That being said, the following description focuses on predicting block 80 in the encoder and decoder. The aim is to improve the rate-distortion performance of video coding by replacing the traditional rigid distinction between intra-prediction, inter-uni-prediction, and inter-bi-prediction with a more general approach that allows greater flexibility in the way the prediction signal is obtained. The idea is to compose several primitive prediction operations, so that the composition results in a better prediction signal than any of its constituent primitive prediction operations. In a simple case, the constituent primitive prediction operations can be either intra-prediction or inter-prediction (uni- or bi-), and the composition operation can be a weighted overlap. In this case, the resulting overall prediction signal q is the sum of the constituent primitive prediction signals p1,...,p N from,

[0030]

number

[0031] It is derived as follows: α n are weighting coefficients and N is the number of constituent primitive predictions. Here and below, p1,...,p N and q is a vector of sample values ​​of the corresponding signal, i.e. a two-dimensional vector of the shape of the block to be predicted. be.

[0032] In a particular embodiment, the global prediction signal is obtained by repeated application of the assembly operation. q1=p1, ν1=1 and recursion relations

[0033]

number

[0034] This section stipulates:

[0035] Assembly operator f n is the intermediate composite prediction signal q n and one or more primitive prediction signals.

[0036]

number

[0037] Let us use the new intermediate prediction signal q n+1 ν n +1 and ν n+1 The value of is the intermediate prediction signal q n+1 The overall prediction signal is calculated by the final intermediate prediction signal q=q K+1 K is obtained as Note that the number of assembly operations used is specified by K ≥ 0, K ≥ 1, or K > 1, for example, and an upper bound such as 1 or 2 may also be applied. With the total number of constituent primitive prediction signals given as N, this means that v K+1 =N.

[0038] To further illustrate this, see Figure 4. The set of primitive prediction modes supported by the present invention is illustrated at 100. The frame 100 may include an intra-prediction mode 102 and an inter-prediction mode 104. The inter-prediction modes and the bi-prediction inter-prediction modes form separate elements of the set 100. 4, or may be interpreted as a differently parameterized version of the inter prediction mode 104. The block currently to be predicted is indicated at 106. To form a composite prediction signal q, 108, for a given block 106, the decoder and encoder select a set of primitive predictions 110, i.e., p1~

[0039]

number

[0040] The encoder and decoder use the set of prediction modes 100 to derive a set 110, or more precisely, a set 112 of primitive prediction modes from the set 100, which may be equal to the set 100 or may be an individual assignment to the prediction modes in the set 100. Each primitive prediction p i In particular, for the derivation 114 of the primitive prediction set 110, each primitive prediction p i But, set 100 The prediction modes may be derived from the associated prediction modes of the set 110. All prediction models so associated with at least one of the primitive predictions of Based on the set of primitive predictions, i.e., 110, the decoder and the encoder then synthesize the set of primitive predictions, i.e., , constructing a composite prediction signal 108 for a given block 106. As shown through the last equation, this synthesis 116 may be performed stepwise or continuously with iterations. is denoted above by K. In detail, how do we make the normal forecast, i.e. the underlying forecast, The first primitive prediction p1 thus formed is then further multiplied to obtain an intermediate prediction signal q2. First, a first subset of further primitive predictions is calculated through a function f1, i.e.

[0041]

number

[0042] The latter is then combined with a further subset of further primitive predictions, namely:

[0043]

number

[0044] is subjected to another function f2 along with the function f K The result is the final composite predicted signal 108, i.e. , giving q.

[0045] As shown in Figure 4, for each primitive prediction p i and the composite prediction signal q, as well as all intermediate prediction signals q i is a vector that associates predicted sample values ​​with each sample position in block 106. As described above with respect to FIG. 1 and FIG. 2, the encoder performs the steps of the process of converting the composite prediction signal 108 to a signal 106, i.e., to modify the composite prediction signal 108 to reconstruct the block 106. In response, the prediction residual for block 106 is encoded into data stream 14 .

[0046] Solely for ease of understanding of the following description, FIG. 5 shows individual prediction modes based on their associated prediction modes. Each primitive prediction p i We present an environment that requires parameterization to be shared between the encoder and the decoder with respect to the question of how to derive . is the primitive prediction p i For each, the prediction to be selected for block 106 The selection of the prediction modes is shown at 120 and their parameterization at 122. If the prediction mode selected at 120 is, for example, an intra-prediction mode, then the parameterization selected at 122 is the intra-mode parameters. The set of intra mode parameters 124 may be, for example, For example, one may distinguish between different angular modes, and optionally one or more further modes, such as a DC mode and a planar mode as indicated above. If the mode is an inter prediction mode, one or more of the inter mode parameters The set 126 includes a motion vector and, optionally, a reference picture index and, optionally, , a predictor index. In particular, the motion vector in the parameter set 126 The block 106 performs spatial and / or temporal prediction to obtain the spatial and / or temporal The motion vector may be signaled as a motion vector differential relative to a motion vector predictor obtained from a general neighborhood, and if the parameter set 126 includes a predictor index, it may be One of several such candidate predictors is chosen as the criterion for motion vector difference. Therefore, the primitive prediction p i For each block 106, the data stream 14 This primitive prediction p i 128 prediction modes for, and parameterized accordingly Using this mode, we predict p i , i.e., to provide a prediction parameter set 130 132, which allows the decoder to derive an associated set of one or more prediction parameters for parameterizing the corresponding mode 128. The primitive predictions are then combined using a combination 116 to produce a final combined prediction signal q, 108. As described in more detail below, various mechanisms may be used to alleviate the burden associated with the signaling overhead associated with keeping the encoder and decoder synchronized, or alternatively speaking, to signal information 128 and 130 per primitive prediction to the decoder. Another parameter that controls the synthesis 116, and thus the assembly of the prediction signal 108, is subject to sub-picture level variation by the encoder, according to an embodiment of the present application described in more detail below, and may relate to: 1) The number of recursions or iterations, K. As shown in FIG. 5 at 134, K may be varied with sub-picture granularity, for example, by block, such as block 106. 2) The number of recursions or iterations, K, may be varied when using the iterative assembly approach of FIG. 6. When K is varied, this indirectly affects the cardinality of the prediction set 110, as well as the number of additional predictions p2~

[0047]

number

[0048] In the case where more than one mode for is allowed, the cardinality of the mode set 112 is also varied. However, one or both of the latter cards may also be varied when not using the iterative approach. 3) Compositing 116 can be controlled with sub-picture granularity. When using an iterative construction of the above equation: For example, the function f n may be subject to encoder variations. As will be outlined in more detail below, to perform the composition of synthesis 116 accordingly, The assembly control information 138 is submitted or signaled to the decoder via the data stream 14. While ringing, function f n is that the encoder is a function f n Select the parameterization of It may be digitalizable.

[0049] The signaling is related to the assembly 116, i.e. the number of iterations K, as shown in FIG. or controlling it, the prediction model for each primitive prediction and assembly control 138 involved The code 128 and its parameterization do not have to be explicitly signaled in the data stream to the block 106. That is, these information items are not required to be explicitly signaled to the block 106 or is sent as extra information for some sub-region of picture 12 in which block 106 is located. Rather, as outlined in more detail below, some or all of this information may be transmitted to another block related to the same information type but adjacent, such as block 106, or to a block related to residual coding, etc. Which other coding parameters issue may be signaled via implicit signaling, meaning that the decoder can infer the respective information entity from other data in the data stream 14. An embodiment is described below.

[0050] In other words, the primitive prediction signals p1,...,p N For the prediction mode, All information128, prediction mode such as intra mode, reference index, and motion vectors Prediction control information, such as parameterization related information 130, should be known to the decoder. FIG. 5 makes clear that this prediction-related information should therefore be transmitted in the side information in the data stream 14. Moreover, this prediction-related information may be transmitted or synchronized, either explicitly or implicitly. It has been outlined that the signaling may be signaled by the block 80. Explicit signaling may be described as transmitting some or all of the prediction related information, such as intra prediction mode or reference index, motion vector predictor index, motion vector predictor index or motion vector differential, especially for the block 80 or some sub-region of the picture in which the block 80 is located, while implicit signaling may be described as meaning that the prediction related information, or parts of it, can be inferred from other parts of the data stream 14, such as parts of the data stream 14 that relate to other blocks other than the currently predicted block 80, i.e., for blocks in which the block 80 is not located. See, for example, FIG. The currently predicted block 106 is indicated therein using the reference sign 106. This reference sign is used to indicate that the tasks shown in FIG. 4 are performed specifically for this block 106. However, block 106 is a block 80, as indicated by 80 in brackets after 106, and the tasks related to prediction assembly may alternatively be performed for all blocks 80, or for blocks 80 for which, for example, p1 is to be of inter prediction mode. Thus, some of the information involved in deriving further primitive predictions and their number, or number of iterations, from such blocks in the neighborhood may be provided via, for example, merge indicators or merge flags, as described with respect to FIG. The other neighboring blocks may be treated as neighboring blocks with K being 0, i.e., the number of additional primitive predictions is 0.

[0051] In an example outlined in more detail below, for example, implicit signaling may be used to adapt a merging scheme or mode, such as that used in HEVC or H.265. In certain embodiments, for example, information 128 and 130, or only 130, are both explicitly and implicitly associated with the complementary set. Implicitly, P ⊂ {p1,...,p N} are signaled in data stream 14 for a subset of

[0052] For example, as far as primitive predictions other than the first primitive prediction p1 are concerned, the prediction model The code may, for example, be set by default by the decoder and encoder.

[0053] As outlined above with reference to reference numeral 138, the composition operators f1,...,f K should also be known to the decoder. They can either be fixed, or inferred from already transmitted syntax elements, or explicitly signaled in the bitstream.

[0054] In one particular embodiment, each f1,...,f K teeth,

[0055]

number

[0056] can be obtained from the generalized construction operator h as:

[0057] Here, the number of constituent primitive prediction signals is the sum of all the assembly operators f1,...,f K , i.e., ν n+1 -ν n = m. Vector α n is a specific construction operator f n We parameterize the generic construction operator h such that we obtain α n Only α needs to be specified. n The dimension of

[0058]

number

[0059] (and q n ), and α n Note that α may be 1, making it a scalar. n The value of f is the construction operator n , so that it is also known to the decoder. It can either be fixed, or estimated, or signaled in the bitstream.

[0060] Average-preserving weighted linear superposition and one primitive predication in each assembly operation The measured signal (i.e., ν n+1 -ν n For the particular case of ∑ i = 1), the general construction operator h is h(q n ,p n+1 ,α n )=α n p n+1 +(1-α n )·q n can be defined as, where:

[0061]

number

[0062] is a weighting factor or assembly factor. Weighting factor α n should be known to the decoder. Since it is possible to use α, it can either be fixed, estimated, or signaled in the bitstream. n A (usually small) number of If only a few values ​​are possible, the index value

[0063]

number

[0064] may be sent instead, which is α n The actual value of α n The actual value of α is then derived, either by use of a lookup table, or by calculation, or by other means. n Note that the permissible values ​​of do not have to be the same for all n. n or (1-α n ) may be negative leading to the subtraction of the corresponding predicted signal.

[0065] The subsequent steps performed by the decoder and encoder to provide a composite prediction signal 108. The procedure is shown in Figure 6. K+1 primitive predictions p1...p K+1 There exists K iterations, i.e., consecutive additions 1501 to 150 K is executed. Each iteration 150 i In the corresponding contribution coefficient α i in Weighted next primitive prediction p i+1 is the intermediate sum formed so far, i.e., q i where q1 is the contribution coefficient α i minus 1-α i Therefore, an additional primitive prediction p2 is, for example, , in effect, it is not α1 but α1·(1-α2)·(1-α3)·...·(1-α K ) to the final composite prediction signal 108. In practice, this is particularly true for For previous primitive predictions, i.e., primitive predictions with smaller indices, the effective weighting can be set very finely, for example, the individual contribution coefficients α1 to α K This means that the setting of α is limited to a limited number of discrete weight values. For example, the contribution value α iSettings and data stream Regarding their signaling by Team 14, see Figure 7, which shows some possibilities. In detail, FIG. 7 shows how the encoder calculates the contribution value α i may be set to it, i.e., may be allowed to be set to it by the encoder and may be signaled implicitly or explicitly in the data stream 14. i The value area 160 in FIG. 7. It is shown that the weights can be limited to a discrete number of values, as indicated by the crosses. , the limited number of discrete weight values ​​may comprise at least one negative value and at least one positive value. Additionally or alternatively, this contribution value α i to itself or (1-α i ) can be negative. Alternatively, for example, only positive values ​​may be allowed. As already mentioned above, the contribution value α i To signal the So, α i The input to the arithmetic relationship between the information signaled on the one hand and the weight values ​​on the other hand The number and values ​​of the discrete weight values ​​in the value region 160 may be determined by the contribution value α i may be equal between or may differ for the contribution values. i or (1-α i Note that ) can be signaled in the data stream.

[0066] As above,

[0067]

number

[0068] is a two-dimensional vector, and the general construction operator h is h(q n ,pn+1 ,α n )=(α n )1·p n+1 +(α n )2·q n It can be defined as:

[0069] As above, (α n )1 and (α n The value of θ2 should be known to the decoder and can be either fixed, estimated, or signaled in the bitstream.

[0070]

number

[0071] The previously described generalized construction operator h with Here, (α n )2=1-(α n ) 1.

[0072] In a further particular embodiment, the composition operator f n Clipping and / or rounding during Whether clipping and / or rounding operations should be performed depends on the , which can either be fixed, estimated, or signaled in the bitstream. K Clipping is possible only for a subset of It is also possible to include rounding and / or rounding operations (e.g., when the overall predicted signal q=q K+1 If clipping and / or rounding should be performed only on f K contains clipping and / or rounding operations).

[0073] For example, see dashed box 170 in FIG. 6. They are the intermediate sums q2 to q K It has been shown that {circumflex over (x)} may be subject to a clipping and / or rounding operation 170. Additionally, a clipping and / or rounding operation 172 may be performed to provide the final composite prediction signal q. Final sum q K+1 Any rounding may be applied when intermediate sums are calculated and It will be apparent that this forms a quantization that is significantly coarser than the computational precision with which it is represented. The clipping and / or rounding operations 172, for example, ensure that the sample values ​​of the composite prediction signal q, 108 are within an allowed representation range or value domain of the sample values ​​with which the picture 12 is coded.

[0074] Furthermore, the construction operator f n is the (new intermediate) predicted signal q at a particular sample position. n+1 The sample values ​​obtained from

[0075]

number

[0076] and the intermediate prediction signal q n It can be scalar in the sense that it depends only on the value of

[0077] Again, see FIG. 6 for an illustration. For each primitive prediction p i are composite forecasts, respectively. For each sample position 180 of the measured signal 108 or for each sample position 180 of the block 106 / 80, is a two-dimensional vector with components or sample values, where each sample location 180 of the predicted signal 108 is simply a primitive prediction p i Based on the corresponding sample positions placed together in The definition is done in such a way that it is possible to filter some of the intermediate sums. One alternative could be that some of the intermediate sums are subjected to some kind of filtering, such as FIR filtering.

[0078] Intermediate prediction signals q1,...,q K+1 The domain (e.g., dynamic range, bit depth, precision) in which (or a subset of them) are represented is determined by the primitive prediction signals p1,...,p N The area may be different from that of the

[0079] Multiple color planes (e.g. R, G, B, luma, chroma, depth, alpha channels, etc.) In case of joint coding, the composition operator can either be shared between (sub)sets of planes or can be independent. Which planes use the same composition operator can either be fixed, estimated or signaled in the bitstream.

[0080] Assembly operator f n Alternatively, h may be defined for the entire video sequence or At a given granularity (e.g., random access period level, picture level, slice level, The granularity can be fixed or can vary at a fixed The number of operators K, together with the composition operators themselves, can be either of the same or different grains. The upper bound K that limits the maximum number of composition operators can vary within a certain degree. max It is possible. max The value of is either fixed or estimated or signaled in the bitstream.

[0081] Assembly operator f n or h (for example, the parameter vector α n By signaling Either explicitly or (similar to the merge mode in HEVC / H.265, for example) implicitly it can be signaled. In the latter case, a reference to an already encoded set of assembly operators is signaled, and those assembly operators are used (optionally, after adaptation to, for example block size, color channel, bit depth, etc.). A mixture of implicit signaling and explicit signaling is also possible, for example, the first k implicit <K assembly operators

[0082]

Number

[0083] are implicit, i.e., by reference to already signaled assembly operators, and the remaining K - k implicit <assembly operators

[0084]

Number

[0085] are signaled explicitly, i.e., by directly signaling the information necessary for the decoder to perform the assembly operation. Which assembly operators are signaled explicitly and which are signaled implicitly can be either fixed, inferred, or signaled within the bitstream.

[0086] Before proceeding with the description of obtaining an implementation of the embodiments of the present application by modifying the HEVC codec, assume that the latter aspect after combining the concept of merging with the concept of composite prediction is as shown with respect to FIG. 8. FIG. 8 shows block 106, which is the block currently being processed, i.e., currently decoded or currently encoded, i.e., block 80. Within that neighborhood are blocks 190a and 190b. They are blocks in decoding / coding order. 8. The two adjacent blocks 190a and 190b shown in FIG. 8 are chosen for illustrative purposes only. Also note that only one adjacent block may be used, or more than two may be used. Furthermore, it is assumed that both adjacent blocks 190a and 190b are equal in size to block 106. The fact that blocks 190a and 190b are shown as blocks 80 is also for illustrative purposes. In effect, blocks 190a and 190b are also blocks 80, i.e., for these blocks, the prediction signal 108 has been determined in the same manner as outlined above. The decoder and encoder use all previously processed Blocks 190a and 190b of all blocks, i.e., preceding blocks in coding order, may be identified based on, for example, one or more predetermined sample positions of block 106. For example, block 190a may be determined to be the block comprising the sample to the left of top-left sample 192 of block 106, and block 190b may be determined to be block 80 comprising the sample above top-left sample 192. However, However, other examples are possible. Block candidates may also be, for example, Blocks 106 may include blocks with co-located sample locations. The merge candidate may comprise a block of another picture, such as a block of a merging candidate. The selection result is that when merging is used for block 106, data stream 14 The ACK can be signaled in

[0087] Since blocks 190a and 190b are predicted blocks 80, i.e. blocks for which a prediction signal 100a has been determined, there is prediction-related information 194 for each of these blocks, as exemplarily shown in FIG. 8 for block 190a. More precisely, the prediction-related information 194 has, for block 190a, led to the composite prediction signal 108 for block 190a. The prediction-related information 194 may, for example, be related to a prediction mode underlying the derivation of primitive prediction p1. Additionally, information 194 may include information about the additional FIG. 8 shows that the prediction signal construction follows the concept of FIG. 6. For illustrative purposes, assume that the prediction-related information 194 indicates a number K of additional primitive predictions, each of which is equal to the number of iterations 150 applied. For valid possibilities, K>0, the prediction-related information 134 additionally indicates additional primitive predictions p2...p K+1 Additionally, the primitive prediction p i For each, the corresponding contribution weight α i-1 The prediction related information 194 for the neighboring block 190a is explicitly conveyed in the data stream 14. It will be apparent that the prediction related information 194 need not necessarily be signaled in the data stream 14, but may be signaled, at least in part, implicitly in the data stream 14. The reader and decoder, at the time of processing block 106, Have access to or knowledge of information 194. Signaling overhead To save time, the encoder has the opportunity to choose a merge mode for block 106. , thereby obtaining at least some pieces of corresponding prediction-related information for block 106. , the prediction related information 194 of block 190a, or the corresponding prediction related information of block 190b, etc. That is, the encoder may signal the activation of the merge mode for the block 106 in the data stream 14 via the merge information 196, which may include the merge mode to optionally indicate the merge candidates that should be used.

[0088] In some cases, the merge information 196 additionally comprises information about which fragments of the prediction related information 194 of the merge candidate should be used for inferring the corresponding portion of the prediction related information 198 for the current block 106. According to one option, for example, only the information about how to derive the first primitive prediction p1 is provided by the brace 200. Therefore, the corresponding information 200' in the prediction related information 198 is p2 will be set equal to the information 200. For primitive prediction, prediction related information or parameters are signaled in the data stream for that block 106 via information pointing into a list of prediction parameters used for neighboring blocks and related to the prediction mode of that particular primitive prediction. Note that the neighboring blocks that contribute to the merge candidate list and those that contribute to the later lists, and therefore the blocks whose prediction-related information is pointed to in those lists by the merge information 196 and signaling 206, may be different. For example, prediction p1 may be an inter-predicted signal and p2 is an intra-predicted signal.

[0089] One alternative has just been outlined, which is the two hypotheses of the bi-predictive mode for block 190a. Which of them does not form a criterion for the uni-prediction mode of the primitive prediction p1 of block 106? Additional signaling may be provided to direct the bi-predictive mode for p1 of block 190a to the uni-predictive mode for p1 of block 106, while additionally selecting whether the bi-predictive mode for p1 of block 190a should be changed to the uni-predictive mode for p1 of block 106. One alternative is for such a video to be bi-predictively coded. Alternatively, the merge information 196 may include additional signaling to restrict the determination of the merge candidates to those using a bi-predictive mode for p1, while additionally signaling as to which of the two hypotheses of a suitable merge block should form the criterion for the primitive prediction p1 of block 106. In both alternatives, the mode of p1 of block 106 is set to be a uni-predictive mode. In the latter alternative, which will be described in more detail again herein below, the mode of p1 of block 106 is then set to be a uni-predictive mode. In the latter alternative, which will be described in more detail again herein below, the mode of p1 of block 106 is then set to be a uni-predictive mode. In the latter alternative, which will be described in more detail again herein below, the mode of p1 of block 106 is then set to be a uni-predictive mode. In the latter alternative, which will be described in more detail later ... later, the mode of p1 of block 106 is then set to be a uni-predictive mode. In the latter alternative, which will be described in more detail later, the mode of p1 of block 106 is then set to be a uni-predictive mode. In the latter alternative, which will be described in more detail later, the mode of p1 of block 106 is then set to be a uni-predictive mode. The merge information 196 controls the formation of a set of merge candidates, including bi-predicted inter blocks, possibly signaling information about which blocks will be selected as merge partners. In the former alternative, this restriction is suspended and the signaled merge candidates may be uni-predictive or bi-predictive with respect to p1, and in the bi-predictive case, the signaled uni-predictive mode derivation of p1 for block 106 is parameterized. Only hypotheses are used.

[0090] Another option is, for example, to add - in addition to the portion 200 - a number K of additional primitive predictions, as well as corresponding information on how the corresponding primitive predictions should be derived and how the corresponding contribution values ​​should be set, as indicated by the braces 202. In this case, the prediction-related information in block 106 is The corresponding portion 202' of the information 198 corresponds to that portion 202 of the block 190a, i.e., the additional primitive prediction.

[0091]

number

[0092] For the modes, associated prediction parameters, and contribution values ​​for implicit Information on the time In other words, according to option 202, the predicted derived information, i.e. That is, all K additional primitive predictions p2 to p K+1 For The prediction parameters and the contribution weights are then fed to the composite prediction signal 106. The corresponding primitive prediction derivation information and contribution weight information for the same number of primitive predictions for constructing the primitive prediction derivation information and contribution weight information are used to form the corresponding primitive prediction derivation information and contribution weight information for the same number of primitive predictions for constructing the ...

[0093]

number

[0094] However, as shown in FIG. 8, the encoder additionally signals or triggers an implicit guess of the prediction parameters and contribution values ​​for the neighboring blocks 190a k so that it is equal to K implicit Regarding setting, The encoder may decide to extend the number of additional primitive predictions. The encoder may decide to use offsets or or different Kk implicit may be signaled in the data stream 14. Thus, The prediction related information 198 for the block 106 is then

[0095]

number

[0096] Explicitly tell block 106 in data stream 14 how to derive The K in the information content 198 indicates an additional primitive prediction for the block 106. It will be clear that while the number of measurements is related to the K in information 194, block 190a, both parameters can be set differently. Both of them can be set to default values ​​or signaled in data stream 14, as indicated above, with some K max may be limited by

[0097] Alternatively to option 202, it may be possible for the encoder to have the additional freedom to signal that not all additional primitive predictions K of the neighboring block 190a should be used to set up the prediction related information 198 for the current block 106. In other words, the data stream 14 may include k implicit , i.e., how to modify the merge candidate. The latter option is illustrated in FIG. 8 using braces 204. Which of the options 200-204 is used may depend on the implementation. For example, one of the options 200-204 may be fixed by the encoder and decoder. Alternatively, some information may be used in two of options 200 to 204. Or you can switch between all of them. implicit , and notifying the decoder about them via signaling in the data stream 14. Instead, the encoder is provided with an opportunity to modify K in the information 194 associated with block 190a and k for block 106 to giveimplicit The relationship between may be fixed by default or may be determined by implicit signaling.

[0098] With reference to FIG. 8, further primitive predictions p2 to p3 of block 106 in the prediction related information 198 K+1 Note that for any of p1, p2, p3, p4, p5, p6, p7, p8, p9, p10, p11, p12, p13, p14, p25, p26, p27, p38, p49, p50, p61, p70, p82, p93, p15, p16, p17, p28, p29, p30, p40, p51, p62, p70, p83, p94, p18, p19, p20, p21, p22, p23, p34, p45, p51, p63, p70, p84, p95, p19, p12, p13, p24, p14, p25, p26, p38, p49, p50, p61, p70, p85, p96, p15, p16 For example, p1 of block 106 may be of some mode such as inter mode. For example, for block 106 where For example, merge options / behaviors are simply activated by the respective merge flags. It may be that p1 in block 106 is of several modes, or the merge activation itself will simultaneously reveal that p1 in block 106 is of several modes, since the merge candidate set / list has been configured accordingly by simply putting in the candidates for which p1 should be of the respective prediction modes.

[0099] We now turn to present a possible implementation of the embodiments of the present application, which is achieved by modifying the HEVC / H.264 codec. In HEVC / H.265, each picture is divided into several Coding Tree Units (CTUs), each of which may be further subdivided into Coding Units (CUs). The CUs may again be further divided into Prediction Units (PUs) and Transform Units (TUs). The above mentioned hybrid prediction is carried out by the PU level. The normal prediction parameters of HEVC / H.265 (i.e., intra prediction) can be signaled by the bell. In addition to the prediction mode or motion vector and reference index), further prediction parameters are The prediction parameters (again, either intra or inter) may be signaled together with assembly information, which indicates how the individual prediction signals obtained from the individual prediction parameters are assembled into a resulting overall prediction signal. That is, the previously described block 106 may be a PU block according to the HEVC nomenclature. Availability of additional prediction parameters The property may be indicated by one additional syntax element, If it indicates that there are no additional prediction parameters, no further data needs to be transmitted. Otherwise, a syntax element corresponding to the additional prediction signal follows, together with data specifying how the assembly operation of the normal HEVC / H.265 prediction signal and the additional prediction signal should be performed. In the simple case, a weighting or contribution factor for the additional prediction signal is transmitted. This factor can be signaled either directly or as an index into a look-up table from which the actual weighting factor is obtained. Two or more If more predictive signals are used, the signalling starts again from the beginning, i.e. there is one syntax element in the signalling to indicate whether more predictive signals are to follow. The signaling then continues as previously described.

[0100] In the latter discussion, K or alternatively Kk for block 106 implicit Signaling One way of doing this is disclosed in detail and in the syntax example provided below. As illustrated in the example of FIG. 1, whether an explicitly signaled additional primitive prediction follows in the data stream 14 for the current block, and therefore whether the prediction parameters and their contribution weights follow for this further additional primitive prediction, can be indicated in the data stream 14 successively for the block 106 by the additional primitive prediction, i.e., via a corresponding flag. These flags may be transmitted in the data stream 14 interleaved with corresponding explicit information about the primitive prediction derivation information and the corresponding contribution weights, as illustrated below. implicit The primitive predictions are explicitly Kk is determined by the implicit k primitive predictions. implicit The parameters controlling the primitive predictions are derived from the merging candidates. implicit pieces The number of explicitly specified additional primitive predictions is calculated based on the data sequence for block 106. The predictions are signaled in the stream 14. This may be done by sending a flag of some state followed by one bit of other state, one for each additional primitive prediction that is explicitly specified (optionally up to a maximum number K max Explicitly specified information about primitive prediction, i.e., 206 in FIG. 8, is provided by block 106. , in a data stream 14.

[0101] Note that FIG. 8 shows that for each primitive prediction that participates in the construction of the prediction signal for block 106, the mode is indicated by information 198. This does not mean that for each of these primitive predictions, this mode indication will need to be conveyed in data stream 14. Rather, for some of these primitive predictions, it may be known by default at least as to which mode the respective primitive prediction is of. For example, some of the embodiments outlined in more detail below may signal the further primitive predictions p2,...,p K+1 One of the It is estimated that one of them is of inter prediction mode.

[0102] We briefly compare the description of FIG. 8 with the merge mode of HEVC and how HEVC may be modified with respect to the merge mode to form one example implementation for the embodiment described with respect to FIG. In HEVC / H.265, the merge mode allows for the use of inter prediction parameters from already transmitted adjacent or temporally co-located blocks. This reduces the amount of data required. (Whether list 0 is used, list 1 is used, or bi-prediction is used) (indicates whether to use inter_pred_idc one or more motion vector predictor indices (in the bi-predictive case) One or more reference picture indices (in the case of bi-prediction) Motion vector difference Instead of signaling all of the above, only a merge index is signaled, which indicates the prediction unit (PU) whose prediction parameters should be reused for the current PU.

[0103] As described with respect to FIG. 8, it is also possible to use a merge mode for one or more of the primitive prediction signals. In other words, for example, the first primitive If a merge mode is used for the prediction signal p1, one or more additional prediction signals, That is, it is possible to transmit explicitly defined prediction signals and assemble them into one overall prediction signal as explained above. Ringing allows for restricting merge modes such that only a portion of the available prediction data is used for p1 (e.g., list 0 or list 1 prediction instead of bi-prediction), or for the available prediction data to be modified (e.g., a shift to the resulting motion vector grid). quantization to full-pel or half-pel motion vector accuracy with or without quantization The manner in which the merge mode is restricted can be indicated by further syntax elements (e.g., for the bi-predictive to uni-predictive case, with one flag indicating whether list 0 prediction or list 1 prediction should be used).

[0104] If the merge candidate used (as indicated by the merge index) uses composite prediction, all or a subset of the constituent primitive prediction signals may be used for the current primitive prediction signal, i.e., the implicitly defined primitive prediction. N+1 Which subset of is used can either be fixed, estimated, or explicitly signaled. For example, in the above case of a merge mode restricted from bi-prediction to uni-prediction, only one of the two motion parameters that specify the bi-prediction signal is discarded. In another example, if no such restriction is imposed, all primitive prediction parameters of the used merging candidates may be used for the current block.

[0105] In HEVC / H.265, the merge candidate list is constructed in such a way that redundant entries are avoided. In the context of mixed prediction, this means that It is implied that not only the motion parameters, but also the motion parameters of all other primitive prediction signals can be checked for equality.

[0106] One example for the order of predictors as specified in the bitstream is See FIG. 9, which shows a fragment of a PU syntax for specifying the first prediction hypothesis p1. The first prediction hypothesis p1 may be a "normal" (i.e., intra, uni-predictive inter, or bi-predictive inter) prediction signal. For the special case of merge mode in HEVC / H.265 (or something similar), i.e., a prediction mode in which a reference is made to another coded block and prediction parameters from it are also used for the current block, the ... and if yes, which of the two [List 0 or List 1] prediction signals will be used Note that it is possible to restrict the use of bi-prediction to one of the two constituent prediction signals by up to two syntax elements (indicating whether the first "normal" prediction hypothesis should be used). After this first "normal" prediction hypothesis, it follows a series of syntax elements.

[0107] The variable NumMergedAdditionalHypotheseis208 is used to determine the number of blocks that have additional hypotheses. gives the number of additional hypotheses that are "inherited" from via the merge mode. The variable MaxNumAdditionalHypotheses 209 constrains the total number of additional hypotheses. Its value can be fixed, or given by some profile / level constraints, or can be specified by the bitstream. The packet may be sent either within a packet transmission system or within a packet transmission window.

[0108] In particular, according to the example of FIG. 9, the number of explicitly defined primitive predictions is signaled via a sequence of flags 210, namely additional_hypotheseis_flag. The number of flags 210 with some state, i.e. 1, specifies the number of explicitly specified primitive predictions, followed by flags 210 with other states, i.e. 0. Each flag 210 with a 1 is followed by information on how the respective additional primitive prediction is interpreted. In this example, it is presumed that each of these additional primitive predictions is of inter prediction mode. Thus, for each explicitly specified additional primitive prediction, the following syntax elements are transmitted: ref_idx_add_hyp 212 indicates the reference index i of the reference picture of each explicitly specified additional primitive prediction, i.e. the reference index i for which the ith flag 210 should be 1; The coding part mvp_coding 214 calculates the motion vector differential, i.e. the latter motion vector predictor mvp_add_hyp_flag 216 provides a difference to the motion vector predictor that when added to gives the motion vector for setting up / deriving the ith primitive prediction, and mvp_add_hyp_flag 216 adds more states Instead of the flag when a syntax element having the following formula can be used: A flag to select one of the sensors, or a flag to select one of the sensors in the encoder and decoder. The syntax element add_hyp_weight_idx 218 indicates the contribution weight that the i-th primitive prediction contributes to the composite prediction signal, and may be omitted if only one predictor is used. n or (1-α n ) may be indicated by 218. The concept of FIG. 6 may be used according to FIG. 9. As can be seen in FIG. 9, only syntax elements 212-218 follow the i-th flag 210, which is 1, and the corresponding information 212-218 are interleaved if the latter flag is 1. Furthermore, In the case where the maximum number of additional primitive predictions allowed as specified by 209 has already been reached, Flag 210 is not sent if it is already known that no further primitive predictions can follow due to the fact that the threshold has been reached. As already explained above , the encoder may signal the value of the variable 209 in the data stream, for example, for the entire video, for a sequence of pictures, or on a picture-by-picture basis. As already explained above, the variable 208 is already an implicitly defined primitive prediction. According to one embodiment, this variable is necessarily set to 0, i.e. That is, all additional primitive predictions are explicitly defined primitive predictions, and according to another embodiment, this variable 200 is the number k of FIG. implicit This section stipulates:

[0109] In the syntax table given above, the value of add_hyp_weight_idx[x0][y0][i] corresponds to the weight for the i-th additional hypothesis at spatial location (x0,y0) (given in luma samples). The weighting factor is specified (by indexing into a lookup table) by the criterion 14. Hence, the spatial granularity is the prediction block level (CU or PU in HEVC / H.265).

[0110] Overcoming the non-iterative approach of synthesizing several primitive predictions, the iterative synthesis shown in Fig. 6 Note the advantage of using a single buffer to accumulate the individual prediction hypotheses. However, since another buffer stores the current prediction signal, the number of prediction sample buffer arrays required does not increase compared to bi-prediction. Besides that, it allows a medium complexity coding algorithm, where individual hypotheses are determined one after the other in the sense of a "greedy algorithm" (i.e., local optimization), possibly followed by a refinement stage, where the prediction parameters (i.e., motion vectors) of all hypotheses are determined. The functions (rules) vary within a local neighborhood of their previous values ​​until a maximum number of iterations is reached, or can be achieved across all hypotheses until either no further improvement is achieved Repeat as many times as possible.

[0111] Additionally, in forming the composite prediction signal, rounding and / or clipping operations 170 and 172 may be used. A few remarks shall be made regarding the possibility of using any nonlinear operations: Regardless of the question of what depth accuracy (e.g., 14 bits) is used, from a practical point of view, there must be at least some non-linear rounding operations after a new predictor / hypothesis is accumulated ("added"), since otherwise the bit depth required to store the new accumulated prediction signal would increase by 1 bit for each additional predictor. (If the accumulation bit depth is 10 and the samples accumulated so far at a given location are 10 bits, then the total bit depth would be 10 bits.) Suppose the sample value for the current additional hypothesis is 1023, and the corresponding sample value for the current additional hypothesis is 1022. The resulting value is 1022.5 if both predictors are weighted by 0.5. This means that the image cannot be stored in 10 bits - so to keep the bit depth constant Either there should be some rounding to keep the bit depth constant, or the bit depth should increase with each new predictor. It is unavoidable, and as a result the assembly should be done iteratively and not expanded into one big weighted sum (or something similar).

[0112] Furthermore, the weight α in Fig. 6 i Note that,is not restricted to be in the range [0..1]. In detail, the weights for the current (accumulated) predictor are {3 / 4,9 / 8,17 / 16} and, correspondingly, for the additional hypotheses are {1 / 4,-1 / 8,-1 / 16}, i.e., These can be used as (1-α) and α, respectively. Operations with Clipping in Addition to Rounding 170 , 172, the predicted sample values ​​obtained are calculated by the intermediate sum q i and the final It is prevented from being outside the range for the composite predictor q (eg, <0 or >1023 for 10 bits).

[0113] The syntax table in Figure 9 shows that the value of NumMergedAdditionalHypotheseis is already set during parsing. The list of merge candidates, and therefore the determination of which merge candidate will be used, can be avoided during the parsing process and is left to the actual decoding. i.e., the computation of the reconstructed sample values) can be deferred until This cannot happen because the task can be a Explicit information for defining the parameters, as well as the explicitly defined process of block 106. The parsing of the number of primitive predictions depends on the prediction-related information of the last selected merge candidate, i.e., in particular the latter number K of additional primitive predictions. However, the transmission loss If the merge candidates cannot be determined reliably at the decoder side due to the above, the decoder may determine the number and prediction parameters related to the explicitly defined primitive predictions 206 of the block. 10, these information items, i.e., the signaling information via syntax elements 212-216 together with their associated contribution weights 218, are not parsed correctly, thereby exacerbating the transmission loss problem. The number of prediction parameters to be nulled and the number of prediction parameters to be nulled are signaled via flag 210. The number of explicitly defined primitive predictions of block 106 that are to be performed is calculated by dividing the number of the primitive predictions by the number of the data streams. Before parsing from the group, implicit This dependency is broken by setting 0. In other words, the coding and parsing of the latter information items must be consistent with any merge candidate configuration, and in particular with any k implicit ,oh and especially from the last selected merging candidate among the merging candidates. However, in the corresponding decoding process, the following two aspects must be observed: Additional hypotheses p2...p K+1 The valid list of,is the additional hypotheses to be signaled, i.e. , the hypotheses transmitted according to FIG. 10 using flags 210 and syntax elements 212-218 independent of the K neighboring blocks, i.e.

[0114]

number

[0115] The merged k implicit additional hypotheses, namely,

[0116]

number

[0117] This is obtained by adding ·Constraint K on the maximum size of a valid list max That is, given by 209 If too many additional hypotheses are signaled, such that the valid list is too large (k implicit , plus the number of explicitly signaled predictions, such as those signaled via 210-218, is K max ), the bitstream is invalid.

[0118] There may also be restrictions on the merge candidate list. In the syntax table of FIG. 11, the changes compared to HEVC / H.265 are highlighted. Merging as activated by syntax element 226 In the case of mode, the merge flag for the B slice, i.e., the modified merge candidate An additional syntax element restricted_merge_flag 230 is sent indicating that the list 0 predictions should be used. If this flag 230 is true (i.e. equal to 1), a further syntax element restricted_merge_list 232 is sent indicating how the merge candidates should be modified. If restricted_merge_list==0, then only list 0 predictions are taken for the merge candidates to be used. Similarly, if restricted_merge_list==1, then only list 0 predictions are taken for the merge candidates to be used. In either case, if restricted_merge_flag==1, all potentially available additional hypotheses for the merge candidates used are is discarded, i.e., k implicit is necessarily set to 0. Alternatively, syntax element 230 may be used to define a merge candidate list in that only bi-predicted merge candidates are allowed. This possibility is outlined above with respect to FIG. do.

[0119] Further examples are provided in FIG. 12. FIG. 12 shows an example CU syntax and highlights the changes compared to HEVC to demonstrate that the embodiment provided with respect to FIGS. 9-11 can be used in conjunction with inter- In the example of FIG. 12, the prediction of block 106 is shown to be not limited to use. The concept of using a combination of predictors also applies to the intra prediction block 106 / 80. Figure 12 shows the CU syntax. The number of additional primitive predictions that are explicitly specified is again signaled by flag 210. However, the number of additional primitive predictions that are explicitly specified is not always the same. For primitive prediction, syntax element 220 indicates the mode, i.e., explicit The additional primitive prediction of the hyp-th primitive is interpreted according to the intra prediction mode. The HYP_PRIMARY_MODE indicates whether the primitive prediction is interpreted by the inter prediction mode or by the inter prediction mode. The intra prediction related syntax element 222 that specifies the primitive prediction of All the syntax elements 210, 212, 214, and 216 that define the hyp additional primitive prediction in detail follow. In both cases, the contribution weights, i.e., 218 or 228, respectively, are also transmitted in the data stream. However, according to the example of FIG. The concept of composite prediction signal is not only used for intra prediction block 106, but also for inter prediction block as shown in FIG. 13, which is a CU syntax of FIG. Here we show the prediction unit syntax called by the For each prediction-based prediction p1, one further intra- or inter-prediction-related Primitive prediction p2~p K+1 modes may be signaled, i.e., the same syntax as shown in Figure 12 for intra predicted blocks applies for inter predicted PUs, and therefore the same reference numbers are used in Figure 13.

[0120] In other words or using different terminology, the above embodiment is directed to, among other things, decoding / encoding video from / to a data stream using block-based predictive decoding / encoding. Having thus disclosed a video decoder and video encoder for predicting a given block 106, prediction for the given block 106 involves the following: first prediction information being conveyed in the data stream 14. This may use the merge mode by activating the merge mode. That is, the first prediction information may comprise a merge flag 226. If the flag does not activate the merge mode, the first prediction information may include a prediction mode and The first prediction information may be used to determine / derive a first prediction signal p1, such as a part of the derivation 114 in FIG. 4. In addition, a number K is derived from the data stream 14. In this embodiment, this means that only the block 80 to which the inter prediction mode should be applied to p1 may be subject to hybrid prediction, but it should be noted that only the block 80 to which the intra prediction mode should be applied to p1, or both blocks, i.e., the block to which the inter prediction mode should be applied to p1 and the block to which the intra prediction mode should be applied to p1, may be subject to hybrid prediction. Based on the first prediction information, a first prediction signal p1, such as a part of the derivation 114 in FIG. 4, is determined / derived. Furthermore, a number K is derived from the data stream 14. In this embodiment, this means that K max This is done via a flag 210 that is sent K+1 times or K times in succession depending on whether the Instead, a different coding may be used. In particular, the interleaving of the flag 210 in the data stream with the subsequently described information may be resolved differently. Furthermore, K may be predictively coded in the data stream 14. For example, The above k implicit is simply Kk implicit is transmitted and can be seen as a predictor for K. K further prediction signals p2...p K+1 is determined and for each of the K further prediction signals The assembly weights are determined to keep the decoder and encoder synchronized, i.e., p2...p K+1 Explicit and / or implicit signaling may be used to transmit a set of one or more prediction parameters for the prediction of the prediction of the contribution weights. For example, p2...p K+1 , a set of one or more prediction parameters may be explicitly transmitted for all predicted signals p1...p K+1 Against 5. In FIGS. 9-13, this set included 212-216 or 222 depending on the mode. A mode indication 220 may also be included or signaled. However, all p2...p K+1 By default, the The contribution weights α1...α K may also be transmitted explicitly and / or implicitly. For example, all of them may be transmitted explicitly via syntax elements 218 / 228. Indexing may be used as explained above with respect to FIG. 7. The given block 106 may include the first predicted signal and K further predicted signals. For prediction, as taught in relation to FIG. 6, each of the K further prediction signals is weighted with the assembly weight for the respective further prediction signal, and the intermediate sums of successive additions to which the respective further prediction signals are added are multiplied by 1 to the assembly weight for the respective further prediction signal. The contribution weights for each of the K further prediction signals may be successively added to the first prediction signal while being weighted by minus A data stream is split in such a way that it takes on one value out of a range of equal values. The value range may be equal for the K further prediction signals. At least one value, α1...α K 172 and / or another non-linear operation may be outside [0;1] for one of It may be applied to a subset of the intermediate sums.

[0121] Please also note that the above example assumes that for the first prediction p1, it is a kind of constraint. The possibility of a merge being restricted by merge flag 226 has been clarified. For which block 106 the merge mode is activated, a merge candidate restriction signal is generated. Ring 230 is signaled in the data stream. If the prediction parameter 230 indicates a merge candidate restriction to bi-prediction prediction parameter merge candidates, From the set of data merging candidates, a uni-predictive prediction parameter merging candidate, i.e., information 194 is In contrast, the prediction parameters of blocks 190a and 190b indicate that bi-prediction is not to be used for p1. While excluding merge candidates, the merge candidate restriction signaling 230 also adjusts the prediction parameters of the bi-prediction. meter merge candidates, i.e., blocks for which the corresponding information 194 suggests bi-prediction In addition to blocks 190a,b, corresponding information 194 indicates uni-prediction for blocks 190a,b. If the prediction parameter merging candidate of the uni-prediction is not included in the set of prediction parameter merging candidates, the prediction parameter merging candidate of the uni-prediction is included in the set of prediction parameter merging candidates. A determination of a set of prediction parameter merging candidates for block 106 is made. Note that the set of prediction parameter merging candidates may be effectively an ordered set, i.e., a list. The ordering may involve ordering each prediction parameter with some estimate or setting for block 106. The prediction parameter merge candidates, or merge candidates as described herein and in the previous description, may be obtained from only one neighboring block or from two or more such neighbors, such as by some averaging or some other combination, in prediction-related settings such as 200, 202, and 204. Note that the neighboring blocks may also be located in other pictures than block 106, as outlined above. The set of merge candidates can be additionally determined by adding a fixed number or cardinality of predicted parameters for merge candidates in the set / list when some neighboring blocks are missing. Complemented by one or more default forecast parameter settings to achieve For a given block, one of the set of prediction parameter merging candidates is selected. An index such as merge_idx in FIG. 12 may be used for this purpose. It indexes one of the set of prediction parameter merging candidates. The merge candidate restriction signaling 230 is used to restrict the number of merge candidates to bi-predictive prediction parameter merge candidates. If a merge candidate restriction is indicated, the data stream includes a hypothesis selection indication 232. If the ring 230 indicates a merge candidate restriction to bi-predictive prediction parameter merge candidates, then the selected Single prediction parameter Single prediction parameterized according to one of the two hypotheses of the merge candidate A prediction of a prediction, where one hypothesis is selected according to a hypothesis selection display 232; and The merge candidate restriction signaling 230 is a merge candidate restriction signaling for bi-prediction prediction parameter merge candidates. If no restriction is indicated, the prediction by the selected prediction parameter merging candidate, i.e., if the selected prediction parameter merging candidate is bi-predictive, the bi-predictive prediction parameterized according to the two hypotheses of the selected prediction parameter merging candidate, and the prediction by the selected prediction parameter merging candidate. If the meter merge candidate is uni-predictive, determining prediction information for a given block is obtained by using a uni-predictive prediction parameterized according to the selected prediction parameter merge candidate. As explained above, in uni-prediction, the prediction signal may be a shifted and interpolated region of a reference picture, i.e., a picture used for reference. The reference picture used is specified by a reference index, and the location of the possibly interpolated region in the reference picture is specified relative to the current block by a motion vector. The reference index and the motion vector are taken from the merge candidate, or, speaking differently, from the selected hypothesis of the uni-predictive candidate or the bi-predictive candidate, and are used to parameterize the uni-prediction of the block 106. Bi-predictive In the prediction, the two motion-compensated prediction signals are linearly overlapped, such as by using a factor of 0.5 for both constituent prediction signals, or some other weighting ratio. Thus, in the bi-predictive case, two reference indices and motion vectors are The merging candidates are either adopted from the merge candidates or used to parameterize the bi-prediction, where the combination of both hypotheses is equal, as is true for all references to bi-prediction in this specification. This can be done in a fixed manner by summing both hypotheses with the same weight or with some weight ratio that is signaled in the data stream on a picture-by-picture basis. Thus, the merge candidate restriction signaling 230 can be used to limit the number of merge candidates to bi-predictive prediction parameter merge candidates. Depending on whether or not the constraint is present, the derivation of p1 may be done differently in this embodiment than in the construction of the front end, i.e., merge candidate list. However, according to an alternative embodiment, the video decoder and video encoder may derive p2...p K+1 Addition of further hypotheses in the form does not support multiplication, but simply multiplication for inter prediction blocks 106 in the manner just outlined. 12 and 13. In the example, there is only p1, and similarly p2...p K+1 It is not related to the addition of HEVC syntax. In addition to the , there are only syntax elements 230 and 32. All the details presented above, insofar as they relate to the just-highlighted problem of constrained merge candidate list construction, focus on the merge with respect to p1 independent of any other prediction signal, such as all the details presented above with respect to FIGS. 1-3, i.e., how to implement the encoder and decoder internally, as well as how to subdivide the picture into blocks 80, including the currently processed block, i.e., 106, as well as the recent It will form a reservoir for further details on the highlighted embodiment.

[0122] Additionally, note that the above example also illustrates how the prediction settings of the merge candidates are reused for the current block 106, rather than restricting the merge candidate list formation to bi-predictive candidates. In terms of the scope of use, i.e., in the case where the selected merging candidate is a bi-predictive candidate, i.e., a candidate to which a bi-predictive mode should be applied, in terms of the number of hypotheses to be adopted, the first prediction p1 is likely to be subject to some kind of controlled limited merging. For blocks 106 where merge mode is activated, such as by using merge_flag, a set of prediction parameter merge candidates for a given block is generated based on the decoded The determination is made in the decoder and the encoder. The determination is made as described in FIG. 8 or in the previous paragraph, etc. In the manner already described above, one of the set of prediction parameter merging candidates for a given block is selected, such as using signaling of an index to the one selected in the data stream, as already described above with respect to FIG. 8 or in the previous paragraph. Merge candidate restriction signaling 230 is performed in the data stream. This is done either necessarily, i.e., independently of whether the selected merge candidate is bi-predictive or not, or omitting signaling 230 if the selected merge candidate is uni-predictive, in order to increase error robustness. The merge candidate restriction signaling 230 may be provided in response to the selected merge candidate being bi-predictive. If the merge candidate restriction signaling 230 indicates a constrained merging operation, a hypothesis selection indication 232 may additionally be provided in the data stream. Prediction information for a given block may then be provided as: 1) a uni-predictive prediction parameterized according to the selected prediction parameter merge candidate if the selected prediction parameter merge candidate is uni-predictive; 2) a hypothesis selection indication according to the selected prediction parameter merge candidate if the selected prediction parameter merge candidate is bi-predictive if the merge candidate restriction signaling 230 indicates a constrained merging operation. Unipredictive predictions parameterized according to one of the two hypotheses of parameter merging candidates 2) a prediction, in which one hypothesis is selected according to a hypothesis selection indicator 232; and 3) a selected If the prediction parameter merging candidates are bi-predictive, the merging candidate restriction signaling 230 If they do not exhibit restricted merging behavior, the two hypotheses of the selected predictive parameters merge candidates In this manner, p1 for block 106 is determined. However, according to an alternative embodiment, For example, a video decoder and a video encoder can be used to K+1 Further hypothesis addition in the form of 106 in the manner just outlined. Merging is processed, i.e., there is only p1 for such block 106. In this case, all the details presented above, as far as the just highlighted problem of restricted merge candidate list construction, are concerned with merging with respect to p1 independent of any other predictive signal, such as all the details presented above with respect to Figs. 1-3, i.e., internal The present invention focuses on how to implement the encoder and decoder in a practical manner, and how to subdivide the picture into blocks 80, including the currently processed block, i.e., 106. This application is intended to form a reservoir for further details on the recently highlighted embodiments.

[0123] While some aspects are described in the context of an apparatus, it will be apparent that these aspects also represent descriptions of corresponding methods, where a block or device corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method step also represent descriptions of a corresponding block or item, or feature of a corresponding apparatus. Some or all of the method steps may be performed (or may be implemented) by a hardware apparatus, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, among the most important method steps, may be performed by such an apparatus.

[0124] The data stream of the present invention may be stored on a digital storage medium or transmitted over a transmission medium, such as a wireless or wired transmission medium, such as the Internet.

[0125] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. Implementations may use a digital storage medium, such as a floppy disk, DVD, Blu-Ray, CD, ROM, PROM, EPROM, EEPROM, or FLASH memory, having electronically readable control signals stored thereon, which cooperates (or is capable of cooperating) with a programmable computer system on which the respective methods are executed. The digital storage medium may thus be computer readable.

[0126] Some embodiments of the present invention are implemented by performing one of the methods described herein. The device further comprises a data carrier having electronically readable control signals capable of cooperating with a programmable computer system such as a

[0127] Generally, embodiments of the present invention can be implemented as a computer program product having program code operable to perform one of the methods when the computer program product runs on a computer. The code may for example be stored on a machine readable carrier.

[0128] Another embodiment is one of the methods described herein stored on a machine-readable carrier. The present invention relates to a computer program product for executing the above-mentioned method.

[0129] In other words, an embodiment of the inventive method therefore comprises a computer program for performing one of the methods described herein when the computer program runs on a computer. It is a computer program having a program code.

[0130] A further embodiment of the inventive method is therefore a data carrier (or a digital storage medium or a computer readable medium) comprising recorded thereon a computer program for performing one of the methods described herein. A storage medium, or recording medium, is typically tangible and / or non-transitory.

[0131] A further embodiment of the method of the invention therefore comprises one of the methods described herein A data stream or a sequence of signals representing a computer program for executing one of the programs. The data stream or the sequence of signals may for example be arranged to be transferred via a data communication connection, for example via the Internet.

[0132] A further embodiment comprises a processing means, e.g. adapted to perform one of the methods described herein. The present invention relates to a computer or programmable logic device configured or adapted to perform the steps of the present invention.

[0133] A further embodiment is a computer for performing one of the methods described herein. The computer includes a computer having a data program installed thereon.

[0134] A further embodiment according to the invention is a device for carrying out one of the methods described herein. The apparatus or system may comprise, for example, a file server for transferring the computer program to the receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may comprise, for example, a file server for transferring the computer program to the receiver.

[0135] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array (FPGA) may be used. cooperates with a microprocessor to carry out one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0136] The apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0137] The apparatus described herein, or any components of the apparatus described herein, may be implemented at least in part in hardware and / or software.

[0138] The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0139] Any of the methods described herein, or components of the apparatus described herein, may be implemented at least in part by hardware and / or software.

[0140] The above-described embodiments are merely examples for the principles of the present invention. It is understood that modifications and variations of the configurations and details described herein are obvious to others skilled in the art. It is therefore intended to be limited only by the scope of the forthcoming patent claims, and not by the specific details presented throughout the description and explanation of the embodiments herein. [Explanation of symbols]

[0141] 10 Encoders 11 Videos 12 Pictures 14 Data Stream 20 Decoder 22 Prediction residual signal former 24 Prediction residual signal 26 Predictive Signals 28 Converter 32 Quantizer 34 Entropy Coder 36 Prediction Stage 38 Inverse quantizer 40 Inverse converter 42 Synthesizer 44 Prediction Module 46 Reconstructed Signals 50 Entropy Decoder 52 Inverse quantizer 54 Inverse converter 56 Synthesizer 58 Prediction Module 80 Coding Blocks 84 Conversion Blocks Set of 100 primitive prediction modes 102 Intra Prediction Mode 104 Inter Prediction Mode 106 Prescribed Block 108 Composite Predictive Signal 110 Primitive Prediction Ensemble 112 Primitive Prediction Mode Set 114 Derivation 116 Synthesis 124 Set of intra mode parameters 126 Set of Intermode parameters 128 Prediction Mode 130 Forecast Parameter Set, Forecast Mode Parameterization Related Information 138 Assembly Control Information 150 Addition, Repeat 160 Value Area 170, 172 Clipping and / or rounding operations 180 sample positions 190 adjacent blocks 194 Forecast Parameter Information 194 Forecast-related information 196 Merge Information 198 Forecast-related information 206 Forecast Parameter Information 210 Flag 212, 214, 216 Prediction parameters 218 Function Prediction Parameter Information 222 Prediction parameters 226 Merge Flags 228 Function Prediction Parameter Information 230 Merge Restrictions Display 232 Hypothesis Selection Display

Claims

1. 1. A video decoder for decoding video (11′) from a data stream (14) using block-based predictive decoding, comprising: A first prediction value p for the current block (106) using inter prediction mode １ Derive A second prediction value p for the current block (106) using an intra prediction mode. ２ Derive determining a scalar weighting parameter α based on prediction parameter information of blocks adjacent to the current block; The scalar weighting parameter α is used to estimate the first prediction p １ and the second predicted value p ２ The weighted sum of α·p １ +(1-α) p ２ determining a synthetic prediction signal (108) q for the current block (106) as A video decoder comprising a processor or electronic circuit configured to perform the above-mentioned steps, or a programmed computer.

2. The video decoder of claim 1 , wherein the prediction parameter information of the blocks neighboring the current block is a prediction mode of the blocks neighboring the current block.

3. 1. A video encoder for encoding a video (11) into a data stream (14) using block-based predictive coding, comprising: A first prediction value p for the current block (106) using inter prediction mode １ Derive A second prediction value p for the current block (106) using an intra prediction mode. ２ Derive determining a scalar weighting parameter α based on prediction parameter information of blocks adjacent to the current block; The scalar weighting parameter α is used to estimate the first prediction p １ and the second predicted value p ２ The weighted sum of α·p １ +(1-α) p ２ determining a synthetic prediction signal (108) q for the current block (106) as A video encoder comprising a processor or electronic circuit configured to perform the above-mentioned steps, or a programmed computer.

4. The video encoder of claim 3 , wherein the prediction parameter information of the blocks neighboring the current block is a prediction mode of the blocks neighboring the current block.

5. 1. A video decoding method for decoding video (11′) from a data stream (14) using block-based predictive decoding, comprising the steps of: A first prediction value p for the current block (106) using inter prediction mode １ and A second prediction value p for the current block (106) using an intra prediction mode. ２ and determining a scalar weighting parameter α based on prediction parameter information of blocks adjacent to the current block; The scalar weighting parameter α is used to estimate the first prediction p １ and the second predicted value p ２ The weighted sum of α·p １ +(1-α) p ２ determining a synthetic prediction signal (108) q for the current block (106) as A method for providing the above.

6. The method of claim 5 , wherein the prediction parameter information of the blocks neighboring the current block is a prediction mode of the blocks neighboring the current block.

7. A video encoding method for encoding a video (11) into a data stream (14) using block-based predictive coding, comprising the steps of: A first prediction value p for the current block (106) using inter prediction mode １ and A second prediction value p for the current block (106) using an intra prediction mode. ２ and determining a scalar weighting parameter α based on prediction parameter information of blocks adjacent to the current block; The scalar weighting parameter α is used to estimate the first prediction p １ and the second predicted value p ２ The weighted sum of α·p １ +(1-α) p ２ determining a synthetic prediction signal (108) q for the current block (106) as 13. A video encoding method comprising:

8. The method of claim 7 , wherein the prediction parameter information of the blocks neighboring the current block is a prediction mode of the blocks neighboring the current block.

9. A non-transitory digital storage medium storing a computer program, the computer program being capable of, when executed by a computer: A first prediction value p for the current block (106) using inter prediction mode １ and A second prediction value p for the current block (106) using an intra prediction mode. ２ and determining a scalar weighting parameter α based on prediction parameter information of blocks adjacent to the current block; The scalar weighting parameter α is used to estimate the first prediction p １ and the second predicted value p ２ The weighted sum of α·p １ +(1-α) p ２ determining a synthetic prediction signal (108) q for the current block (106) as A non-transient digital storage medium implementing a video decoding method comprising:

10. A non-transitory digital storage medium storing a computer program, the computer program being capable of, when executed by a computer: A first prediction value p for the current block (106) using inter prediction mode １ and A second prediction value p for the current block (106) using an intra prediction mode. ２ and determining a scalar weighting parameter α based on prediction parameter information of blocks adjacent to the current block; The scalar weighting parameter α is used to estimate the first prediction p １ and the second predicted value p ２ The weighted sum of α·p １ +(1-α) p ２ determining a synthetic prediction signal (108) q for the current block (106) as A non-transient digital storage medium implementing a video encoding method comprising:

Images