Encoder, decoder, encoding method and decoding method
Through block-based prediction codec and flexible configuration file options, the problems of low efficiency and increased complexity of video encoding and decoding in the prior art are solved, and efficient encoding and decoding of high-resolution and rotary motion scenes are realized. Ensures flexibility of bitstream and matching capability of decoder.
Patent Information
- Application Number
- CN202411142815.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2019-03-11
- Filing Date
- 2020-03-10
- Publication Date
- 2024-11-12
AI Technical Summary
Existing video encoding and decoding technologies have problems with inefficiency and increased complexity in handling parallel processing and visual artifact prevention, especially in high resolution and rotary motion scenarios, and configuration file and level-related bitstream attribute descriptions Inflexible enough, it makes it difficult for the decoder to effectively match the video stream needs.
Adopting block-based prediction codecs, supporting affine motion model and transform domain encoding, optimize motion vector extraction through surround processing and sub-block segmentation, combined with flexible configuration files and level-related encoding options, adapting to videos of different resolutions and formats Stream and independently parse buffer cycles and picture-timed SEI messages to improve decoding efficiency.
It improves the parallel processing capabilities of video encoding and decoding, reduces visual artifacts, enhances the encoding efficiency of rotational motion, ensures the flexibility of the bitstream and the matching ability of the decoder, and improves the overall encoding efficiency and decoding performance.
Smart Images

Figure CN118945339A_ABST
Abstract
Description
[0001] This application is a divisional application of the Chinese patent application "Encoder and decoder with profile and level-related encoding options, encoding method and decoding method" (application number: 202080035030.X) with a filing date of March 10, 2020. Technical Field
[0002] The present invention relates to video encoding and video decoding, and in particular to an encoder and a decoder with profile and level dependent encoding options, an encoding method, and a decoding method. Background Art
[0003] H.265 / HEVC (HEVC=High Efficiency Video Coding) is a video codec that already provides tools for improving or even enabling parallel processing at the encoder and / or at the decoder. For example, HEVC supports the subdivision of a picture into an array of tiles that are coded independently of each other. Another concept supported by HEVC relates to WPP, according to which CTU rows or CTU lines of a picture can be processed in parallel from left to right, for example in strips, provided that some minimum CTU offsets (CTU=Coding Tree Unit) are observed when processing consecutive CTU lines. However, it would be advantageous to have a video codec with parallel processing capabilities that more efficiently support a video encoder and / or video decoder.
[0004] In the following, an introduction to the VCL partitioning according to the state of the art is described (VCL=Video Coding Layer).
[0005] Typically, in video coding, the coding process of picture samples requires smaller partitions, where the samples are divided into rectangular areas for joint processing, such as prediction or transform coding. Therefore, the picture is divided into blocks of a certain size, which is constant during the coding of the video sequence. In the H.264 / AVC standard, fixed-size blocks of 16x16 samples, so-called macroblocks, are used (AVC = Advanced Video Coding).
[0006] In the state-of-the-art HEVC standard (see [1]), there are coding tree blocks (CTBs) or coding tree units (CTUs) of maximum size 64x64 samples. In the further description of HEVC, the more general term CTU is used for this type of block.
[0007] The CTUs are processed in raster scan order, starting with the top-left CTU and processing the CTUs in the picture in lines until the bottom-right CTU.
[0008] The coded CTU data is organized into a container called a slice. Originally, in previous video coding standards, a slice meant a fragment of one or more consecutive CTUs of a picture. Slices are used for segmentation of coded data. From another point of view, a complete picture can also be defined as one large slice, so historically, the term slice still applies. In addition to the coded picture samples, a slice also includes additional information about the encoding process of the slice itself, which is placed in the so-called slice header.
[0009] According to current technology, VCL (Video Coding Layer) also includes techniques for slicing and spatial partitioning. For example, such partitioning can be applied in video coding for various reasons, including processing load balancing in parallelization, CTU size matching in network transmission, error mitigation, etc. Summary of the invention
[0010] It is an object of the present invention to provide improved concepts for video encoding and video decoding.
[0011] The objects of the invention are solved by the subjects of the independent claims.
[0012] Preferred embodiments are provided in the dependent claims. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, in which:
[0014] Figure 1 A video encoder according to an embodiment is shown.
[0015] Figure 2 A video decoder according to an embodiment is shown.
[0016] Figure 3 A system according to an embodiment is shown.
[0017] Figure 4 A viewport-dependent tile arrangement with an overview is shown.
[0018] Figure 5 Affine motion by sub-block partitioning is shown.
[0019] Figure 6 Scaling motion with affine prediction is shown.
[0020] Figure 7 The sub-block partitioning for affine prediction in different channels is shown.
[0021] Figure 8 A current technology example bitstream order for this case is shown.
[0022] Fig. 9A video encoder is shown.
[0023] Fig.10 A video decoder is shown.
[0024] Fig.11 The relationship between the reconstructed signal, ie the reconstructed picture, on the one hand and the combination of the prediction residual signal and the prediction signal as signaled in the data stream, on the other hand is shown. DETAILED DESCRIPTION
[0025] The following description of the drawings starts with the presentation of a description of an encoder and a decoder of a block-based prediction codec for encoding pictures of a video, in order to form an example of a coding framework in which embodiments of the present invention may be embedded. Figures 9 to 11 In the following, a description of embodiments of the concepts of the present invention and how these concepts are respectively constructed into Fig. 9 The encoder and Fig.10 's decoder, although the subsequent Figures 1 to 3 and subsequent descriptions can also be used to form Fig. 9 The encoder and Fig.10 The encoder and decoder operate under the encoding framework of the decoder.
[0026] Fig. 9 A video encoder is shown, an apparatus for predictively encoding a picture 12 into a data stream 14, exemplarily using transform-based residual coding. The apparatus or encoder is indicated with reference numeral 10. Fig.10 A corresponding video decoder 20 is shown, i.e. an apparatus 20 configured to predictively decode a picture 12' from a data stream 14 also using transform-based residual decoding, wherein a prime sign has been used to indicate that the picture 12' reconstructed by the decoder 20 deviates from the picture 12 originally encoded by the apparatus 10 in terms of coding losses introduced by quantization of the prediction residual signal. Fig. 9 and Fig.10 Transform-based prediction residual coding is used illustratively, although the embodiments of the present application are not limited to such prediction residual coding. Fig. 9 and Fig.10 The same is true for the other details of the description, as will be outlined below.
[0027] The encoder 10 is configured to perform a spatial to spectral transform on the prediction residual signal and encode the prediction residual signal obtained thereby into the data stream 14. Likewise, the decoder 20 is configured to decode the prediction residual signal from the data stream 14 and perform a spectral to spatial transform on the prediction residual signal obtained thereby.
[0028] Internally, the encoder 10 may include a prediction residual signal former 22, which generates a prediction residual 24 in order to measure the deviation of the prediction signal 26 from the original signal, i.e. from the picture 12. The prediction residual signal former 22 may be, for example, a subtractor, which subtracts the prediction signal from the original signal, i.e. from the picture 12. The encoder 10 then also includes a transformer 28, which performs a spatial to spectral transformation on the prediction residual signal 24 to obtain a spectral domain prediction residual signal 24', which is then quantized by a quantizer 32 also included in the encoder 10. The prediction residual signal 24" thus quantized is encoded into the bitstream 14. To this end, the encoder 10 may optionally include an entropy encoder 34, which entropy encodes the transformed and quantized prediction residual signal to encode it into the data stream 14. The prediction signal 26 is generated by a prediction stage 36 of the encoder 10 based on the prediction residual signal 24" encoded into the data stream 14 and decodable from the data stream 14. To this end, as Fig. 9 As shown, the prediction stage 36 may internally include an inverse quantizer 38, followed by an inverse transformer 40, the inverse quantizer 38 inverse quantizing the prediction residual signal 24" to obtain a spectral domain prediction residual signal 24"' corresponding to the signal 24' except for the quantization loss, and the inverse transformer 40 inversely transforms the latter prediction residual signal 24"', that is, spectrum to spatial transformation, to obtain a prediction residual signal 24"" corresponding to the original prediction residual signal 24 except for the quantization loss. Then, the combiner 42 of the prediction stage 36 recombines the prediction signal 26 and the prediction residual signal 24"" by, for example, addition, to obtain a reconstructed signal 46, that is, a reconstruction of the original signal 12. The reconstructed signal 46 may correspond to the signal 12'. Then, the prediction module 44 of the prediction stage 36 generates the prediction signal 26 based on the signal 46 by using, for example, spatial prediction (i.e., intra-picture prediction) and / or temporal prediction (i.e., inter-picture prediction).
[0029] Likewise, if Fig.10 As shown, the decoder 20 may be internally composed of components corresponding to the prediction stage 36 and interconnected in a manner corresponding to the prediction stage 36. Specifically, the entropy decoder 50 of the decoder 20 may entropy decode the quantized spectral domain prediction residual signal 24″ from the data stream, and then the inverse quantizer 52, the inverse transformer 54, the combiner 56 and the prediction module 58 interconnected and cooperating in the manner described above with respect to the modules of the prediction stage 36 recover the reconstructed signal based on the prediction residual signal 24″, so that Fig.10 As shown, the output of combiner 56 produces a reconstructed signal, namely picture 12'.
[0030] Although not specifically described above, it is easy to understand that the encoder 10 can set some encoding parameters, including, for example, prediction modes, motion parameters, etc., according to some optimization schemes (for example, in a manner to optimize some rate-distortion related criteria, such as coding loss). For example, the encoder 10 and the decoder 20 and the corresponding modules 44 and 58 can support different prediction modes, such as intra-frame coding mode and inter-frame coding mode, respectively. The granularity of the encoder and the decoder switching between these prediction mode types can correspond to the subdivision of the pictures 12 and 12' into coding segments or coding blocks, respectively. For example, in units of these coding segments, the pictures can be subdivided into intra-frame coding blocks and inter-frame coding blocks. As outlined in more detail below, the intra-frame coding blocks are predicted based on the spatially encoded / decoded neighborhood of the corresponding block. There can be several intra-frame coding modes, including directional or angular intra-frame coding modes, and the intra-frame coding mode is selected for the corresponding intra-frame coding segment. According to the selected intra-frame coding mode, the corresponding segment is filled by extrapolating the sample values of the neighborhood into the corresponding intra-frame coding segment along a direction specific to the corresponding directional intra-frame coding mode. The intra coding mode may also, for example, include one or more other modes, such as a DC coding mode and / or a plane intra coding mode, according to which the prediction of a corresponding intra coding block assigns a DC value to all samples within a corresponding intra coding segment, according to which the prediction of the corresponding block is approximated or determined as a spatial distribution of sample values at sample positions of the corresponding intra coding block described by a two-dimensional linear function, wherein the slope and offset of the plane defined by the two-dimensional linear function are driven based on neighboring samples. In contrast, inter coding blocks may, for example, be predicted temporally. For an inter coding block, a motion vector may be signaled within the data stream, the motion vector indicating a spatial displacement of a portion of a previously encoded picture of the video to which the picture 12 belongs, at which position the previously encoded / decoded picture is sampled in order to obtain a prediction signal for the corresponding inter coding block. This means that, in addition to the residual signal encoding included in the data stream 14, such as the entropy encoded transform coefficient levels representing the quantized spectral domain prediction residual signal 24", the data stream 14 may encode therein coding mode parameters for assigning coding modes to individual blocks, prediction parameters for some blocks (such as motion parameters for inter-frame coded fragments), and optionally other parameters (such as parameters for controlling and signaling the subdivision of the pictures 12 and 12', respectively, into fragments). The decoder 20 uses these parameters to subdivide the pictures in the same way as the encoder, assign the same prediction modes to the fragments, and perform the same prediction to produce the same prediction signal.
[0031] Fig.11The relationship between the reconstructed signal, ie the reconstructed picture 12', on the one hand, and the combination of the prediction residual signal 24"" and the prediction signal 26 signaled in the data stream 14 on the other hand is shown. As mentioned above, the combination may be an addition. The prediction signal 26 is Fig.11 , the picture region is shown to be subdivided into intra-frame coding blocks schematically indicated by hatching and inter-frame coding blocks schematically indicated by non-hatching. The subdivision may be any subdivision, such as a regular subdivision of the picture region into rows and columns of square blocks or non-square blocks, or a multi-tree subdivision (such as quadtree subdivision, etc.) of the picture 12 from a tree root block into a plurality of leaf blocks of different sizes, wherein Fig.11 A hybrid of these is shown, where the picture region is first subdivided into rows and columns of tree root blocks, and then the rows and columns of the tree root blocks are further subdivided into one or more leaf blocks according to a recursive multi-tree subdivision.
[0032] Again, the data stream 14 may have encoded therein an intra-coding mode for an intra-coding block 80, which assigns one of several supported intra-coding modes to the corresponding intra-coding block 80. For the inter-coding block 82, the data stream 14 may have encoded therein one or more motion parameters. In general, the inter-coding block 82 is not limited to being encoded in time. Alternatively, the inter-coding block 82 may be any block predicted from a previously encoded portion other than the current picture 12 itself (e.g., a previously encoded picture of the video to which the picture 12 belongs, or, in the case where the encoder and decoder are scalable encoders and decoders, respectively, a picture of another view or a lower layer of the hierarchy).
[0033] Fig.11 The prediction residual signal 24″″ in is also shown as a subdivision of the picture area into blocks 84. These blocks may be referred to as transform blocks in order to distinguish them from the coding blocks 80 and 82. In practice, Fig.11 It is shown that the encoder 10 and the decoder 20 can respectively subdivide the picture 12 and the picture 12' into blocks using two different subdivisions, namely one subdivision into coding blocks 80 and 82 respectively, and another subdivision into blocks 84. The two subdivisions may be the same, that is, each coding block 80 and 82 can also form a transform block 84, but Fig.11The following situation is shown: in which, for example, the subdivision into a transform block 84 forms an extension of the subdivision into coding blocks 80, 82, so that any border between two blocks of blocks 80 and 82 covers the border between two blocks 84, or alternatively, each block 80, 82 either coincides with one of the transform blocks 84 or with a cluster of transform blocks 84. However, the subdivision can also be determined or selected independently of each other, so that the transform block 84 can alternatively cross the block border between blocks 80, 82. With regard to the subdivision into transform blocks 84, similar statements are therefore correct as proposed with respect to the subdivision into blocks 80, 82, i.e. the block 84 can be the result of a simple subdivision of the picture area into blocks (arranged in rows and columns or not arranged in rows and columns), the result of a recursive multi-tree subdivision of the picture area or a combination thereof or any other type of blockation. By the way, it is to be noted that the blocks 80, 82 and 84 are not limited to square, rectangular or any other shape.
[0034] Fig.11 It is also shown that the combination of the prediction signal 26 and the prediction residual signal 24"" directly produces the reconstructed signal 12'. However, it should be noted that according to alternative embodiments more than one prediction signal 26 may be combined with the prediction residual signal 24"" to produce the picture 12'.
[0035] exist Fig.11 In the embodiment described below, the transform block 84 should have the following meaning. The transformer 28 and the inverse transformer 54 perform their transforms in units of these transform blocks 84. For example, many codecs use some kind of DST or DCT for all transform blocks 84. Some codecs allow skipping transforms so that for some transform blocks 84, the prediction residual signal is encoded directly in the spatial domain. However, according to the embodiments described below, the encoder 10 and the decoder 20 are configured in a way that they support several transforms. For example, the transforms supported by the encoder 10 and the decoder 20 may include:
[0036] o DCT-II (or DCT-III), where DCT stands for Discrete Cosine Transform
[0037] o DST-IV, where DST stands for Discrete Sine Transform
[0038] o DCT-IV
[0039] o DST-VII
[0040] oIdentity Transformation (IT)
[0041] Naturally, the transformer 28 will support all forward transformed versions of these transforms, while the decoder 20 or inverse transformer 54 will support their corresponding inverse or reverse versions:
[0042] oInverse DCT-II (or Inverse DCT-III)
[0043] oInverse DST-IV
[0044] oInverse DCT-IV
[0045] oInverse DST-VII
[0046] oIdentity Transformation (IT)
[0047] The subsequent description provides more details on which transforms may be supported by the encoder 10 and decoder 20. In any case, it should be noted that the set of supported transforms may include only one transform, such as a spectral to spatial or spatial to spectral transform.
[0048] As mentioned above, it has been presented Figures 9 to 11 As an example, the inventive concept further described below may be implemented to form a specific example of an encoder and a decoder according to the present application. In this regard, Fig. 9 The encoder and Fig.10 The decoders may represent possible implementations of the encoder and decoder described below in this document, respectively. However, Fig. 9 and Fig.10 However, an encoder according to an embodiment of the present application may perform block-based encoding of picture 12 using the concepts outlined in more detail below. Fig. 9 The encoder of the embodiment is different in the following aspects: it is not a video encoder but a still picture encoder, does not support inter-frame prediction, or uses the same Fig.11 The subdivision into blocks 80 is performed in different ways of the example. Likewise, a decoder according to an embodiment of the present application can use the coding concepts outlined further below to perform block-based decoding of pictures 12' from the data stream 14, but the decoder can be used with Fig.10 The decoder 20 of the embodiment is different in the following aspects: it is not a video encoder but a still picture encoder, does not support intra-frame prediction, or uses the same frame as the reference frame. Fig.11 The described methods subdivide the picture 12 ′ into blocks in a different manner and / or derive the prediction residual from the data stream 14 not in the transform domain, but for example in the spatial domain.
[0049] In the following, a general video encoder according to an embodiment is described in Figure 1 As described in, the general video decoder according to the embodiment is Figure 2 Described in, and according to the general system of the embodiment Figure 3 Described in.
[0050] Figure 1 A generic video encoder 101 according to an embodiment is shown.
[0051] The video encoder 101 is configured to encode a plurality of pictures of a video by generating an encoded video signal, wherein each picture of the plurality of pictures comprises original picture data.
[0052] The video encoder 101 comprises a data encoder 110 configured for generating an encoded video signal comprising encoded picture data, wherein the data encoder is configured to encode a plurality of pictures of a video into the encoded picture data.
[0053] Furthermore, the video encoder 101 comprises an output interface 120 configured for outputting encoded picture data of each picture of the plurality of pictures.
[0054] Figure 2 A generic video decoder 151 according to an embodiment is shown.
[0055] The video decoder 151 is configured to decode the encoded video signal including the encoded picture data to reconstruct a plurality of pictures of the video.
[0056] The video decoder 151 comprises an input interface 160 configured to receive an encoded video signal.
[0057] Furthermore, the video decoder includes a data decoder 170 configured to reconstruct a plurality of pictures of the video by decoding the encoded picture data.
[0058] Figure 3 A general system according to an embodiment is shown.
[0059] The system includes Figure 1 The video encoder 101 and Figure 2 The video decoder 151 is provided.
[0060] The video encoder 101 is configured to generate an encoded video signal. The video decoder 151 is configured to decode the encoded video signal to reconstruct a picture of the video.
[0061] Hereinafter, the first aspect of the present invention is now described in detail.
[0062] Specifically, the first aspect relates to motion vector wrapping.
[0063] When encoding omnidirectional content using certain projection formats such as the widely deployed ERP format (ERP = equirectangular projection), samples on the left and right borders of the projected picture are actually adjacent in 3d space after re-projection. In order to exploit this property and prevent visual artifacts in state-of-the-art video coding, the typical process of extrapolating image sample values on the left and right picture boundaries is adjusted when inter-frame prediction based on motion compensation blocks reaches outside the picture boundary.
[0064] Instead of creating the traditional boundary filling sample values by vertical extrapolation of the last pixel columns on the left and right sides of the picture, the following clipping operation is used in the derivation of the reference sample positions, which results in the motion vectors and reference blocks wrapping around the defined sample columns:
[0065] xInt i =sps_ref_wraparound_enabled_flag?
[0066] ClipH(sps_ref_wraparound_offset,picW,(xInt L +i-3)):
[0067] Clip3(0,picW-1,xInt L +i-3)
[0068] yInt i =Clip3(0,picH-1,yInt L +i-3)
[0069] where ClipH(o,W,x) is defined as:
[0070] mod(x,o); for x<0
[0071] mod((xW),o)+Wo; for x>W-1, or
[0072] X; Other
[0073] Clip3 is defined in [1] as:
[0074]
[0075] Problems arise when such content is further used for omnidirectional video services, for example based on the OMAF viewport-dependent profile (OMAF = Omnidirectional Media Format) using tiles. In such services, low-resolution variants of ERP content versions are often mixed with other high-resolution segments of the full content to create a viewport-dependent stream. The necessity to arrange the segments of the high-resolution full scene and the low-resolution variants of the content onto a rectangular video frame can result in a rotation of the low-resolution ERP content, whereby the encoding cannot exploit the above-mentioned properties to avoid visual artifacts. For example, in Figure 4 In Figure 2, two arrangements are shown, featuring a low-resolution ERP overview and a high-resolution tile. As can be seen, the ERP tile is either rotated or adjacent to the tile on the right.
[0076] therefore, Figure 4 The two tile arrangements shown in place demands on MV surround that cannot be met by current technology designs.
[0077] Figure 4 A viewport-dependent tile arrangement with an overview is shown according to an embodiment.
[0078] According to this aspect, the MV wraparound may be adapted to allow the above tile arrangement, for example, in the following manner. In one embodiment, the signaling sps_ref_wraparound_idc is used to distinguish the MV wraparound options, such as Figure 4 The rotation on the left side of :
[0079]
[0080] In another embodiment, the surrounding area is defined as covering Figure 4 The scene depicted on the right side.
[0081] Hereinafter, the second aspect of the present invention is now described in detail.
[0082] Specifically, the second aspect relates to the affine prediction runtime cost.
[0083] Video codecs of current technology are generally designed to cover a wide range of applications, use cases, and scenarios. For example, codecs such as HEVC include technologies for streaming, broadcast, live conversation, or monitoring services. Each of these services typically operates at a different resolution depending on the service constraints: broadcast typically uses a higher resolution than conversation, and so on. While the numerous individual tools included by the codec are tuned to fit this wide range of associated requirements, there may be efficiency losses or increased complexity at each individual operating point.
[0084] For example, video coding traditionally uses a translation-only motion model, i.e. rectangular blocks are displaced according to a two-dimensional motion vector to form a motion compensated predictor of the current block. Such a model cannot express rotational or scaling motions that are common in many video sequences, and therefore efforts have been made to slightly extend the traditional translational motion model, such as the affine motion in what is known as JEM (JEM = Joint Exploration Model). Figure 5 As shown, in this affine motion model, two (or three) so-called control point motion vectors (v0, v1, ...) per block are used to model rotation and scaling motion. The control point motion vectors are selected from the MV candidates of neighboring blocks. The resulting motion compensated predictor is derived by computing the sub-block based motion vector field and relying on conventional translational motion compensation of rectangular sub-blocks.
[0085] Figure 5 The affine motion by sub-block segmentation is shown. Specifically, Figure 5 The resulting sub-block reference or predictor (dashed red line) for the top right sub-block (solid red line) is exemplarily shown.
[0086] Although it depends on the control point motion vector, Figure 5 represents a simpler or less simple rotational motion, but it can happen that individual sub-block references span wide distances in the picture, for example when representing a zoom motion. In this case, the resulting sub-block references can be, for example, Figure 6 The positioning shown is exemplarily shown with a smaller block size resulting in a 2x2 sub-block.
[0087] Specifically, Figure 6 Scaling motion with affine prediction is shown.
[0088] In this case, the decoder needs to extract individual sub-block references from a larger total picture area covered by the complete current block. In order to implement this affine prediction mode cost-effectively, the extraction of individual sub-block references needs to be coordinated on currently available hardware. Such coordination can be, for example, a joint extraction of consecutive reference picture areas including all sub-block references. Therefore, implementing extreme scaling motions on the decoder side can be expensive since efficient reference extraction strategies are excluded.
[0089] In order to limit the worst-case reference picture access without sacrificing the substantial coding gain of the affine prediction mode, a method of limiting the extension of the prediction block in the reference picture is proposed in the current technology.
[0090] According to this aspect, such a limit may be derived, for example, depending on the maximum memory bandwidth allowed. This may be specified in terms of overall maximum memory bandwidth (e.g. specification of usage level limits), approximately limiting the memory bandwidth overhead (defining the maximum increase in required reference memory accesses relative to conventional translation prediction) or explicitly signaled by the encoder in high-level parameters. These approaches may be hybrid.
[0091] In one embodiment, the limit on the extension of the reference sub-block is enforced by ensuring that the bounding box around the resulting sub-block reference block does not exceed a threshold. Such a threshold can be derived from the block size, picture resolution, maximum allowed bandwidth or explicitly signaled in high-level parameters.
[0092] In another embodiment, the restriction on the reference sub-block extension depends on the coded picture resolution or is defined by a level restriction.
[0093] Hereinafter, the third aspect of the present invention is now described in detail.
[0094] Specifically, the third aspect relates to affine predicted chrominance processing.
[0095] The coding efficiency of the above affine prediction model is inversely proportional to the sub-block size, since smaller sub-blocks allow for better approximation of affine motion using the underlying translation block-based motion model. However, fine-grained reference picture access comes with a significant runtime cost.
[0096] When the luma and chroma components of the video are not uniformly sampled, such as in the main consumer video format 4:2:0, the prediction / reference blocks are usually sampled at the same ratio. For example, when a 16x16 luma block is predicted by traditional methods (such as translational motion compensation block-based prediction), the same luma block motion vector is used to predict the corresponding 8x8 chroma block.
[0097] In the context of the above affine prediction model, since the luma subblocks are already very fine-grained or already small, the current state-of-the-art solution for processing chroma blocks bases the chroma processing on the same luma block size regardless of the format subsampling. Thus, a 4x4 subblock in the luma channel in affine prediction results in a 4x4 subblock in the chroma channel, whereby the number of chroma channels is less than the number of luma channels, as Figure 7 shown.
[0098] Figure 7 The sub-block partitioning for affine prediction in different channels is shown.
[0099] Therefore, a method is needed to derive motion information for the corresponding chrominance block. There are different styles of solutions. For example, the simplest current technology solution averages the motion information mvLX[] of all four corresponding luminance sub-blocks, such as shown below:
[0100] mvAvgLX=(mvLX[(xSbIdx>>1<<1)][(ySbIdx>>1<<1)]+
[0101] mvLX[(xSbIdx>>1<<1)+1][(ySbIdx>>1<<1)]+
[0102] mvLX[(xSbIdx>>1<<1)][(ySbIdx>>1<<1)+1]+
[0103] mvLX[(xSbIdx>>1<<1)+1][(ySbIdx>>1<<1)+1]+2)>>2
[0104] Another option is to just average the corresponding top-left and bottom-right luma motion vectors, as follows:
[0105] mvAvgLX=(mvLX[(xSbIdx>>1<<1)][(ySbIdx>>1<<1)]+
[0106] mvLX[(xSbIdx>>1<<1)+1][(ySbIdx>>1<<1)+1]+1)>>1
[0107] Additionally, symmetric rounding can be introduced as follows:
[0108] mvAvgLX=(mvLX[(xSbIdx>>1<<1)][(ySbIdx>>1<<1)]+
[0109] mvLX[(xSbIdx>>1<<1)+1][(ySbIdx>>1<<1)+1])
[0110] mvAvgLX[0]=(mvAvgLX[0]>=0?
[0111] (mvAvgLX[0]+1)>>1:
[0112] -((-mvAvgLX[0]+1)>>1))
[0113] mvAvgLX[1]=(mvAvgLX[1]>=0?
[0114] (mvAvgLX[1]+1)>>1:
[0115] -((-mvAvgLX[1]+1)>>1))
[0116] However, problems arise when using chroma formats other than 4:2:0, where other averaging schemes beyond the current style described above are needed. For example, when using a 4:4:4 chroma format, averaging should be avoided because each 4x4 chroma subblock corresponds to exactly one 4x4 luma subblock, while a 4:2:2 chroma format only requires averaging in one dimension. Therefore, according to this aspect, the correct luma block involved in the averaging operation is derived by introducing the chroma subsampling rate into the derivation of mvAvgLX. In one embodiment, the derivation is implemented as follows:
[0117] mvAvgLX=(mvLX[(xSbIdx>>SubWidthC< <SubWidthC)]
[0118] [(ySbIdx>>SubHeightC< <SubHeightC)]+
[0119] mvLX[(xSbIdx>>SubWidthC< <SubWidthC)+SubWidthC]
[0120] [(ySbIdx>>SubHeightC< <SubHeightC)+SubHeightC])
[0121] mvAvgLX[0]=(mvAvgLX[0]>=0?
[0122] (mvAvgLX[0]+1)>>1:
[0123] -((-mvAvgLX[0]+1)>>1))
[0124] mvAvgLX[1]=(mvAvgLX[1]>=0?
[0125] (mvAvgLX[1]+1)>>1:
[0126] -((-mvAvgLX[1]+1)>>1))
[0127] where SubWidthC and SubHeightC are derived from the applied chroma format as follows:
[0128] Chroma Format SubWidthC SubHeightC 4:2:0 1 1 4:2:2 1 0 4:4:4 0 0
[0129] Hereinafter, the fourth aspect of the present invention is now described in detail.
[0130] Specifically, the fourth aspect relates to interpolation filter overhead.
[0131] Motion compensated inter-frame prediction can use sample values at sub-sample positions (e.g., half-pixel, quarter-pixel), which are interpolated from sample values at integer sample positions by convolution filters. Current codecs generally use a fixed-tap filter kernel, such as 8 taps, to generate these sub-pixel sample values.
[0132] However, when sub-block based inter prediction, such as affine prediction or DMVR (decoder side motion vector refinement) results in relatively small luma and chroma sub-blocks (e.g. 4x4 size), memory bandwidth issues occur. As the (sub)block size decreases relative to the fixed interpolation filter size, the overhead in terms of the total number of samples accessed for the generated reference luma sub-pixel samples of the (sub)block increases. Therefore, it has been proposed to perform sub-pixel interpolation on small sub-blocks (e.g. 4x4) using relatively short tap filter kernels, which shows a minimal coding penalty that decreases as the content resolution increases.
[0133] According to this aspect, the selection of shorter tap interpolation filters for sub-pixel interpolation is adapted depending on the resolution of the encoded content in order to benefit from gains in peak memory bandwidth reduction for high resolution content without sacrificing coding efficiency for lower resolution content.
[0134] In one embodiment, several sub-pixel interpolation filters are available (eg, bilinear, 4-tap, 6-tap, 8-tap), and the content resolution and block size are used to determine the applicable filter.
[0135] Each increase in the number of individual filters that need to be implemented in an encoder or decoder and kept readily available makes the implementation more expensive. It is therefore advantageous to reduce the number of individual filters to be implemented in parallel in an encoder or decoder.
[0136] For example, the current technology is that the chroma components of the video signal use spatial subsampling, such as when using a 4:2:0 chroma format. In this case, the chroma components also typically use shorter tap sub-pixel interpolation filters than the luminance components. Therefore, in addition to the traditional longer tap sub-pixel interpolation filter core for the luminance component, the decoder / encoder implementation also needs to be prepared for such shorter tap sub-pixel interpolation filters. In an embodiment, in the case where sub-block processing requires reduced tap filters to reduce peak memory bandwidth, reuse of shorter tap sub-pixel interpolation filters is introduced, for example depending on the coded picture size or selected by the encoder and indicated by the bitstream.
[0137] Hereinafter, the fifth aspect of the present invention is now described in detail.
[0138] Specifically, the fifth aspect relates to bitstream attribute description and profile set operations.
[0139] Typically, a video stream is accompanied by signaling that describes its characteristics and allows identification of the type of capabilities required by a video decoder so that the bitstream can be decoded.
[0140] Current state-of-the-art video coding standards, such as AVC (Advanced Video Coding), HEVC, include profile and level indications in parameter sets in the bitstream, allowing identification of which tools need to be supported and, for example, resolution limitations of a video coding sequence, making it easy to identify whether a decoder can decode the video sequence.
[0141] In terms of profiles, profiles that support only intra-frame coding tools or intra-frame and inter-frame tools, but only 8-bit configuration or 10-bit configuration, layered or non-layered coding, etc. can be identified.
[0142] In terms of levels (more related to bitrate, resolution and / or frame rate), one can distinguish 1080p@30Hz, 4Kp@30Hz, 4Kp@60Hz, 8Kp@30Hz, etc.
[0143] This information is typically used in video streaming systems for capability exchange so that characteristics are exposed and different receivers can match their capabilities to the requirements for decoding the provided bitstream.
[0144] For example, in DASH (Dynamic Adaptive Streaming over HTTP), the Media Presentation Description (MPD) may include several videos provided for the same content with corresponding different parameters. This is done by indicating in an attribute called @codec="xxx", where the codec, profile, level, etc. are indicated. Similarly, in RTP streaming (RTP = Real-time Transport Protocol), there is a Session Description Protocol (SDP), by which the operating point of the session is negotiated. The parties involved in the session then agree on which codec, profile and level should be used to establish the session. Furthermore, in the transport stream MPEG2-TS (MPEG = Moving Picture Experts Group; TS = Transport Stream), which is heavily used in broadcast transmissions, there are means to expose this value to the terminal device. This is done by including in the program table the corresponding video descriptor, which identifies the codec, profile and level of the program (i.e., for example a TV channel), so that the appropriate channel that can be decoded by the terminal device can be selected.
[0145] Currently, the properties included in the bitstream apply only to Coded Video Sequences (CVS). A CVS starts with the first IRAP (Intra Random Access Point) and ends with an IRAP with one of the following properties:
[0146] It is IDR (Instantaneous Decoder Refresh)
[0147] It is BLA (Broken Link Access) (only relevant for HEVC)
[0148] • It has an End of Sequence (EOS) SEI message (SEI = Supplemental Enhancement Information).
[0149] In most video services, a "session" may include several temporally consecutive CVSs, so the parameters indicated in parameter sets for AVC and HEVC are not sufficient, as they describe only the first CVS of all CVSs involved in a "session". This problem has been addressed in MPEG2-TS with the following text:
[0150] “NOTE—In one or more sequences in an AVC video stream, a level lower than the level signaled in the AVC video descriptor may also be present while a profile that is a subset of the profile signaled in the AVC video descriptor may also be present. However, throughout the AVC video stream, only the tools included in the profile signaled in the AVC video descriptor, if present, shall be used. For example, if the Main Profile is signaled, then the Baseline Profile may be used in some sequences, but only those tools in the Main Profile are used. If sequence parameter sets in an AVC video stream signal different profiles, and no additional constraints are signaled, then the stream may need to be checked to determine which profile, if any, the entire stream conforms to. If an AVC video descriptor is to be associated with an AVC video stream that does not conform to a single profile, then the AVC video stream MUST be partitioned into two or more substreams such that the AVC video descriptor may signal a single profile for each such substream.”
[0151] In other words, the profile and level indicated in the descriptor is a profile that is a superset of any profile used in a CVS of an AVC video stream, and the level is the highest level of any CVS.
[0152] In a similar way, the profile and level indicated in the MPD (Media Presentation Description) or SDP can be "higher" than the profile and level in the video bitstream (meaning a superset of the profile, or more tools required than the profile per CVS).
[0153] It would be desirable for the bitstream itself to include such signaling so that it is clear from the outset which is the set of tools needed to decode all CVSs of a video "session". (Here, the term video session also includes, for example, TV channel programs.) The signaling consists of a set of parameters that applies to several CVSs (i.e. to the bitstream).
[0154] There are two cases of particular interest:
[0155] CVS with configuration files, where there is no configuration file that is a superset of both CVSs.
[0156] • CVS that may change over time relative to the number of layers.
[0157] One option would be to use a per-tool flag to signal all tools needed to decode all CVSs, but this would lead to an undesirable solution since it is often more convenient to work based on configuration files than multiple tools which could lead to an extremely high number of combinations.
[0158] More than one profile active within a bitstream can result in the following four situations. The tools required to decode the bitstream correspond to the following:
[0159] An intersection of two profiles where there is no profile corresponding to the intersection.
[0160] The intersection of two profiles, where one profile is the intersection of two profiles.
[0161] The union of two profiles where there is no profile corresponding to the intersection.
[0162] The union of two profiles, where there is a profile corresponding to the intersection.
[0163] For situations where there are configuration files corresponding to all required tools, a single configuration file indication will be sufficient.
[0164] However, for cases where no profile exists to describe all necessary tools, more than one profile should be signaled, and the intersection or union should be signaled to indicate whether support for a single profile is sufficient to decode the bitstream (intersection) or support for both profiles is required to decode the bitstream.
[0165] Hereinafter, the sixth aspect of the present invention is now described in detail.
[0166] Specifically, the sixth aspect relates to the buffer period SEI and picture timing SEI message parsing correlation
[0167] In existing designs such as HEVC, syntax elements in the buffering period SEI message (SEI = Supplemental Enhancement Information), picture timing SEI message and decoding unit SEI message depend on syntax elements carried in the SPS (Sequence Parameter Set). This dependency includes the number of bits used to encode a specific syntax element and the presence of the syntax element.
[0168] However, it is not desirable to put all buffering period / picture timing / decoding unit related information in the same syntax location because:
[0169] 1) The syntax elements currently in SPS are useful for session / capability negotiation, which is done by exchanging parameter sets. Therefore, they cannot be signaled only in the buffering period SEI message.
[0170] 2) The syntax elements in the buffering period SEI message cannot be moved to the SPS, because activating a new SPS is only possible at an IRAP picture (IRAP = Intra Random Access Point), while a buffering period can start at any picture associated with temporal sub-layer 0. It is also undesirable to need to activate a new parameter set at each buffering period.
[0171] The above parsing dependency leads to a startup problem: there is no active SPS before the first VCL NAL unit of the video bitstream is parsed. Since SEI messages are sent before the first VCL NAL unit (NAL = Network Abstraction Layer), they will arrive before the SPS is activated.
[0172] Furthermore, in out-of-band signaling of parameter sets, intermediate devices such as splicers interested in understanding HRD parameters of a bitstream (e.g., for splicing bitstreams) may not be able to parse the corresponding SEI message because the relevant parameter sets are not included in the bitstream.
[0173] Furthermore, for a simple intermediary device (such as a splicer), it may be difficult to parse the entire SPS until it reaches the HRD parameters in the VUI.
[0174] In addition, the order of NAL units is not explicitly constrained to require the SPS to be sent before the buffering period SEI message that references the SPS. In this case, the decoder may have to store the corresponding SEI message in the bitstream format (without being able to parse it) until the SPS arrives and is activated. Only then can the SEI message be parsed.
[0175] Figure 8 A current technology example bitstream order for this case is shown.
[0176] In an embodiment, parsing of the corresponding SEI message is performed independently of the SPS in the following manner:
[0177] All of the following syntax elements are moved to the buffering period SEI message:
[0178] Specify the length of another syntax element in the corresponding SEI message, or
[0179] Not needed for session / capability negotiation (sub-picture clock divisor).
[0180] The length of all syntax elements in the buffering period SEI is completely independent of other syntax structures. The length of syntax elements in the timing SEI message and the decoding unit SEI message may depend on the syntax elements in the buffering period SEI message, but not on any parameter set value.
[0181] The presence flags of specific HRD parameters (NAL, VCL, DU, etc.), counts of CPB parameters (vui_cpb_cnt_minus1[i] and bp_cpb_cnt_minus1), and other syntax parts are repeated in parameter sets and buffering period SEI messages, and this aspect includes restricting the values of these repeated syntax elements to be equal in both positions.
[0182] The picture timing SEI message and the decoding unit SEI message are made independent by moving all syntax elements indicating the presence of other syntax elements (eg, frame_field_info_present_flag) into the corresponding SEI message or duplicating it.
[0183] As part of this aspect, in order to achieve bitstream consistency, the values of the above syntax elements should be equal to the corresponding values in the parameter set.
[0184] As part of this, unless concatenation_flag is equal to 1 (ie, splicing occurs), the tick_devisors (indicating sub-picture timing granularity) in the buffering period will have the same value.
[0185] As already outlined, in an embodiment, it is proposed to add HRD parameters, buffering period, picture timing and decoding unit information SEI messages to the VVC specification. The general syntax and semantics are based on the corresponding HEVC structure. The parsing of SEI messages may be independent of the presence of SPS.
[0186] As mentioned above, in order to check the consistency of the bitstream, a buffer model is needed.The VVC buffer model may, for example, be based on some simplifications of the Hypothetical Reference Decoder (HRD) specification used in HEVC.
[0187] HRD parameters are present in the SPS VUI. Buffering period SEI, picture timing SEI and decoding unit information SEI messages are added to the access units they refer to.
[0188] In HEVC, there is a parsing dependency between SPS and HRD-related SEI messages. Although the semantics only allow reference to SPS active in the same access unit, the bitstream order allows for situations where SEI can arrive before SPS. In this case, the coded SEI message needs to be stored until the correct SPS arrives.
[0189] Another case where this parsing dependency is problematic is with network elements and splicers, especially when parameter sets are exchanged out-of-band (e.g., in session negotiation). In this case, the parameter sets may not be included in the bitstream at all, making it impossible for the network element or splicer to decode the buffering / timing information at all. These elements may still be of interest when identifying values such as decoding time and frame / field information.
[0190] In an embodiment, it is proposed to make the parsing of the buffering period, picture timing and decoding unit information SEI messages independent of the SPS VUI / HRD parameters in the following way:
[0191] - Move all syntax elements indicating the length or presence of other syntax elements from the VUI to the buffering period SEI message. #
[0192] - Duplicate syntax elements that are needed in both locations. Constrain the values of these syntax elements to be the same.
[0193] Design Decisions:
[0194] - Not all syntax elements should be moved from VUI to buffering period SEI, because parameter sets are usually exchanged for session negotiation. So parameters important for session negotiation remain in VUI.
[0195] - Moving all buffering period parameters into the SPS is not possible, since the buffering period may start from a non-IRAP picture, where activating a new SPS is not possible.
[0196] - The correlation between the buffering period SEI and (the picture timing SEI or the decoding unit SEI) is not considered harmful, since there is also a logical correlation between these messages.
[0197] A syntax example based on HEVC syntax is given below.
[0198] The following syntax diagram is based on HEVC syntax, changes are marked in red. The semantics are usually moved to the new position of the syntax element (adapted to the syntax element name). Whenever a syntax element is repeated, a bitstream constraint is added: these syntax elements shall have the same value.
[0199] VUI / HRD parameters:
[0200]
[0201]
[0202]
[0203] Buffering period SEI message:
[0204]
[0205]
[0206] Picture timing SEI message:
[0207]
[0208] Decoding unit information SEI message:
[0209]
[0210] Although some aspects have been described in the context of an apparatus, it will be clear that these aspects also represent a description of a corresponding method, wherein a block or device corresponds to a method step or a feature of a method step. Similarly, the aspects described in the context of a method step also represent a description of a feature of a corresponding block or item or a corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware device (such as a microprocessor, a programmable computer, or an electronic circuit). In some embodiments, one or more of the most important method steps may be performed by such an apparatus.
[0211] According to certain implementation requirements, embodiments of the present invention may be implemented in hardware or software, or at least partially in hardware, or at least partially in software. Implementation may be performed using a digital storage medium (e.g., a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory) having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system to perform the corresponding method. Thus, the digital storage medium may be computer readable.
[0212] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.
[0213] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer.The program code may, for example, be stored on a machine readable carrier.
[0214] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
[0215] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
[0216] A further embodiment of the inventive method is therefore a data carrier (or a digital storage medium or a computer-readable medium) having recorded thereon the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium is typically tangible and / or non-transitory.
[0217] Therefore, another embodiment of the inventive method is a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transmitted via a data communication connection (eg, via the Internet).
[0218] A further embodiment comprises a processing means, for example a computer or a programmable logic device, configured to or adapted to perform one of the methods described herein.
[0219] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
[0220] Another embodiment according to the invention comprises an apparatus or system configured to transmit a computer program to a receiver (e.g. electronically or optically), the computer program being used to perform one of the methods described herein. The receiver may be, for example, a computer, a mobile device, a storage device, etc. The apparatus or system may, for example, comprise a file server for transmitting the computer program to the receiver.
[0221] In some embodiments, a programmable logic device (e.g., a field programmable gate array) can be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array can collaborate with a microprocessor to perform one of the methods described herein. Typically, the method is preferably performed by any hardware device.
[0222] The devices described herein may be implemented using hardware devices, or using computers, or using a combination of hardware devices and computers.
[0223] The methods described herein may be performed using a hardware device, or using a computer, or using a combination of a hardware device and a computer.
[0224] The above embodiments are merely illustrative of the principles of the present invention. It should be understood that modifications and variations of the arrangements and details described herein will be apparent to other persons skilled in the art. Therefore, it is intended that the scope of the present invention be limited only by the scope of the appended patent claims and not by the specific details given by way of the description and explanation of the embodiments herein.
[0225] References
[0226] [1] ISO / IEC, ITU-T. High efficiency video coding. ITU-T Recommendation H.265 | ISO / IEC 23008 10 (HEVC), 1st edition, 2013; 2nd edition, 2014.
Claims
1. A method for decoding a picture from a data stream, the method comprising: Decoding chroma format information describing how chroma of the picture is sampled from the data stream; Determining a motion vector corresponding to a chroma subblock of the picture based on previously determined motion vector information corresponding to at least one luma subblock of the picture, wherein: In response to the chroma format information indicating that the chroma sub-block is sampled in a 4:4:4 format, determining a motion vector corresponding to the chroma sub-block based on motion vector information corresponding to a luma sub-block, In response to the chroma format information indicating that the chroma sub-block is sampled in a 4:2:2 format, determining a motion vector corresponding to the chroma sub-block based on averaging motion vector information corresponding to two luminance sub-blocks in one dimension, and In response to the chroma format information indicating that the chroma subblock is sampled in a 4:2:0 format: selecting an upper left luminance subblock and a lower right luminance subblock, and averaging motion vector information corresponding to the two selected luminance subblocks to determine a motion vector corresponding to the chroma subblock; as well as The picture is reconstructed based on the motion vector information of the at least one luminance sub-block and the motion vector of the chrominance sub-block.
2. The method according to claim 1, wherein: When the chroma sub-block is sampled in a 4:2:0 format, averaging the motion vector information corresponding to the two selected luma sub-blocks further comprises: Symmetric rounding is performed on the motion vector information corresponding to the two selected luma sub-blocks.
3. The method according to claim 1, wherein: In response to the chroma format information indicating that the chroma sub-block is sampled in a 4:2:2 format or a 4:2:0 format, the method further includes: Based on the chroma format information, deriving a subWidthC chroma format parameter and a subHeightC chroma format parameter; selecting two luma sub-blocks based on the subWidthC chroma format parameter and the subHeightC chroma format parameter; and The motion vector information corresponding to the two selected luma sub-blocks is averaged.
4. The method according to claim 1, further comprising: Based on the chroma format information, deriving a subWidthC chroma format parameter and a subHeightC chroma format parameter; as well as When the subWidthC chroma format parameter and the subHeightC chroma format parameter are different values, it is determined that the chroma sub-block is sampled in a 4:2:2 format.
5. An electronic device for decoding a picture from a data stream, the electronic device comprising: A processor, the processor being configured to: Decoding chroma format information describing how chroma of the picture is sampled from the data stream; Determining a motion vector corresponding to a chroma subblock of the picture based on previously determined motion vector information corresponding to at least one luma subblock of the picture, wherein: In response to the chroma format information indicating that the chroma sub-block is sampled in a 4:4:4 format, the processor is configured to: determine a motion vector corresponding to the chroma sub-block based on motion vector information corresponding to a luma sub-block, In response to the chroma format information indicating that the chroma sub-block is sampled in a 4:2:2 format, the processor is configured to: determine a motion vector corresponding to the chroma sub-block based on averaging motion vector information corresponding to two luminance sub-blocks in one dimension, and In response to the chroma format information indicating that the chroma subblock is sampled in a 4:2:0 format, the processor is configured to: select an upper left luminance subblock and a lower right luminance subblock, and average motion vector information corresponding to the two selected luminance subblocks to determine a motion vector corresponding to the chroma subblock; as well as The picture is reconstructed based on the motion vector information of the at least one luminance sub-block and the motion vector of the chrominance sub-block.
6. The electronic device according to claim 5, wherein: When the chroma sub-block is sampled in a 4:2:0 format, in order to average the motion vector information corresponding to the two selected luma sub-blocks, the processor is configured to: Symmetric rounding is performed on the motion vector information corresponding to the two selected luma sub-blocks.
7. The electronic device according to claim 5, wherein: In response to the chroma format information indicating that the chroma sub-block is sampled in a 4:2:2 format or a 4:2:0 format, the processor is configured to: Based on the chroma format information, deriving a subWidthC chroma format parameter and a subHeightC chroma format parameter; selecting two luma sub-blocks based on the subWidthC chroma format parameter and the subHeightC chroma format parameter; as well as The motion vector information corresponding to the two selected luma sub-blocks is averaged.
8. The electronic device according to claim 5, wherein the processor is further configured to: Based on the chroma format information, deriving a subWidthC chroma format parameter and a subHeightC chroma format parameter; and When the subWidthC chroma format parameter and the subHeightC chroma format parameter are different values, it is determined that the chroma sub-block is sampled in a 4:2:2 format.
9. A method for encoding a picture into a data stream, the method comprising: encoding into the data stream chroma format information describing how the chroma of the picture is sampled; Determining a motion vector corresponding to a chroma subblock of the picture based on previously determined motion vector information corresponding to at least one luma subblock of the picture, wherein: In response to the chroma format information indicating that the chroma sub-block is sampled in a 4:4:4 format, determining a motion vector corresponding to the chroma sub-block based on motion vector information corresponding to a luma sub-block, In response to the chroma format information indicating that the chroma sub-block is sampled in a 4:2:2 format, determining a motion vector corresponding to the chroma sub-block based on averaging motion vector information corresponding to two luminance sub-blocks in one dimension, and In response to the chroma format information indicating that the chroma subblock is sampled in a 4:2:0 format: selecting an upper left luminance subblock and a lower right luminance subblock, and averaging motion vector information corresponding to the two selected luminance subblocks to determine a motion vector corresponding to the chroma subblock; as well as The picture is reconstructed based on the motion vector information of the at least one luminance sub-block and the motion vector of the chrominance sub-block.
10. An electronic device for encoding a picture into a data stream, the electronic device comprising: A processor, the processor being configured to: encoding into the data stream chroma format information describing how the chroma of the picture is sampled; Determining a motion vector corresponding to a chroma subblock of the picture based on previously determined motion vector information corresponding to at least one luma subblock of the picture, wherein: In response to the chroma format information indicating that the chroma sub-block is sampled in a 4:4:4 format, determining a motion vector corresponding to the chroma sub-block based on motion vector information corresponding to a luma sub-block, In response to the chroma format information indicating that the chroma sub-block is sampled in a 4:2:2 format, determining a motion vector corresponding to the chroma sub-block based on averaging motion vector information corresponding to two luminance sub-blocks in one dimension, and In response to the chroma format information indicating that the chroma subblock is sampled in a 4:2:0 format: selecting an upper left luminance subblock and a lower right luminance subblock, and averaging motion vector information corresponding to the two selected luminance subblocks to determine a motion vector corresponding to the chroma subblock; as well as The picture is reconstructed based on the motion vector information of the at least one luminance sub-block and the motion vector of the chrominance sub-block.
11. A computer program for implementing the method according to claim 1 or 9 when executed on a computer or a signal processor.
12. A system comprising: The video encoder according to claim 10, and The video decoder according to any one of claims 1 to 4, wherein the video encoder is configured to generate an encoded video signal, and The video decoder is configured to decode the encoded video signal to reconstruct a picture of the video.
