Encoding / decoding video images partitioned in CTU grid

Abstract

The present application relates to encoding / decoding a video image into / from a bitstream of encoded video image data. The coding method comprises: obtaining a coding tree unit grid to segment at least one component of a video image into coding tree units, each coding tree unit being an image region subdivided according to a coding tree; determining at least one shift offset by aligning at least one boundary of the coding tree unit grid with at least one boundary separating an image region having low spatial activity from an image region having high spatial activity of the video image; shifting the coding tree unit grid according to at least one shift offset; obtaining encoded video image data by encoding at least one coding unit (CU) of a coding tree associated with each coding tree unit of the shifted coding tree unit grid; and writing the encoded video data to the bitstream.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of European Patent Application No. 22305406.5 filed with the European Patent Office on March 31, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application generally relates to video image encoding and decoding. In particular, but not exclusively, the technical field of the present application relates to partitioning a video image in a CTU grid for encoding video image data into a bitstream or decoding video image data from a bitstream. Background Art

[0004] This section is intended to introduce the reader to various aspects of the art that may be related to various aspects of at least one exemplary embodiment of the present application described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of various aspects of the present application. Therefore, it should be understood that these statements should be read in this light, and not as admissions of prior art.

[0005] In state-of-the-art video compression systems such as HEVC (ISO / IEC 23008-2 High Efficiency Video Coding, ITU-T Recommendation H.265, https: / / www.itu.int / rec / T-REC-H.265-202108-P / en) or VVC (ISO / IEC 23090-3 Generic Video Coding, ITU-T Recommendation H.266, https: / / www.itu.int / rec / T-REC-H.266-202008-I / en), low-level and high-level picture partitioning are provided to divide a video picture into picture regions, so-called coding tree units (CTUs), which for HEVC typically have a size between 8×8 and 64×64 pixels, while for VVC the size of the coding tree unit (CTU) is 32×32, 64×64 or 128×128 pixels.

[0006] The CTU partitioning of the video image VP (including the image area equal to the entire video image area) forms a grid of CTUs of a fixed size, i.e., a CTU grid, in which the upper and left borders coincide spatially with the upper and left borders of the video image VP, such as Figure 1Grey shaded rectangles represent image areas associated with whole CTUs of the CTU grid, i.e. CTUs with the same width and height, and white shaded rectangles represent partial (incomplete) CTUs, i.e. CTUs with a width lower than the width of the whole CTU and / or a height lower than the height of the whole CTU. Figure 1 A 2D coordinate system (x,y) with an origin (0,0) is also shown to indicate that any spatial location of a substructure of the CTU grid is defined by the origin.

[0007] For example, Figure 2 As shown, the spatial position of the upper left corner of the CTU of the CTU grid is determined according to the CTU address ctuAddr, which defines the spatial position of the upper left corner of the CTU from the origin.

[0008] A coding tree is associated with each CTU to determine the tree partitioning of the CTU.

[0009] like Figure 2 As shown, in HEVC, the coding tree is a quadtree partition of a CTU, where each leaf is called a coding unit (CU). The spatial position of a CU in a video image VP is defined by a CU index cuIdx, which defines the spatial position from the origin. The CU is spatially partitioned into one or more prediction units (PUs). The spatial position of a PU in a video image VP is defined by a PU index puIdx, which defines the spatial position from the upper left corner of the CTU, and the spatial position of an element of a partitioned PU can be defined by a PU partition index puPartIdx, which defines the spatial position from the upper left corner of the PU. Each PU is assigned some intra-frame or inter-frame prediction parameters (prediction information). Intra-frame or inter-frame coding modes are assigned at the CU level. The CU can also be spatially partitioned into one or more transform units (TUs). The spatial position of a TU in a video image is defined by a TU index tuIdx, which defines the spatial position from the upper left corner of the PU. Each TU is assigned some transform parameters (prediction information). The transform mode is assigned at the PU level, and 2D individual transform is performed at the TU level during encoding or decoding of an image block.

[0010] The PU partition types in HEVC are as follows: Figure 3As shown. They include square partitions (2N×2N and N×N), which are the only partitions used in both intra-CU and inter-CU, symmetric non-square partitions (2N×N, N×2N, used only in inter-CU), and asymmetric partitions (used only in inter-CU). For example, PU type 2N×nU represents an asymmetric horizontal partition of a PU, where the smaller partition is located at the top of the PU. According to another example, PU type 2N×nL represents an asymmetric horizontal partition of a PU, where the smaller partition is located at the top of the PU.

[0011] like Figure 4 As shown, in VVC, the coding tree starts from the root node (i.e., CTU). Next, a quadtree (or quadtree) partitioning divides the root node into 4 nodes (solid lines) corresponding to 4 sub-blocks of equal size. Next, the quadtree (or quadtree) leaves can then be further partitioned by the so-called multi-type tree, which involves Figure 5 Binary or ternary segmentation of one of the 4 segmentation patterns shown. These segmentation types are vertical and horizontal binary segmentation patterns, denoted SBTV and SBTH, and vertical and horizontal ternary segmentation patterns SPTTV and STTH.

[0012] In the case of joint coding shared by luma and chroma components, the leaves of the coding tree of a CTU are CUs.

[0013] In intra video images (i.e., video images that are encoded without inter prediction), separate coding trees may be used for the luma component on one side and the chroma component on the other side. The luma component portion of a CTU is referred to as a luma coding tree block (luma CTB). The luma CTB is then associated with a coding tree, and the leaves of the coding tree are associated with luma coding blocks. Furthermore, in the case of separate luma / chroma coding trees, in the case of a 3-component image, the two chroma component portions of a CTU share the same coding tree, which is referred to as a chroma coding tree block (chroma CTB). The chroma CTB is then associated with a coding tree, and the leaves of the coding tree are associated with chroma coding blocks.

[0014] Contrary to HEVC, in VVC, in most cases, CU, PU and TU have equal size, which means that CU is generally not partitioned into PU or TU except in some specific coding modes.

[0015] Compared with HEVC, VVC's block structure has greater flexibility, as the CU can be square or rectangular, and many rectangular block shapes are allowed. This greater flexibility in the coding structure improves compression efficiency by about 15%, that is, the average bit rate is reduced by 15% at the same image quality.

[0016] Figure 6 An example of coding parameters for encoding tree partitions according to VVC configurations is schematically shown.

[0017] The coding parameter CTU size refers to the size of the root node of a quadtree (or quad tree). The coding parameter CtbSizeY refers to the size of the luma CTB. The coding parameter MinCbSizeY refers to the minimum allowable coded block size for luma (Y). The coding parameter MinQTSize refers to the minimum allowable quadtree leaf size, the coding parameter MaxBtSize refers to the maximum allowable binary tree root node size, the coding parameter MaxTtSize refers to the maximum allowable ternary tree root node size, the coding parameter MaxMttDepth refers to the maximum allowable hierarchical depth of the multi-type tree split from the quadtree leaf, the coding parameter MinBtSize refers to the minimum allowable binary tree leaf size, and the coding parameter MinTtSize refers to the minimum allowable ternary tree leaf size.

[0018] The size of a VVC-encoded video image is a multiple of 8. Thus, the video image size may not be present in a complete set of CTUs, i.e., the video image size may not be a multiple of 32, 64, or 128. In such cases, some partial CTUs may be present at the bottom or right side of the video image. Some block partitioning rules are typically used to handle the block partitioning of partial CTUs of the CTU grid, if any, that are spatially located on the right and bottom video image borders ( Figure 1 the white shaded rectangles in).

[0019] In an exemplary embodiment of the block partitioning rule, if a tree node block (the block associated with the tree node) extends beyond the bottom and right image boundaries ( Figure 7 ), if the block is associated with a quadtree (QT) node and the size of the block is greater than MinQTSize, then the QT split mode (dashed line) can be forced, otherwise, the SBTH split mode ( Figure 8 ) can be forced.

[0020] If a tree node block extends beyond the bottom or right video image border, if the block is a quadtree (QT) node, the block size > MinQTSize, and the block size > MaxBtSize, then QT split may be forced ( Fig. 9 ). Otherwise, if the block is a QT node, and the block size > MinQTSize and the block size <= MaxBtSize, then split may be forced, and the encoder may choose between QT or binary tree split with forced boundary orientation (SBTH or SBTV) ( Fig.10 ). Otherwise (the block is a multi-type tree or the size < MinQTSize), binary tree split may be forced in the image boundary orientation ( Fig.11).

[0021] In VVC, advanced image partitioning is also provided, where basically a video image can be partitioned into sub-images, slices and tiles.

[0022] A tile is a series of CTUs covering a rectangular image area of ​​a video image. The CTUs in a tile are usually scanned in a raster scan order within the tile.

[0023] A slice consists of an integer number of tiles or an integer number of consecutive complete CTU rows within a tile. Therefore, each vertical slice boundary is always a vertical tile boundary. The horizontal boundaries of a slice may not be tile boundaries, but instead consist of horizontal CTU boundaries within a tile; this occurs when a tile is split into multiple rectangular slices, each of which consists of an integer number of consecutive complete CTU rows within the tile.

[0024] Two slice modes are supported, namely raster scan slice mode and rectangular slice mode. In raster scan slice mode, a slice contains a series of complete tiles in the tile raster scan order of the video image. In rectangular slice mode, a slice contains multiple complete tiles that together form a rectangular image area, or contains several consecutive complete CTU rows of a tile that together form a rectangular image area. The tiles in a rectangular slice are scanned in the rectangular image area corresponding to the slice in the tile raster scan order.

[0025] A sub-image consists of one or more slices that together cover a rectangular image area. Therefore, every sub-image boundary is always also a slice boundary, and every vertical sub-image boundary is always also a vertical tile boundary.

[0026] Each sub-image and tile should satisfy one or both of the following conditions:

[0027] All CTUs in a sub-image belong to the same tile.

[0028] All CTUs in a tile belong to the same sub-image.

[0029] Fig.12 An example of a video image partitioned with an 18x12 CTU grid, 12 tiles of 6x3 CTUs each, and 3 raster scan slices (2 white shades and 1 grey shade) in raster scan slice mode is shown.

[0030] Fig.13 An example of a video image partitioned with an 18×12 CTU grid, 24 tiles (6 tile columns of 3 CTUs wide and 4 tile rows of 3 CTUs high) and 9 rectangular slices (4 shades of white and 5 shades of grey) is shown.

[0031] Fig.14An example of a video image partitioned with an 18x12 CTU grid, 4 tiles (2 tile columns of 9x6 CTU each) and 4 rectangular slices (2 white shades and 2 grey shades) is shown.

[0032] Fig.15 An example of a sub-image partitioned with a 20×12 CTU grid, 18 tiles (12 tiles, each covering a slice of 4×4 CTU, 6 tiles, each covering 2 vertically stacked slices of 2×2 CTU), resulting in a total of 24 slices and 24 sub-images of different sizes (each slice is a sub-image).

[0033] As discussed above, in VVC and HEVC, the CTU sizes (CTU width CTU DW and CTU height CTU DH) of all CTUs of the CTU grid are equal to the same default CTU size of all CTUs of the CTU grid. In addition, the CTU grid is aligned with the top and left borders of the video picture VP ( Figure 1 ). Therefore, the CTU partitioned into CUs is designed to optimize the coding of the video picture VP by adapting the CU size and by aligning the CU with the spatial structure of the video picture VP, i.e., maximizing the CU size in areas with homogenous content in terms of spatial activity. The goal is to optimize coding efficiency, which is usually achieved by means of large CUs covering low spatial activity image areas and smaller CUs covering high spatial activity image areas.

[0034] Because the CTUs of the CTU grid all have the same fixed width and height, some CTUs may cover high spatial activity image areas. Partitioning such CTUs into CUs results in defining a large number of CUs with small sizes. This leads to a potential lack of compression efficiency, since it is usually more efficient to compress low activity image areas with the largest possible CUs (blocks).

[0035] There is a need to increase the flexibility of the representation of the CTU grid in order to improve the compression efficiency of video images without increasing the number of CUs resulting from partitioning of CTUs of the CTU grid.

[0036] At least one exemplary embodiment of the present application has been designed in view of the foregoing. Summary of the invention

[0037] The following section presents a simplified summary of at least one exemplary embodiment in order to provide a basic understanding of some aspects of the present application. This summary is not an exhaustive overview of the exemplary embodiments. It is not intended to identify the key or important elements of the exemplary embodiments. The following summary only presents some aspects of at least one exemplary embodiment in a simplified form as a prelude to a more detailed description provided elsewhere in the document.

[0038] According to a first aspect of the present application, a method for encoding a video image into a bitstream of encoded video image data is provided. The method includes obtaining a coding tree unit grid to divide at least one component of the video image into coding tree units, each coding tree unit being an image region subdivided according to a coding tree; determining at least one shift offset by aligning at least one boundary of the coding tree unit grid with at least one boundary separating an image region with low spatial activity from an image region with high spatial activity of the video image; shifting the coding tree unit grid according to at least one shift offset; obtaining encoded video image data by encoding at least one coding unit of a coding tree associated with each coding tree unit of the shifted coding tree unit grid; and writing the encoded video data into a bitstream.

[0039] In an exemplary embodiment, the spatial activity of a video image is determined by detecting vertical edges and horizontal edges from the video image content, and at least one boundary separating image regions of the video image having low spatial activity from image regions having high spatial activity is a detected vertical edge or horizontal edge.

[0040] In an exemplary embodiment, at least one shift offset is determined by minimizing a spatial distance between a boundary of the coding tree unit grid and a detected vertical edge or horizontal edge.

[0041] According to a second aspect of the present application, a method for decoding a video image from a bitstream of decoded video image data is provided. The method comprises determining a coding tree unit grid representing a spatial partition of the video image, each coding tree unit representing an image region of the video image subdivided according to a coding tree; shifting the coding tree unit grid according to at least one shift offset; obtaining video image data by decoding at least one coding unit of a coding tree associated with each coding tree unit of the shifted coding tree unit grid from the bitstream; and obtaining a video image from the video image data.

[0042] In an exemplary embodiment, the method further comprises signaling into the bitstream a syntax element indicating whether the coding tree unit grid is vertically shifted, horizontally shifted, or both.

[0043] In an exemplary embodiment, the method further comprises signaling into the bitstream shift information data comprising binary data indicating whether at least one shift offset is present in the bitstream, and if the shift information data indicates that at least one shift offset is present in the bitstream, signaling the at least one shift offset into the bitstream.

[0044] In an exemplary embodiment, the shift information data is signaled into a syntax element comprising a sequence parameter set, a picture parameter set or a picture header.

[0045] In an exemplary embodiment, the video image includes a luma component, the luma component includes a set of luma samples, and at least one shift offset is an integer multiple of a maximum between 8 and a minimum coding block size of luma samples.

[0046] In an exemplary embodiment, at least one shift offset is an integer multiple of eight, or is indexed by a multiple of eight.

[0047] In an exemplary embodiment, at least one shift offset is smaller than a size of a fractional coding tree unit size.

[0048] According to a third aspect of the present application, there is provided a bit stream of an encoded video image generated by one of the methods according to the first aspect of the present application.

[0049] According to a fourth aspect of the present application, a device for encoding a video image into a bit stream of encoded video image data is provided. The device comprises means for executing one of the methods according to the first aspect of the present application.

[0050] According to a fifth aspect of the present application, a device for decoding a video image from a bit stream of encoded video image data is provided, wherein the device comprises a device for executing one of the methods according to the second aspect of the present application.

[0051] According to a sixth aspect of the present application, a computer program product is provided, comprising instructions, and when the program is executed by one or more processors, the instructions cause the one or more processors to execute the method according to the first aspect of the present application.

[0052] According to a seventh aspect of the present application, a non-transitory storage medium is provided, carrying program code instructions for executing the method according to the first aspect of the present application.

[0053] According to an eighth aspect of the present application, a computer program product is provided, comprising instructions, and when the program is executed by one or more processors, the instructions cause the one or more processors to execute the method according to the second aspect of the present application.

[0054] According to a ninth aspect of the present application, a non-transitory storage medium is provided, carrying program code instructions for executing the method according to the second aspect of the present application.

[0055] Specific properties of at least one embodiment of the exemplary embodiments and other objects, advantages, features and uses of at least one embodiment of the exemplary embodiments will become apparent from the following description of the examples in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Reference will now be made, by way of example, to the accompanying drawings which show exemplary embodiments of the present application, and in which:

[0057] Figure 1 shows an example of a CTU grid aligned with the upper left corner of a video image according to the prior art;

[0058] Figure 2 An example of a coding tree unit according to HEVC is shown;

[0059] Figure 3 An example of partitioning a coding unit into prediction units according to HEVC is shown;

[0060] Figure 4 An example of CTU partitioning according to VVC is shown;

[0061] Figure 5 An example of supported partitioning modes in multi-type tree partitioning according to VVC is shown;

[0062] Figure 6 An example of coding parameters for configuring a coding tree representation of a CTU according to a VVC is shown;

[0063] Figure 7 An example illustrating rules for partitioning a video image border is shown;

[0064] Figure 8 An example illustrating rules for partitioning a video image border is shown;

[0065] Fig. 9 An example illustrating rules for partitioning a video image border is shown;

[0066] Fig.10 An example illustrating rules for partitioning a video image border is shown;

[0067] Fig.11 An example illustrating rules for partitioning a video image border is shown;

[0068] Fig.12 An example of partitioning a video image according to a raster scan slicing pattern is shown;

[0069] Fig.13 An example of partitioning a video image according to a rectangular slice pattern is shown;

[0070] Fig.14 An example of a video image partitioned into tiles and rectangular slices is shown;

[0071] Fig.15An example of sub-image partitioning of a video image is shown;

[0072] Fig.16 A schematic block diagram showing steps of a method 100 for encoding a video image VP according to at least one exemplary embodiment;

[0073] Fig.17 A schematic block diagram showing steps of a method 200 for decoding a video image VP according to at least one exemplary embodiment;

[0074] Fig.18 shows a schematic block diagram of step 110 according to at least one exemplary embodiment;

[0075] Fig.19 shows an example of a diagram of a CTU grid shift according to an exemplary embodiment;

[0076] Fig. 20 shows an example of a diagram of a CTU grid shift according to an exemplary embodiment;

[0077] Fig.21 shows an example of a diagram of a CTU grid shift according to an exemplary embodiment;

[0078] Fig. 22 shows an example of a diagram of a CTU grid shift according to an exemplary embodiment;

[0079] Fig.23 An example of an edge detection filter according to at least one exemplary embodiment is shown;

[0080] Fig.24 An example of a syntax element seq_parameter_set-rbsp according to at least one exemplary embodiment is shown;

[0081] Fig.25 An example of a syntax element coding_tree_unit according to at least one exemplary embodiment is shown;

[0082] Fig.26 A schematic block diagram showing step 210 according to at least one exemplary embodiment; and

[0083] Fig. 27 A schematic block diagram of an example of a system in which various aspects and exemplary embodiments are implemented is illustrated.

[0084] Similar or identical elements are referred to by the same reference numerals. DETAILED DESCRIPTION

[0085] At least one of the exemplary embodiments will be described more fully below with reference to the accompanying drawings, wherein an example of at least one of the exemplary embodiments is depicted. However, the exemplary embodiments may be implemented in many alternative forms and should not be construed as being limited to the examples set forth herein. Thus, it should be understood that the exemplary embodiments are not intended to be limited to the specific forms disclosed. On the contrary, the present disclosure is intended to cover all modifications, equivalents, and alternatives that fall within the spirit and scope of the present application.

[0086] At least one of these aspects generally relates to video image encoding and decoding, another aspect generally relates to transmitting provided or encoded bitstreams, and another aspect relates to receiving / accessing decoded bitstreams.

[0087] At least one of these exemplary embodiments is described for encoding / decoding video images, but is extended to encoding / decoding video images (image sequences) in that each video image is sequentially encoded / decoded as described below.

[0088] In addition, the at least one exemplary embodiment is not limited to MPEG standards such as HEVC, VVC, AVC (ISO / IEC 14496-10, Advanced Video Coding for generic audio-visual services, ITU-T Recommendation H.264, https: / / www.itu.int / rec / T-REC-H.264-202108-P / en), EVC (ISO / IEC 23094-1 Basic Video Coding), but can be applied to other standards and recommendations such as AV1 (AOMedia Video 1, http: / / aomedia.org / av1 / specification / ). The at least one exemplary embodiment can be applicable to existing or future developments and extensions of any such standards and recommendations. Unless otherwise specified or technically excluded, the aspects described in the present application can be used alone or in combination.

[0089] A pixel corresponds to the smallest display unit on the screen, which can be composed of one or more light sources (1 for a monochrome screen and 3 or more for a color screen).

[0090] A video image (also referred to as a frame or image frame) comprises at least one component determined by a specific image / video format, which specifies all information related to pixel values ​​and all information that can be used by a display unit and / or any other device to display and / or decode video image data related to the video image.

[0091] A video image comprises at least one component which is usually represented in the form of an array of samples.

[0092] A monochrome video image includes a single component, while a color video image may include three components.

[0093] For example, a color video image may include a luminance (or luminance) component and two chrominance components when the image / video format is the well-known (Y,Cb,Cr) format, or may include three color components (one for red, one for green, and one for blue) when the image / video format is the well-known (R,G,B) format.

[0094] Each component of the video image comprises the same number of samples, which is equal to the number of pixels of the screen on which the video image is intended to be displayed.

[0095] Alternatively, in case the video format comprises a luma component and two chroma components, such as the (Y, Cb, Cr) format, the chroma components may contain half the number of samples in width and / or height relative to the luma component, depending on the color format considered.

[0096] A sample is the smallest visual information unit that makes up the components of a video image. A sample value can be, for example, a brightness or chrominance value or a color value in (R, G, B) format.

[0097] The pixel value is the value of a screen pixel. For a monochrome video image, a pixel value can be represented by one sample, and for a color video image, a pixel value can be represented by multiple co-located samples. The co-located sample associated with a pixel means a sample corresponding to the position of the pixel in the screen.

[0098] A video image is usually considered as a set of pixel values, with each pixel represented by at least one sample.

[0099] A block of a video image (i.e., an image block) is a group of samples of a component of a video image. When the image / video format is the well-known (Y, Cb, Cr) format, a block of at least one luminance sample or a block of at least one chrominance sample may be considered, or when the image / video format is the well-known (R, G, B) format, a block of at least one color sample may be considered.

[0100] At least one exemplary embodiment is not limited to a particular image / video format.

[0101] In general, the present application relates to encoding a video image into / decoding a video image from a bitstream of encoded video image data. On the encoding side, a CTU grid is determined to partition at least one component of the video image into CTUs. At least one shift offset, i.e., an offset value of a CTU for shifting the CTU grid, is determined by aligning at least one boundary of the CTU grid with at least one boundary separating an image region with low spatial activity from an image region with high spatial activity of the video image. The CTU grid is then shifted according to the at least one shift offset. The encoded video image data is obtained by encoding at least one coding unit of a coding tree associated with each coding tree unit of the shifted CTU grid, and the encoded video data is written into a bitstream.

[0102] At the decoding side, a CTU grid representing a spatial partition of at least one component of a video image is determined and shifted according to at least one shift offset, video image data is obtained by decoding at least one coding unit of a coding tree associated with each coding tree unit of the shifted CTU grid from the bitstream, and a video image is obtained from the video image data.

[0103] The present disclosure allows shifting the CTU grid according to at least one shift offset. This increases the flexibility of the representation of the CTU grid, since the shift offset can be determined to align the boundaries of the CTU grid with the main spatial structure of the video image or components of the video image. Shifting the CTU grid results in the CTU covering a homogeneous spatially active image area, i.e., an image area that is homogeneous in terms of spatial structure, and results in a limited number of CUs resulting from the CTU partitioning. Therefore, the CUs resulting from such a CTU partitioning may cover a homogeneous spatially active image area.

[0104] Therefore, shifting the CTU grid during picture partitioning improves the compression efficiency of the video picture.

[0105] Furthermore, shifting the CTU grid does not significantly increase the complexity of both the encoder and the decoder, and does not incur significant additional cost when signaling the shift offset into the bitstream of the coded video data.

[0106] Fig.16 and 17 An overview of video encoding / decoding methods used in current video standard compression systems (such as, for example, VVC) is provided.

[0107] Fig.16 A schematic block diagram showing the steps of a method 100 for encoding a video image VP according to the prior art is shown.

[0108] In step 110, the video picture VP is partitioned into a CTU grid and the partition information data is signaled into the bitstream B. A coding tree is associated with each CTU of the CTU grid, and each CU of the coding tree associated with each CTU is a block of samples of the video picture VP. In short, a CU of a CTU is a block.

[0109] The CTUs of the CTU grid are considered in a scan order (usually the raster scan order of a video image). Each block of the CTU is also considered in a scan order (usually the raster scan order of the CTU block).

[0110] Each block of each CTU is then encoded using intra or inter prediction coding mode.

[0111] Intra prediction (step 120) consists in predicting the current block with the help of a prediction block based on already coded, decoded and reconstructed samples located around the current block (usually located at the top and to the left of the current block).Intra prediction is performed in the spatial domain.

[0112] In inter prediction mode, motion estimation (step 130) and motion compensation (135) are performed. Motion estimation searches for candidate reference blocks that are good predictors of the current block in one or more reference video images used to predictively encode the current video image. For example, a good predictor of the current block is a predictor that is similar to the current block. The output of the motion estimation step 130 is one or more motion vectors and one or more reference image indices associated with the current block. Next, motion compensation (step 135) obtains a prediction block with the help of the one or more motion vectors and one or more reference image indices determined by the motion estimation step 130. Basically, the block belonging to the selected reference image and pointed to by the motion vector can be used as the prediction block for the current block. In addition, since the motion vector is represented as a fraction of an integer pixel position (known as a sub-pixel precision motion vector representation), motion compensation usually involves spatial interpolation of some reconstructed samples of the reference image to calculate the prediction block samples.

[0113] The prediction information data is signaled into the bitstream B. The prediction information may include prediction mode, prediction information coding mode, intra prediction mode or motion vector(s) and reference picture index(es) and any other information for obtaining the same prediction block at the decoding side.

[0114] The method 100 optimizes the rate-distortion tradeoff to select one of the intra or inter coding modes by considering, for example, the encoding of a prediction residual block calculated by subtracting a candidate prediction block from a current block, and the signaling of prediction information data required to determine the candidate prediction block at the decoding side.

[0115] Typically, the best prediction mode is given as the prediction mode of the best coding mode p* for the current block given by the following equation:

[0116] p * =Argmin p∈P {RD cost (p)} (1)

[0117] Where P is the set of all candidate coding modes for the current block, p represents the candidate coding mode in the set, and RD cost (p) is the rate-distortion cost of candidate coding mode p, usually expressed as:

[0118] RD cost(p) =D(p)+λ.R(p).

[0119] D(p) is the distortion between the current block and the reconstructed block obtained after encoding / decoding the current block with candidate coding mode p, R(p) is the rate cost associated with encoding the current block with coding mode p, and λ is a Lagrangian parameter that represents the rate constraint for encoding the current block and is typically calculated based on the quantization parameter used to encode the current block.

[0120] The current block is usually encoded according to a prediction residual block PR. More precisely, the prediction residual block PR is calculated, for example, by subtracting the best prediction block from the current block. The prediction residual block PR is then transformed (step 140) by using, for example, a DCT (discrete cosine transform) or DST (discrete sine transform) type transform, and the obtained transform coefficient block is quantized (step 150).

[0121] In a variant, the method 100 may also skip the transform step 140 and apply quantization directly to the prediction residual block PR according to a so-called transform skip coding mode.

[0122] The block of quantized transform coefficients (or the block of quantized prediction residuals) is entropy encoded into the bitstream B (step 160).

[0123] Next, the quantized transform coefficient block (or quantized residual block) is dequantized (step 170) and inverse transformed (180) (or not inverse transformed), producing a decoded prediction residual block. The decoded prediction residual block and the predicted block are then combined, usually summed, which provides a reconstructed block.

[0124] Other information data may also be entropy encoded to encode the current block of the video picture VP.

[0125] An in-loop filter (step 190) may be applied to the reconstructed image (including the reconstructed blocks) to reduce compression artifacts. After all image blocks are reconstructed, a loop filter may be applied. For example, they include a deblocking filter, a sample adaptive offset (SAO), or an adaptive loop filter.

[0126] The reconstructed block or filtered reconstructed block forms a reference picture which can be stored into a decoded picture buffer (DPB) so that it can be used as a reference picture for encoding of the next current block of a video picture VP or the next video picture to be encoded.

[0127] Fig.17 A schematic block diagram showing the steps of a method 200 for decoding a video image VP according to the prior art is shown.

[0128] In step 210, partition information data, prediction information data and a quantized transform coefficient block (or a quantized residual block) are obtained by entropy decoding the bitstream B.

[0129] The partition information data defines a CTU grid (layout) on the video picture. The CTU grid divides the video picture VP into a plurality of CTUs. The CTUs of the CTU grid are considered in a scan order (usually a raster scan order of the video picture VP). The blocks of the considered CTUs are also considered in a scan order (usually a raster scan order of the blocks of the CTU).

[0130] Other information data may also be decoded from the bitstream B for decoding a current block of a current CTU of the CTU grid.

[0131] At step 220, entropy decoding is performed on each current block of the current CTU.

[0132] Each decoded current block may be a quantized transform coefficient block or a quantized prediction residual block.

[0133] In step 230, the current block (of the current CTU) is dequantized and possibly inverse transformed (step 240) to obtain a decoded prediction residual block.

[0134] On the other hand, the prediction information data is used to predict the current block. The prediction block is obtained by its intra prediction (step 250) or its motion compensated temporal prediction (step 260). The prediction process performed on the decoding side is the same as the prediction process performed on the encoding side.

[0135] Next, the decoded prediction residual block and the prediction block are then combined, typically summed, which provides the reconstructed block.

[0136] In step 270, the in-loop filter may be applied to the reconstructed image (including the reconstructed block), and the reconstructed block or the filtered reconstructed block forms a reference image, which may be stored in a decoded picture buffer (DPB), as discussed above ( Fig.16 ).

[0137] exist Fig.18 In an exemplary embodiment of step 110 illustrated, a CTU grid may be determined to partition at least one component of the video image VP into CTUs in step 111. For example, the number of columns of the CTU grid is given by dividing the width of the component of the video image VP by the CTU width CTU DW, and the number of rows of the CTU grid is given by dividing the height of the component of the video image VP by the CTU height CTU DH.

[0138] For example, a CTU grid may be defined for each component of the video picture VP. It is also possible to determine a single CTU grid for the video picture VP.

[0139] In step 112, at least one shift offset is determined by aligning at least one boundary of the CTU grid with at least one boundary separating image regions of the video image having low spatial activity from image regions having high spatial activity.

[0140] As a variant, if a CTB grid for luma components of a video picture VP is different from a CTU grid for chroma components of the video picture VP, the CTB grid for the luma components may be shifted according to at least one first shift offset SO1, while the CTB grid for the chroma components may be shifted according to at least one second shift offset SO2 different from the at least one first shift offset SO1.

[0141] In step 113, the CTU grid is shifted according to the at least one shift offset S8. Shifting the CTU grid means shifting each CTU of the CTU grid.

[0142] In an exemplary embodiment of step 113, the CTUs of the CTU grid are shifted by a horizontal shift offset x-S0.

[0143] For example, Fig.19 As illustrated, the horizontal shift offset x-SO is determined to align the top rightmost CTU of the CTU grid with the top right corner of the video picture VP.The partial CTU then appears only along the left border and / or bottom border of the video picture VP.

[0144] In an exemplary embodiment of step 113, the CTUs of the CTU grid are shifted by a vertical shift offset y-S0.

[0145] For example, Fig. 20 As illustrated, the vertical shift offset y-SO is determined to align the bottom leftmost CTU of the CTU grid with the bottom left corner of the video picture VP.The partial CTU then appears only along the right border and / or top border of the video picture VP.

[0146] In an exemplary embodiment of step 113, the CTUs of the CTU grid are shifted by a vertical shift offset y-S0 and a horizontal shift offset x-S0.

[0147] For example, Fig.21 As shown, the vertical shift offset y-SO and the horizontal shift offset x-SO are determined to align the bottom right CTU of the CTU grid with the bottom right corner of the video picture VP. Then, the partial CTU only appears along the left border and / or top border of the video picture VP.

[0148] Fig. 22 Another example of a shifted CTU grid is illustrated, where partial CTUs appear along the left, right, top and bottom borders of a video picture VP.

[0149] In an exemplary embodiment of step 112, the spatial activity of the video image VP can be determined by detecting vertical edges and horizontal edges from the video image content, and at least one boundary separating an image area of ​​the video image with low spatial activity from an image area with high spatial activity is a detected vertical edge or horizontal edge.

[0150] A vertical edge or horizontal edge is a spacing between two groups of samples of a component of a video image VP, each group of samples sharing a specific image property. A vertical edge or horizontal edge also refers to an approximately vertical or horizontal edge, i.e. an edge that makes an angle of about + / - 15 degrees with a vertical or horizontal line.

[0151] An image region with low spatial activity means an image region without (or with few) vertical edges or horizontal edges, whereas an image region with high spatial activity is an image region with more horizontal and vertical edges than an image region with low spatial activity.

[0152] For example, each pixel of a video image may be classified as an edge sample or a non-edge sample, and if the number of edge samples of the image region is below a threshold, the image region is considered to have low spatial activity.

[0153] In an exemplary embodiment of step 112, the vertical and horizontal edges of the video image may be detected by applying a horizontal Sobel filter and a vertical Sobel filter to the video image VP. These Sobel filters have very low complexity and are well suited for tasks such as detecting horizontal and vertical edges from image content.

[0154] Fig.23 An example of coefficients of the horizontal Sobel filter Sh and an example of coefficients of the vertical Sobel filter Sv are shown.

[0155] The present disclosure is not limited to a specific edge detection, but extends to any other well-known method for detecting edges in an image, such as for example the well-known Canny filter or any gradient-based edge detection method.

[0156] In a variant, the rear edge detection filter may discard some of the detected edges and determine at least one shift offset SO from the remaining edges recorded as main edges.

[0157] In a variant, the back edge detection filter may discard short edges, possibly with a threshold, which may be based on a size related to the width, height, or CTU size (height and / or width) of the video image.

[0158] This variant is advantageous because only a limited number of vertical edges or horizontal edges are taken into account for determining the at least one shift offset.

[0159] In an exemplary embodiment of step 112, a spatial direction in the vertical or horizontal direction having fewer detected edges is selected and at least one shift offset SO is determined taking into account the main detected edges along this direction.

[0160] Therefore, at least one horizontal shift offset or at least one vertical shift offset is determined to align the CTU grid.

[0161] In an exemplary embodiment of step 112, at least one shift offset SO is determined by minimizing a spatial distance between a boundary of the CTU grid (including a CTU boundary and / or a sub-CTU boundary) and a vertical edge or a horizontal edge.

[0162] In an exemplary embodiment of step 112, at least one horizontal shift offset x-SO is determined by minimizing a spatial distance between a horizontal boundary and a horizontal edge of the CTU grid.

[0163] In an exemplary embodiment of step 112, at least one vertical shift offset y-SO is determined by minimizing a spatial distance between a vertical boundary of the CTU grid (or a vertical boundary of a sub-CTU) and a vertical edge.

[0164] In an exemplary embodiment of step 112, the spatial distance between the boundary of the CTU grid and the vertical edge or the horizontal edge is the length of a line segment perpendicular to the boundary of the CTU grid starting from the boundary segment and intersecting the vertical edge or the horizontal edge.

[0165] In one exemplary embodiment of step 112, a linear regression method is used to minimize the spatial distance.

[0166] For example, at least one horizontal (or vertical) shift offset o pt is obtained by minimizing the nearest horizontal (or vertical) boundary of the CTU grid CTUBound closest (i) and the horizontal (or vertical) detected edge ContentEdge(i) to obtain the squared spatial distance d (or average distance):

[0167]

[0168] Where O is the set of horizontal (or vertical) shift offsets and n is the number of horizontal (or vertical) edges.

[0169] The nearest horizontal (or vertical) boundary of the CTU grid may be the boundary directly above or directly below (or directly to the right or directly to the left).

[0170] In an exemplary embodiment of step 112, at least one shift offset SO is determined for each video image.

[0171] In an exemplary embodiment of step 112, at least one shift offset SO is determined for each video (having the same shift offset for all video images of the video). The at least one shift offset SO is the most representative shift offset of the video, that is, the shift offset most commonly used in the video images of the video or the shift offset that causes the least distortion in the decoded video.

[0172] The present disclosure is not limited to a specific criterion for determining the most representative shift offset SO for a video from among the shift offsets SO determined for each video image of the video.

[0173] In an exemplary embodiment of step 112, the shift offset SO is defined from the upper left corner, the upper right corner, the lower left corner or the lower right corner of the video image VP.

[0174] Preferably, the shift offset SO is defined from the upper left corner of the video image VP.

[0175] In an exemplary embodiment of step 112 , the shift offset SO may be indicated or defined as a block or CU size, for example in units of 8 samples.

[0176] In an exemplary embodiment of step 112 , the shift offset SO may be indicated or defined as a CU partition depth size, such as MinQTSize or MinBtSize.

[0177] In an exemplary embodiment of step 112, the shift offset SO may be indicated or defined as an index of the partition size.

[0178] In an exemplary embodiment of step 112, the shift offset SO may be indicated or defined as an index indicating on which video picture border the CTU grid alignment occurs.

[0179] In an exemplary embodiment of step 110, in step 114, the shift information data is signaled (encoded) into the bitstream B. The shift information data comprises binary data indicating whether at least one shift offset is present in the bitstream B, and if the shift information data indicates that at least one shift offset is present in the bitstream B, the at least one determined shift offset is signaled (encoded) into the bitstream B.

[0180] In a variant, the shift information data is signaled (coded) into the bitstream in a parameter set like a sequence parameter set, a picture parameter set or in a picture header syntax element according to the terminology of VVC or HEVC.

[0181] Fig.24 An example of a syntax element carrying a sequence parameter set (SPS) denoted seq_parameter_set_rbsp is shown. According to the present disclosure, the syntax element seq_parameter_set_rbsp includes shift information data including a flag sps_ctu_shift_flag, and if the syntax element sps_ctu_shift_flag is equal to 1, the syntax elements sps_ctu_left_offset and sps_ctu_top_offset are encoded into the syntax element seq_parameter_set-rbsp.

[0182] The syntax element sps_ctu_shift_flag is equal to 1 to indicate whether at least one shift offset in the SPS follows the next shift offset. The syntax element sps_ctu_shift_flag is equal to 0 to indicate that no shift offset is present in the SPS.

[0183] The syntax element sps_ctu_left_offset (may also be denoted as sps_ctu_top_left_offset_x) or sps_ctu_top_offset (may also be denoted as sps_ctu_top_left_offset_y) specifies the horizontal shift offset or the vertical shift offset.

[0184] For example, the shift offset signaled by the syntax elements sps_ctu_left_offset and sps_ctu_top_offset may be used to shift the first top left CTU of the CTU grid.When the syntax element is equal to 0, the syntax elements sps_ctu_left_offset and sps_ctu_top_offset are inferred to be equal to 0.

[0185] In an exemplary embodiment of step 114 , the shift offsets (syntax elements sps_ctu_left_offset and sps_ctu_top_offset) are still greater than zero.

[0186] In a variant, the shift offsets (syntax elements sps_ctu_left_offset and sps_ctu_top_offset) may be equal to an integer multiple of Max(8, MinCbSizeY).

[0187] The syntax elements sps_ctu_left_offset and sps_ctu_top_offset may be indexed in multiples of Max(8, MinCbSizeY).

[0188] In a variant, the shift offsets (syntax elements sps_ctu_left_offset and sps_ctu_top_offset) may be equal to an integer multiple of eight.

[0189] The syntax elements sps_ctu_left_offset and sps_ctu_top_offset can be indexed with multiples of 8.

[0190] The syntax elements sps_ctu_left_offset and sps_ctu_top_offset may be renamed sps_ctu_left_offset_div8_minus1 and sps_ctu_top_offset_div8_minus1.

[0191] The last two variants ensure that partial CTUs / split CUs at the borders of the video image can be processed by traditional codec core modules (such as transform, quantization, filtering, etc.).

[0192] In an exemplary embodiment of step 114, the shift offset (syntax elements sps_ctu_left_offset and sps_ctu_top_offset) is lower than the size of the fractional CTU size, ie, the syntax element sps_ctu_left_offset is lower than the fractional CTU width and the syntax element sps_ctu_top_offset is lower than the fractional CTU height.

[0193] In an exemplary embodiment of step 114, the syntax element sps_ctu_left_offset is smaller than the value given by pps_pic_width_in_luma_samples-Floor(pps_pic_width_in_luma_samples÷CtbSizeY)*1<<(sps_log2_ctu_size_minus5+5).

[0194] In an exemplary embodiment of step 114, the syntax element sps_ctu_left_offset is smaller than the CTU width, or the syntax element sps_ctu_top_offset is smaller than the CTU height. The CTU width or CTU height may be specified by the syntax element sps_log2_ctu_size_minus5 in the SPS ( Fig.24 ).

[0195] In an exemplary embodiment of step 114, the syntax element sps_ctu_left_offset is smaller than half of the CTU width, or the syntax element sps_ctu_top_offset is smaller than half of the CTU height.

[0196] The syntax element seq_parameter_set_rbsp includes the syntax element sps_chroma_format_idc. When the syntax element sps_chroma_format_idc is not equal to 0, the corresponding chroma CTB is a CTB with picture coordinates (CtbAddrX / SubWidthC, CtbAddrY / SubHeightC), where (CtbAddrX, CtbAddrY) are the luma CTB coordinates of the specified luma sample.

[0197] In an exemplary embodiment of step 114, the syntax elements sps_ctu_right_alignment_flag and sps_bottom_alignment_flag indicate whether the entire CTU grid (i.e., the entire set of CTUs) is aligned on the right, left, top, or bottom picture border. Thus, an entire CTU grid with sps_ctu_right_alignment_flag and sps_bottom_alignment_flag equal to 1 indicates that the entire CTU grid is aligned on the bottom right picture border, which is the same as Fig.21 Match. Fig. 20 Also matches sps_ctu_right_alignment_flag equal to 0 and sps_bottom_alignment_flag equal to 1. Fig.19 Also matches sps_ctu_right_alignment_flag equal to 1 and sps_bottom_alignment_flag equal to 0.

[0198] In an exemplary embodiment, the current block partitioning rules as in VVC may be extended to handle block partitioning of partial CTUs of a CTU grid, which partial CTUs are spatially located at the left border and / or top border of the video picture VP, if any.

[0199] Essentially, the mechanisms and equations present in the VVC specification for indexing and constructing sub-images, tiles, and slices can be reused with slight adjustments.

[0200] As an example, numbers (indices) are assigned to complete partial CTUs sorted in raster scan order. Introducing the shift offset SO does not fundamentally change this CTU numbering, because index 0 can be assigned to the first upper left partial CTU (if any), and consecutive complete CTUs in raster scan order can be indexed as 1, 2, etc.

[0201] As another example, the number of CTU columns per luma CTB PicWidthInCtbsY and the number of CTU rows per luma CTB PicHeightInCtbsY are generally calculated by the following equations:

[0202] PicWidthInCtbsY=Ceil(pps_pic_width_in_luma_samples÷CtbSizeY)

[0203] PicHeightInCtbsY=Ceil(pps_pic_heigth_in_luma_samples÷CtbSizeY)

[0204] Among them, pps_pic_width_in_luma_samples is the number of samples of the width of the video image VP, pps_pic_height_in_luma_samples is the number of samples of the height of the video image VP, and CtbSizeY is the size of the brightness CTB.

[0205] According to the present disclosure, if the syntax element sps_ctu_shift_flag is equal to 1 and the CTU grid is horizontally shifted, the number of CTU columns per luma CTB PicWidthInCtbsY is given by the following equation:

[0206] PicWidthInCtbsY=Ceil((pps_pic_width_in_luma_samples-sps_ctu_left_offset)÷CtbSizeY)+1

[0207] According to the present disclosure, if the syntax element sps_ctu_shift_flag is equal to 1 and the CTU grid is shifted vertically, the number of CTU rows per luma CTB PicHeightInCtbsY is given by the following equation:

[0208] PicHeightInCtbsY=Ceil((pps_pic_heigth_in_luma_samples-sps_ctu_top_offset)÷CtbSizeY)+1

[0209] As another example, according to the present disclosure, the syntax element coding_tree_unit defining the spatial position coordinates (xCtb, yCtb) of the upper left corner of the luma CTB is modified.

[0210] If the syntax element sps_ctu_shift_flag is equal to 1 and the CTU grid is shifted horizontally, the first luma sample (incomplete or partial) of the CTU of the first CTU column is located at abscissa position 0. The first luma sample (complete or full) of the second CTU column is located at abscissa position sps_ctu_left_offset. The first luma sample (complete) of the third CTU column is located at abscissa position sps_ctu_left_offset+CtbSizeY. The first luma sample (complete) of the fourth CTU column is located at abscissa position sps_ctu_left_offset+2*CtbSizeY, etc. The reasoning for the ordinate and sps_ctu_top_offset is the same.

[0211] Fig.25An example of a syntax element coding_tree_unit according to at least one exemplary embodiment is shown.

[0212] The ctbLeftOffset and ctbTopOffset variables define the shift offset of the CTB (luminance or chrominance) and are given by the following equations:

[0213] ctbLeftOffset=sps_ctu_shift_flag? sps_ctu_left_offset:CtbSizeY

[0214] ctbTopOffset=sps_ctu_shift_flag? sps_ctu_top_offset:CtbSizeY

[0215] As another example, a variable specifying a sample position may be shifted by a shift offset SO (eg, in loop filtering).

[0216] Even though the CTU grid is shifted and the rules for coexistence of CTUs / slices / tiles already in place in VVC are still valid, tiles and slices remain aligned on CTU borders.

[0217] Fig.26 A schematic block diagram of step 210 according to at least one exemplary embodiment is shown.

[0218] A CTU grid representing the spatial partitioning of the video image VP is obtained from the partition information data (step 210). For example, the number of columns of the CTU grid is given by dividing the width of the component of the video image VP by the CTU width CTU DW, and the number of rows of the CTU grid is given by dividing the height of the component of the video image VP by the CTU height CTU DH.

[0219] For example, a CTU grid may be defined for each component of the video picture VP. It is also possible to determine a single CTU grid for the video picture VP.

[0220] In step 211, at least one shift offset SO is obtained.

[0221] As a variant, if a CTB grid for luma components of a video picture VP is different from a CTU grid for chroma components of the video picture VP, the CTB grid for the luma components may be shifted according to at least one first shift offset SO1 and the CTB grid for the chroma components may be shifted according to at least one second shift offset SO2 different from the at least one first shift offset SO1.

[0222] In an exemplary embodiment of step 211 , the at least one shift offset SO is a horizontal shift offset x-SO.

[0223] In an exemplary embodiment of step 211 , the at least one shift offset SO is a vertical shift offset y-SO.

[0224] In an exemplary embodiment of step 211, one of the at least one shift offset SO is a vertical shift offset y-SO and the other shift offset is a horizontal shift offset x-SO.

[0225] In an exemplary embodiment of step 211, shift information data is decoded (signaled) from the bitstream B. The shift information data comprises binary data indicating whether at least one shift offset is present in the bitstream B, and if the shift information data indicates that at least one shift offset is present in the bitstream B, at least one determined shift offset is decoded (signaled) from the bitstream B.

[0226] In a variant, the shift information data is signaled (decoded) from the bitstream in a parameter set like a sequence parameter set, a picture parameter set or in a picture header syntax element according to the terminology of VVC or HEVC.

[0227] In step 212, the CTU grid is shifted according to at least one shift offset SO (or SO1 and SO2).

[0228] Fig. 27 A schematic block diagram illustrating an example of a system 600 in which various aspects and exemplary embodiments are implemented is shown.

[0229] System 600 may be embedded as one or more devices, including various components described below. In various embodiments, system 600 may be configured to implement one or more aspects described in this application.

[0230] Examples of equipment that may constitute all or part of system 600 include personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances, connected vehicles and their associated processing systems, head-mounted display devices (HMD, perspective glasses), projectors (projectors), "caves" (systems including multiple displays), servers, video encoders, video decoders, post-processors for processing outputs from video decoders, pre-processors for providing inputs to video encoders, web servers, video servers (e.g., broadcast servers, video-on-demand servers, or network servers), static or video cameras, encoding or decoding chips, or any other communication devices. The elements of system 600 may be implemented individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 600 may be distributed across multiple ICs and / or discrete components. In various embodiments, system 600 may be coupled to other similar systems or other electronic devices via, for example, a communication bus or by dedicated input and / or output ports.

[0231] The system 600 may include at least one processor 610 configured to execute instructions loaded therein for implementing, for example, various aspects described in the present application. The processor 610 may include embedded memory, input-output interfaces, and various other circuits known in the art. The system 600 may include at least one memory 620 (e.g., a volatile memory device and / or a non-volatile memory device). The system 600 may include a storage device 640, which may include a non-volatile memory and / or a volatile memory, including but not limited to an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a disk drive, and / or an optical drive. As a non-limiting example, the storage device 640 may include an internal storage device, an attached storage device, and / or a network accessible storage device.

[0232] System 600 may include an encoder / decoder module 630, which is configured to, for example, process data to provide encoded / decoded video image data, and the encoder / decoder module 630 may include its own processor and memory. The encoder / decoder module 630 may represent a module (one or more) that may be included in a device to perform encoding and / or decoding functions. As known, a device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 630 may be implemented as a separate element of the system 600, or may be incorporated into the processor 610 as a combination of hardware and software known to those skilled in the art.

[0233] Program code to be loaded onto the processor 610 or the encoder / decoder 630 to perform various aspects described in the present application may be stored in the storage device 640 and subsequently loaded onto the memory 620 to be executed by the processor 610. According to various embodiments, during the execution of the processes described in the present application, one or more of the processor 610, the memory 620, the storage device 640, and the encoder / decoder module 630 may store one or more of various items. Such stored items may include, but are not limited to, video image data, information data for encoding / decoding video image data, bit streams, matrices, variables, and intermediate or final results of equations, formulas, operations, and operation logic processing.

[0234] In several embodiments, memory internal to the processor 610 and / or encoder / decoder module 630 may be used to store instructions and provide working memory for processes that may be performed during encoding or decoding.

[0235] However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 610 or the encoder / decoder module 630) is used for one or more of these functions. The external memory may be a memory 620 and / or a storage device 640, such as a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store the operating system of the television. In at least one embodiment, a fast external dynamic volatile memory such as RAM may be used as working memory for video encoding and decoding operations, such as for MPEG-2 Part 2 (also known as ITU-T Recommendation H.262 and ISO / IEC 13818-2, also known as MPEG-2 Video), AVC, HEVC, EVC, VVC, AV1, etc.

[0236] As indicated in block 690, input to the elements of system 600 may be provided through various input devices. Such input devices include, but are not limited to, (i) an RF section that may receive an RF signal transmitted over the air, for example, by a broadcast device, (ii) a composite input terminal, (iii) a USB input terminal, (iv) an HDMI input terminal, (v) when the present disclosure is implemented in the automotive field, a bus such as a CAN (Controller Area Network), CAN FD (Controller Area Network Flexible Data Rate), FlexRay (ISO 17458), or Ethernet (ISO / IEC 802-3) bus.

[0237] In various embodiments, the input device of block 690 has associated corresponding input processing elements, as known in the art. For example, the RF portion may be associated with elements necessary for: (i) selecting a desired frequency (also referred to as selecting a signal, or limiting a signal band to within a band), (ii) down-converting the selected signal, (iii) limiting the band again to a narrower band to select a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired packet stream. The RF portion of various embodiments may include one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF portion may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or a baseband.

[0238] In one set-top box embodiment, the RF section and its associated input processing elements can receive RF signals transmitted over a wired (e.g., cable) medium. The RF section can then perform frequency selection by filtering, down-converting, and filtering again to a desired frequency band.

[0239] Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions.

[0240] Adding an element may include inserting an element between existing elements, such as, for example, inserting an amplifier and an analog-to-digital converter.In various embodiments, the RF portion may include an antenna.

[0241] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 600 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, in a separate input processing IC or in the processor 610 when necessary. Similarly, various aspects of USB or HDMI interface processing may be implemented in a separate interface IC or in the processor 610 when necessary. The demodulated, error-corrected, and demultiplexed streams may be provided to various processing elements, including, for example, the processor 610 and the encoder / decoder 630, which operate in conjunction with the memory and storage elements to process the data streams for presentation on the output device when necessary.

[0242] The various elements of system 600 may be provided within an integrated housing. Within the integrated housing, the various elements may be interconnected and data transferred between them using a suitable connection arrangement 690, such as an internal bus (including an I2C bus), wiring, and printed circuit boards known in the art.

[0243] The system 600 may include a communication interface 650 that enables communication with other devices via a communication channel 651. The communication interface 650 may include, but is not limited to, a transceiver configured to send and receive data over the communication channel 651. The communication interface 650 may include, but is not limited to, a modem or a network card, and the communication channel 651 may be implemented, for example, within a wired and / or wireless medium.

[0244] In various embodiments, a Wi-Fi network such as IEEE 802.11 may be used to stream data to the system 600. The Wi-Fi signals of these embodiments may be received via a communication channel 651 suitable for Wi-Fi communications and a communication interface 650. The communication channel 651 of these embodiments may typically connect to an access point or router that provides access to external networks including the Internet to allow streaming applications and other over-the-top communications.

[0245] Other embodiments may provide streaming data to system 600 using a set top box that delivers the data through an HDMI connection to input block 690 .

[0246] Still other embodiments may use the RF connection of input block 690 to provide streaming data to system 600 .

[0247] The streamed data may be used as a means of signaling information used by the system 600. The signaling information may include the bitstream B and / or information such as the number of pixels of a video image and / or any encoding / decoding setting parameters.

[0248] It should be appreciated that signaling may be implemented in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, etc. may be used to signal information to a corresponding decoder.

[0249] The system 600 may provide output signals to various output devices, including a display 661, speakers 671, and other peripherals 681. In various examples of embodiments, the other peripherals 681 may include one or more of a stand-alone DVR, a disc player, a stereo system, a lighting system, and other devices that provide functionality based on the output of the system 600.

[0250] In various embodiments, control signals may be communicated between the system 600 and the display 661, speakers 671, or other peripherals 681 using signaling such as AV.Link (Audio / Video Link), CEC (Consumer Electronics Control), or other communication protocols that enable device-to-device control with or without user intervention.

[0251] Output devices may be communicatively coupled to system 600 via dedicated connections through respective interfaces 660 , 670 , and 680 .

[0252] Optionally, output devices may be connected to system 600 using communication channel 651 via communication interface 650. Display 661 and speaker 671 may be integrated into a single unit with other components of system 600 in an electronic device such as, for example, a television.

[0253] In various embodiments, the display interface 660 may include a display driver, such as, for example, a timing controller (TCon) chip.

[0254] For example, if the RF portion of input 690 is part of a separate set-top box, then display 661 and speaker 671 may optionally be separate from one or more of the other components. In various embodiments where display 661 and speaker 671 may be external components, output signals may be provided via dedicated output connections including, for example, an HDMI port, a USB port, or a COMP output.

[0255] exist Figures 1 to 27 In the present invention, various methods are described herein, and each method includes one or more steps or actions to implement the described method. Unless a specific order of steps or actions is required for the correct operation of the method, the order and / or use of specific steps and / or actions can be modified or combined.

[0256] Some examples are described about block diagrams and / or operational flow charts. Each square block represents a portion of a circuit element, module or code, which includes one or more executable instructions for implementing (one or more) specified logical functions. It should also be noted that, in other embodiments, the (one or more) functions marked in the square block may not occur in the order indicated. For example, depending on the functions involved, two square blocks shown in succession can actually be executed substantially concurrently, or sometimes these square blocks can be executed in reverse order.

[0257] The embodiments and aspects described herein may be implemented in, for example, a method or process, an apparatus, a computer program, a data stream, a bit stream, or a signal. Even if only discussed in the context of a single form of embodiment (e.g., discussed only as a method), the embodiments of the features discussed may also be implemented in other forms (e.g., an apparatus or a computer program).

[0258] The method may be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. A processor also includes a communication device.

[0259] In addition, the method can be implemented by instructions executed by a processor, and such instructions (and / or data values ​​generated by the implementation) can be stored on a computer-readable storage medium. The computer-readable storage medium can take the form of a computer-readable program product implemented in one or more computer-readable media and having a computer-readable program code implemented thereon that can be executed by a computer. Considering the inherent ability to store information therein and the inherent ability to provide information retrieval therefrom, the computer-readable storage medium used herein can be considered as a non-transient storage medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. It should be appreciated that although more specific examples of computer-readable storage media to which the present embodiment can be applied are provided below, as those of ordinary skill in the art will readily recognize, it is merely an illustrative and non-exhaustive list: portable computer floppy disk; hard disk; read-only memory (ROM); erasable programmable read-only memory (EPROM or flash memory); portable compact disk read-only memory (CD-ROM); optical storage device; magnetic storage device; or any suitable combination of the foregoing.

[0260] The instructions may form an application program tangibly embodied on a processor-readable medium.

[0261] For example, instructions may be in hardware, firmware, software, or a combination. For example, instructions may be found in an operating system, a separate application, or a combination of both. Thus, a processor may be characterized as, for example, a device configured to perform a process and a device including a processor-readable medium (such as a storage device) having instructions for performing the process. Additionally, in addition to or in lieu of instructions, a processor-readable medium may store data values ​​generated by an embodiment.

[0262] Device can be realized in suitable hardware, software and firmware for example.The example of such device comprises personal computer, laptop computer, smart phone, tablet computer, digital multimedia set-top box, digital television receiver, personal video recording system, connected household appliances, head-mounted display device (HMD, perspective glasses), projector (projector), "cave" (system including multiple displays), server, video encoder, video decoder, post-processor for processing the output from video decoder, pre-processor for providing input to video encoder, web server, set-top box, and any other equipment for processing video image, or other communication equipment.It should be clear that equipment can be mobile and even installed in a mobile vehicle.

[0263] Computer software may be implemented by processor 610 or by hardware, or by a combination of hardware and software. As a non-limiting example, embodiments may also be implemented by one or more integrated circuits. Memory 620 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology (such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples). Processor 610 may be of any type suitable for the technical environment and may encompass one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture, as non-limiting examples.

[0264] As will be apparent to one of ordinary skill in the art, embodiments may generate various signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method or data generated by one of the described embodiments. For example, a signal may be formatted to carry a bit stream of the described embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0265] The term used herein is only used to describe the purpose of specific embodiments and is not intended to be limited. As used herein, the singular form "one / kind (a)", "one / kind (an)" and "the / said (the)" may also be intended to include plural forms, unless the context clearly indicates otherwise. It will be further understood that, when used in this specification, the term "include / comprise" and / or "include / comprising (including / comprising)" may specify the existence of stated, for example, features, integers, steps, operations, elements and / or components, but does not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. Moreover, when an element is referred to as "response" or "connection" or "associated" to another element, it may directly respond or be connected to another element or be associated with another element, or there may be an intermediate element. On the contrary, when an element is referred to as "direct response" or "direct connection" to another element or "directly associated" with another element, there is no intermediate element.

[0266] It should be appreciated that use of any of the symbols / terms " / ", "and / or", and "at least one of" may be intended to encompass selection of only the first listed option (A), or only the second listed option (B), or both options (A and B), such as in the case of "A / B", "A and / or B", and "at least one of A and B". As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C", such wording is intended to encompass selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This may be extended to as many items as listed, as will be apparent to one of ordinary skill in this and related arts.

[0267] Various numerical values ​​may be used in the present application. Specific values ​​may be used for example purposes and the described aspects are not limited to these specific values.

[0268] It will be understood that although the terms first, second, etc. can be used to describe various elements in this article, these elements are not limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the teaching of this application, the first element can be referred to as the second element, and similarly, the second element can be referred to as the first element. There is no suggestion of sorting between the first element and the second element.

[0269] References to "an exemplary embodiment" or "exemplary embodiments" or "one implementation" or "implementation" and other variations thereof are frequently used to convey that a particular feature, structure, characteristic, etc. (described in conjunction with an embodiment / implementation) is included in at least one embodiment / implementation. Thus, the appearances of the phrases "in an exemplary embodiment" or "in an exemplary embodiment" or "in one implementation" or "in an implementation" and any other variations appearing in various places in this application do not necessarily all refer to the same embodiment.

[0270] Similarly, references herein to "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" and other variations thereof are frequently used to convey that a particular feature, structure, or characteristic (described in conjunction with an exemplary embodiment / example / implementation) may be included in at least one exemplary embodiment / example / implementation. Therefore, the expressions "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" appearing in various places in this application do not necessarily all refer to the same exemplary embodiment / example / implementation, nor are separate or alternative exemplary embodiments / examples / implementations necessarily mutually exclusive of other exemplary embodiments / examples / implementations.

[0271] Reference numerals appearing in the claims are for illustration only and have no limiting effect on the scope of the claims.The present embodiments / examples and variants may be employed in any combination or sub-combination although not explicitly described.

[0272] When a figure is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0273] While some diagrams include arrows on communication paths to illustrate a primary direction of communication, it should be understood that communication can occur in the opposite direction to the depicted arrows.

[0274] Various embodiments relate to decoding. As used in the present application, "decoding" may encompass, for example, all or part of a process performed on a received video image (which may include a received bitstream encoding one or more video images) to produce a final output suitable for display or further processing in a reconstructed video domain. In various embodiments, such a process includes one or more of the processes typically performed by a decoder. In various embodiments, for example, such a process also or alternatively includes a process performed by a decoder of the various embodiments described in the present application.

[0275] As a further example, in one embodiment "decoding" may refer only to dequantization, in one embodiment "decoding" may refer to entropy decoding, in another embodiment "decoding" may refer only to differential decoding, and in another embodiment "decoding" may refer to a combination of dequantization, entropy decoding, and differential decoding. Based on the context of the specific description, whether the phrase "decoding process" may be intended to specifically refer to a subset of operations, or generally refer to a broader decoding process will be clear and is believed to be well understood by those skilled in the art.

[0276] Various embodiments all relate to encoding. In a manner similar to the above discussion about "decoding", "encoding" as used in this application may encompass, for example, all or part of a process performed on an input video image to produce an output bitstream. In various embodiments, such a process includes one or more of the processes typically performed by an encoder. In various embodiments, such a process also includes or optionally includes a process performed by an encoder of the various embodiments described in this application.

[0277] As a further example, in one embodiment "encoding" may refer only to quantization, in one embodiment "encoding" may refer only to entropy coding, in another embodiment "encoding" may refer only to differential coding, and in another embodiment "encoding" may refer to a combination of quantization, differential coding, and entropy coding. Based on the context of the particular description, whether the phrase "encoding process" may be intended to specifically refer to a subset of operations, or to generally refer to a broader encoding process will be clear and is believed to be well understood by those skilled in the art.

[0278] Additionally, the present application may refer to "obtaining" various information. Obtaining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory, processing information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0279] Additionally, this application may refer to "receiving" various information. Receiving information may include, for example, one or more of accessing the information or receiving the information from a communication network.

[0280] Moreover, as used herein, the word "signal" especially refers to indicating something to a corresponding decoder, etc. For example, in some embodiments, the encoder signals specific information, such as coding parameters or encoded video image data. In this way, in an embodiment, the same parameter can be used on the encoder side and the decoder side. Therefore, for example, the encoder can transmit (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters and other parameters, signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select specific parameters. By avoiding the transmission of any actual function, bit saving is achieved in various embodiments. It should be recognized that signaling can be completed in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, etc. are used to signal information to the corresponding decoder. Although the verb form of the word "signal" is mentioned above, the word "signal" can also be used as a noun in this article.

[0281] A number of embodiments have been described. However, it should be understood that various modifications may be made. For example, elements of different embodiments may be combined, supplemented, modified, or removed to produce other embodiments. In addition, it will be understood by those of ordinary skill that other structures and processes may replace the disclosed structures and processes, and that the resulting embodiments will perform at least substantially the same (one or more) functions in at least substantially the same (one or more) manners to achieve at least substantially the same (one or more) results as the disclosed embodiments. Thus, the present application contemplates these and other embodiments.

Claims

1. A method for encoding a video image into a bit stream of encoded video image data, wherein the method comprises: - obtaining (111) a coding tree unit grid for partitioning at least one component of the video image into coding tree units, each coding tree unit being an image region subdivided according to a coding tree; - determining (112) at least one shift offset by aligning at least one boundary of the coding tree unit grid with at least one boundary separating image areas of the video image having low spatial activity from image areas having high spatial activity; - shifting (113) the coding tree unit grid according to the at least one shift offset; - obtaining (120-160) coded video image data by encoding at least one coding unit (CU) of a coding tree associated with each coding tree unit (CTU) of the shifted grid of coding tree units; as well as - writing the encoded video data to the bitstream.

2. The method according to claim 1, wherein the spatial activity of the video image is determined by detecting vertical edges and horizontal edges from the video image content, and at least one boundary separating image areas with low spatial activity from image areas with high spatial activity of the video image is a detected vertical edge or horizontal edge. 3 . The method of claim 2 , wherein the at least one shift offset is determined by minimizing a spatial distance between a boundary of the coding tree unit grid and a detected vertical edge or horizontal edge.

4. A method for decoding a video image from a bit stream of decoded video image data, wherein the method comprises - determining (211) a grid of coding tree units representing a spatial partitioning of said video image, each coding tree unit representing an image region of said video image subdivided according to a coding tree; - shifting (212) the coding tree unit grid according to at least one shift offset; - obtaining (220-260) video image data by decoding from the bitstream at least one coding unit (CU) of a coding tree associated with each coding tree unit (CTU) of the shifted grid of coding tree units; and - obtaining said video image from said video image data.

5. The method of claim 4, wherein the method further comprises signaling into the bitstream a syntax element indicating whether the coding tree unit grid is vertically shifted, horizontally shifted, or both.

6. A method according to any one of claims 4 to 5, wherein the method further comprises signaling into the bitstream shift information data comprising binary data indicating whether at least one shift offset is present in the bitstream, and if the shift information data indicates that at least one shift offset is present in the bitstream, signaling the at least one shift offset into the bitstream.

7. The method of claim 6, wherein the shift information data is signaled to a syntax element comprising a sequence parameter set, a picture parameter set or a picture header.

8. The method of any one of claims 4 to 7, wherein the video image comprises a luma component, the luma component comprising a set of luma samples, and the at least one shift offset is an integer multiple of a maximum value between 8 and a minimum decoding block size of luma samples.

9. The method according to any one of claims 4 to 7, wherein the at least one shift offset is an integer multiple of 8, or is indexed in multiples of 8.

10. The method according to any one of claims 4 to 7, wherein the at least one shift offset is smaller than a size of a fractional coding tree unit size.

11. A bit stream formatted to include coded video image data and obtained according to the method of any one of claims 1 to 3.

12. An apparatus for encoding a video image into a bit stream of encoded video image data, the apparatus comprising means for performing one of the methods of any one of claims 1 to 3.

13. An apparatus for video decoding of a video image from a bit stream of encoded video image data, the apparatus comprising means for performing one of the methods of any one of claims 4 to 10.

14. A computer program product comprising instructions, which, when the program is executed by one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 3, or the method of any one of claims 4 to 10.

15. A non-transitory storage medium carrying program code instructions for executing the method of any one of claims 1 to 3, or the method of any one of claims 4 to 10.

Images