Encoder, decoder and corresponding methods for simplified slice header syntax element indication

Abstract

The application provides a method for decoding an image from a video code stream by a decoding device. The code stream comprises a slice header of a current slice and data representing the current slice, and the method comprises: if a condition is satisfied, obtaining a parameter for deriving a number of sub-blocks in the current slice from the slice header, wherein the condition comprises: a slice address of the current slice is not an address of a last sub-block in an image where the current slice is located; and reconstructing the current slice by the number of sub-blocks in the current slice and the data representing the current slice.

Description

[0001] This application is a divisional application of the original application with the application number 202180015669.6 and the original filing date of 01 March 2021, the entire contents of the original application are incorporated herein by reference.

[0002] Related Applications

[0003] This patent application claims priority to international patent application PCT / EP2020 / 055341 filed on February 28, 2020. The entire disclosure of the above-mentioned international patent application is incorporated herein by reference. TECHNICAL FIELD

[0004] Embodiments of the present application (the present application) generally relate to the field of image processing, and more particularly to simplifying slice header syntax element indication. BACKGROUND

[0005] Video coding (encoding and decoding) is widely used in digital video applications, such as broadcast digital TV, internet- and mobile network-based video transmission, real-time conversational applications such as video chat, video conferencing, DVD and Blu-ray discs, video content capture and editing systems, and camcords for security applications.

[0006] Even relatively short videos require large amounts of video data to describe, which can create difficulties when the data is to be streamed or otherwise delivered over a communication network with limited bandwidth capacity. Thus, video data is generally compressed before being delivered through modern telecommunications networks. Memory resources can be limited when videos are stored in storage devices, and the size of the video can also be an issue. Video compression devices often use software and / or hardware at the source side to encode video data, which reduces the amount of data needed to represent digital video images, before the video data is transmitted or stored. The video data is then decoded by a video decompression device at the destination side to reproduce the images. With limited network resources and ever increasing demands of higher video quality, there is a need for improved compression and decompression techniques that improve compression ratios without sacrificing image quality. SUMMARY

[0007] Embodiments of the present application provide various apparatuses and methods for encoding and decoding according to the independent claims.

[0008] The present application provides:

[0009] A method of decoding a picture from a video bitstream by a decoding device. The bitstream includes a slice header of a current slice and data representing the current slice, the method includes: if a condition is satisfied, obtaining a parameter (e.g. um_tiles_in_slice_minus1) for deriving a number of tiles in the current slice from the slice header, wherein the condition includes: a slice address (e.g. slice_address) of the current slice is not an address of a last tile in a picture in which the current slice is located; reconstructing the current slice by the number of tiles in the current slice and the data representing the current slice.

[0010] In the method described above, the slice address of the current slice being the address of the last tile in the picture can include determining that the number of tiles in the picture minus the slice address of the current slice is equal to 1.

[0011] In the method described above, the slice address of the current slice not being the address of the last tile in the picture can include determining that the number of tiles in the picture (e.g. NumTilesInPic) minus the slice address of the current slice is greater than 1.

[0012] Therefore, according to the present application, whether a picture header structure exists in a slice header can be used to control whether a slice address and a number of tiles exist in a slice indication. If there is a single slice in a picture, the slice address should be equal to the first tile in the picture, and the number of tiles in the slice should be equal to the number of tiles in the picture. Therefore, this can improve compression efficiency.

[0013] In the method described above, when the condition is not satisfied, a value of the parameter of the current slice can be inferred as a default value.

[0014] In the method described above, the default value can be equal to 0.

[0015] In the method described above, the slice address can be in units of tiles.

[0016] In the method described above, the condition can further include determining that the current slice is in a raster scan mode.

[0017] In the method described above, the reconstructing the current slice by the number of tiles in the current slice can include determining a scan order of coding tree units in the current slice by the number of tiles in the current slice; reconstructing the coding tree units in the current slice by the scan order.

[0018] The application also provides a method for encoding a video bitstream, implemented by an encoding device. The bitstream comprises a slice header of a current slice and data representing the current slice. The method comprises: if a condition is satisfied, encoding a parameter in the slice header for deriving a number of tiles in the current slice, wherein the condition comprises: a slice address of the current slice is not an address of a last tile in an image in which the current slice is located; and reconstructing the current slice by the number of tiles in the current slice and the data representing the current slice.

[0019] The application also provides a device for decoding an image from a video bitstream. The bitstream comprises a slice header of a current slice and data representing the current slice. The device comprises: an obtaining unit, configured to: if a condition is satisfied, obtain a parameter in the slice header for deriving a number of tiles in the current slice, wherein the condition comprises: a slice address of the current slice is not an address of a last tile in an image in which the current slice is located; and a reconstructing unit, configured to reconstruct the current slice by the number of tiles in the current slice and the data representing the current slice.

[0020] The application also provides a device for encoding a coded video bitstream. The bitstream comprises a slice header of a current slice and data representing the current slice. The device comprises: an encoding unit, configured to: if a condition is satisfied, encode a parameter in the slice header for deriving a number of tiles in the current slice, wherein the condition comprises: a slice address of the current slice is not an address of a last tile in an image in which the current slice is located; and a reconstructing unit, configured to reconstruct the current slice by the number of tiles in the current slice and the data representing the current slice.

[0021] The application also provides an encoder. The encoder comprises processing circuitry configured to perform the above-described method for encoding a video bitstream.

[0022] The application also provides a decoder. The decoder comprises processing circuitry configured to perform the above-described method for decoding a video bitstream.

[0023] The application also provides a computer program product comprising program code. When the program code is executed on a computer or a processor, the program code is configured to perform the above-described method for encoding a video bitstream or the above-described method for decoding a video bitstream.

[0024] The application also provides a decoder. The decoder comprises one or more processors; and a non-transitory computer-readable storage medium coupled to the one or more processors and storing programming for execution by the one or more processors, wherein the programming, when executed by the one or more processors, causes the decoder to carry out the method for decoding a video bitstream described above.

[0025] The application also provides an encoder. The encoder comprises one or more processors; and a non-transitory computer-readable storage medium coupled to the one or more processors and storing programming for execution by the one or more processors, wherein the programming, when executed by the one or more processors, causes the encoder to carry out the method for encoding a video bitstream described above.

[0026] The application also provides a non-transitory computer-readable medium including programming for execution by a computer device. The programming, when executed by the computer device, causes the computer device to carry out the method for encoding a video bitstream described above or the method for decoding a video bitstream described above.

[0027] The application also provides a non-transitory storage medium including a video bitstream. The bitstream comprises a slice header of a current slice and data representing the current slice, the slice header comprising a slice address of the current slice; and the slice header further comprises a parameter used to derive a number of tiles in the current slice from the slice header if a condition is satisfied, the condition comprising that the slice address of the current slice is not an address of a last tile in a picture in which the current slice is located.

[0028] One or more embodiments are set forth in detail below. Other features, objects, and advantages are apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF DRAWINGS

[0029] The embodiments of the present application are described below in detail with reference to the accompanying drawings.

[0030] FIG. 1A A block diagram of an example of a video coding system for implementing embodiments of the present application.

[0031] FIG. 1B A block diagram of another example of a video coding system for implementing embodiments of the present application.

[0032] FIG. 2 A block diagram of an example of a video encoder for implementing embodiments of the present application.

[0033] FIG. 3 A block diagram of an example of a video decoder for implementing embodiments of the present application.

[0034] FIG. 4 A schematic diagram of a video coding device is provided for an embodiment of the present application.

[0035] FIG. 5 A simplified block diagram of an apparatus is provided for an embodiment of the present application.

[0036] FIG. 6 An example of image partitioning into CTUs is shown.

[0037] FIG. 7 An example of raster scan order is shown.

[0038] FIG. 8 An example of tiles, slices and subpictures is shown.

[0039] FIG. 9 An example of raster scan order in a picture composed of CTUs and a slice in the picture is shown.

[0040] FIG. 10 An example of image partitioning into 12 tiles and 3 slices is shown.

[0041] FIG. 11 A block diagram for an example of LMCS architecture.

[0042] FIG. 12 A block diagram of an example structure of a content supply system 3100 for implementing a content distribution service.

[0043] FIG. 13 A block diagram of an example structure of a terminal device.

[0044] FIG. 14 A flowchart of a method of decoding a video bitstream is provided for an embodiment of the present application.

[0045] FIG. 15 A flowchart of a method of decoding a video bitstream is provided for an embodiment of the present application.

[0046] FIG. 16 A flowchart of a decoder for decoding a video bitstream is provided for an embodiment of the present application.

[0047] FIG. 17 A flowchart of an encoder for encoding a video bitstream is provided for an embodiment of the present application.

[0048] Hereinafter, if not explicitly otherwise stated, the same reference numerals refer to the same features or at least functionally equivalent features. DETAILED DESCRIPTION

[0049] In the following description, reference is made to the accompanying drawings which form a part hereof, and which are shown by way of illustration of specific aspects or can be used in conjunction with implementations of aspects. It is understood that other aspects can be used and structural or logical changes can be made without departing from the scope of the present disclosure. The following detailed description, therefore, is not to be taken in a limiting sense, as the scope of the present application is defined by the appended claims.

[0050] For example, it is to be understood that the disclosure in relation to a described method can equally apply to a corresponding device or system for performing the described method, and vice versa. For example, if one or more specific method steps are described, a corresponding device can include one or more units (for example, functional units) to perform the described one or more method steps (for example, one unit performing the one or more steps, or a plurality of units each performing one or more of the plurality of steps), even if such one or more units are not explicitly described or illustrated in the figures. On the other hand, for example, if a specific apparatus is described based on one or more units (for example, functional units), a corresponding method can include a step to perform the functionality of one or more units (for example, one step performing the functionality of one or more units, or a plurality of steps each performing the functionality of one or more of the plurality of units), even if such one or more steps are not explicitly described or illustrated in the figures. Further, it is understood that the features of the various exemplary embodiments and / or aspects described herein can be combined with each other, unless specifically mentioned otherwise.

[0051] Video coding generally refers to the processing of a sequence of pictures forming a video or video sequence. In the field of video coding, the terms “frame” and “picture / image” can be used as synonyms. Video coding (or generally coding) includes both video encoding and video decoding. Video encoding is performed at the source side, typically including processing (e.g., by compression) of original video pictures to reduce the amount of data required to represent the video pictures (to store and / or transmit more efficiently). Video decoding is performed at the destination side, typically including inverse processing with respect to the encoder to reconstruct the video pictures. Embodiments relate to “coding” of video pictures (or generally pictures), which is to be understood as “encoding” or “decoding” of video pictures or corresponding video sequence. The encoding part and the decoding part are also referred to as coding (encoding and decoding) collectively.

[0052] In lossless video coding, the original video pictures can be reconstructed, i.e., the reconstructed video pictures have the same quality as the original video pictures (assuming no transmission losses or other data loss during storage or transmission). In lossy video coding, further compression is performed, e.g., by quantization, to reduce the amount of data representing the video pictures, and the video pictures cannot be completely reconstructed at the decoder side, i.e., the reconstructed video pictures have a lower or worse quality than the original video pictures.

[0053] Several video coding standards belong to the group of "lossy hybrid video codecs" (i.e., combining spatial and temporal prediction in the sample domain with 2D transform coding in the transform domain for applying quantization). Each picture in a video sequence is typically partitioned into a set of non-overlapping blocks, and coding is typically performed at block level. In other words, the encoder typically processes, i.e., encodes, the video at block (video block) level, e.g., by generating a prediction block by spatial (intra) prediction and / or temporal (inter) prediction, subtracting the prediction block from the current block (block currently processed / pending processing) to obtain a residual block, transforming the residual block in the transform domain and quantizing the residual block to reduce the amount of data to be transmitted (compressed), and the decoder applies inverse processing with respect to the encoder to the encoded or compressed block to reconstruct the current block for representation. Furthermore, the encoder replicates the decoder processing loop such that the encoder and the decoder generate the same predictions (e.g., intra and inter predictions) and / or reconstructions to process, i.e., code, subsequent blocks.

[0054] In the following embodiments of a video coding system 10, a video encoder 20 and a video decoder 30 operate according to FIG. 1A through FIG. 3 are described.

[0055] FIG. 1A For illustrative coding system 10, e.g., a video coding system 10 (or simply coding system 10) can utilize the techniques of the present application. A video encoder 20 (or simply encoder 20) and a video decoder 30 (or simply decoder 30) in the video coding system 10 are two examples of devices that can be used to perform various techniques according to various examples described in the present application.

[0056] As shown in FIG. 1, the coding system 10 includes a source device 12 for providing encoded picture data 21 to a destination device 14, etc., for decoding the encoded picture data 21. FIG. 1A

[0057] The source device 12 includes the encoder 20, and can additionally (i.e., optionally) include a picture source 16, a preprocessor (or pre-processing unit) 18 (e.g., picture preprocessor 18), and a communication interface or unit 22.

[0058] ​The image source 16 can comprise or be any type of image capturing device, such as a camera, for capturing real-world images, and / or any type of image generating device, such as a computer graphics processor, for generating computer-animated images, or any other device for acquiring and / or providing real-world images, computer-generated images (e.g. screen content, virtual reality (VR) images) and / or any combination thereof (e.g. augmented reality (AR) images). The image source can be any type of memory (storage) storing any of the aforementioned images.

[0059] For the sake of distinguishing the processing performed by the pre-processor 18 and the pre-processing unit 18, the image or image data 17 can also be referred to as raw image or raw image data 17.

[0060] The pre-processor 18 is configured to receive the (raw) image data 17 and perform pre-processing on the image data 17 to obtain pre-processed image data 19 or pre-processed image 19. The pre-processing performed by the pre-processor 18 can include trimming, color format conversion (e.g. from RGB to YCbCr), toning or de-noising, etc. It can be appreciated that the pre-processing unit 18 can be an optional component.

[0061] The video encoder 20 is configured to receive the pre-processed image data 19 and provide encoded image data 21 (more details are provided below in connection with FIG. 2 etc.).

[0062] The communication interface 22 in the source device 12 can be configured to receive the encoded image data 21 and transmit the encoded image data 21 (or data resulting from further processing of the encoded image data 21) to another device, such as the destination device 14 or any other device, over the communication channel 13 for storage or direct reconstruction.

[0063] The destination device 14 comprises a decoder 30 (e.g. a video decoder 30) and can additionally (i.e. optionally) comprise a communication interface or communication unit 28, a post-processor 32 (or post-processing unit 32) and a display device 34.

[0064] The communication interface 28 in the destination device 14 is configured to receive the encoded image data 21 (or data resulting from further processing of the encoded image data 21) directly from the source device 12 or from any other source, such as a storage device (e.g. an encoded image data storage device) and provide the encoded image data 21 to the decoder 30.

[0065] The communication interface 22 and the communication interface 28 can be configured to transmit or receive the encoded image data 21 or the encoded data 13 over a direct communication link (e.g. a direct wired or wireless connection) between the source device 12 and the destination device 14 or over any type of network (e.g. a wired network, a wireless network or any combination thereof, or any type of private and public network, or any combination thereof).

[0066] For example, the communication interface 22 can be configured to encapsulate the encoded image data 21 into a suitable format (e.g. data packets) and / or to process the encoded image data by any type of transmission encoding or processing in order to transmit it over the communication link or the communication network.

[0067] For example, the communication interface 28 corresponding to the communication interface 22 can be configured to receive the transmitted data and to process it using any type of corresponding transmission decoding or processing and / or de-encapsulation in order to obtain the encoded image data 21.

[0068] The communication interface 22 and the communication interface 28 can both be configured as FIG. 1A one-way communication interfaces as indicated by the arrow of the communication channel 13 pointing from the source device 12 to the destination device 14 or as bi-directional communication interfaces and can be configured to transmit and receive messages and the like in order to establish a connection, to confirm and exchange any other information related to the communication link and / or the data transmission (e.g. the encoded image data transmission), and the like.

[0069] The decoder 30 is configured to receive the encoded image data 21 and to provide decoded image data 31 or decoded images 31 (further details are provided below in connection with Fig. 2). FIG. 3 or FIG. 5 Further details are provided below.

[0070] The post-processor 32 in the destination device 14 is configured to post-process the decoded image data 31 (also referred to as reconstructed image data) (e.g. the decoded images 31) in order to obtain post-processed image data 33 (e.g. post-processed images 33). The post-processing performed by the post-processing unit 32 can include color format conversion (e.g. from YCbCr to RGB), toning, clipping or resampling, or any other processing in order to provide the decoded image data 31 for display by a display device 34 or the like, and the like.

[0071] The display device 34 in the destination device 14 is configured to receive the post-processed picture data 33 for displaying the picture to a user or viewer or the like. The display device 34 can be or can comprise any type of display for representing the reconstructed picture, e.g. an integrated or external display or display screen. For example, the display can comprise a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS) display, a digital light processor (DLP) or any type of other display.

[0072] Although FIG. 1A The source device 12 and the destination device 14 are shown as separate devices, but the device embodiments can also comprise both devices or the functionality of both devices, i.e. the source device 12 or the corresponding functionality and the destination device 14 or the corresponding functionality, at the same time. In these embodiments, the source device 12 or the corresponding functionality and the destination device 14 or the corresponding functionality can provide the same hardware and / or software or be realized by separate hardware and / or software or any combination thereof.

[0073] It is obvious to a person skilled in the art, based on the description, FIG. 1A The presence and (precise) functional split of the different units or functionalities within the shown source device 12 and / or destination device 14 can differ depending on the actual device and application.

[0074] The encoder 20, e.g. the video encoder 20, or the decoder 30, e.g. the video decoder 30, or the encoder 20 and the decoder 30 can be implemented by FIG. 1B The processing circuitry shown. The processing circuitry comprises one or more microprocessors, one or more digital signal processors (DSPs), one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), one or more discrete logic components, one or more hardware processors, one or more video coding dedicated processors, or any combination thereof. The encoder 20 can be implemented by the processing circuitry 46 to include various modules described with reference to the encoder 20 in FIG. 2 The decoder 30 can be implemented by the processing circuitry 46 to include various modules described with reference to the decoder 30 inFIG. 3 The various modules described in connection with decoder 30 in FIG. 1, and / or any other decoder system or subsystem described herein. The processing circuitry can be used to perform the various operations discussed infra. As FIG. 5 indicated, if the techniques described above are implemented in software, an apparatus can store instructions for the software in a suitable, non- transitory computer-readable storage medium and can execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Video encoder 20 or video decoder 30 can be integrated in a single device as part of a combined encoder / decoder (CODEC), for example, as FIG. 1B indicated.

[0075] Source device 12 and destination device 14 can comprise any of a variety of devices including handheld or fixed devices, for example, notebook or laptop computers, mobile phones, smart phones, tablet or slate computers, camcorders, desktop computers, set-top boxes, televisions, display devices, digital media players, video gaming consoles, video streaming devices (e.g., content service servers or content distribution servers), broadcast receiver devices, broadcast transmitter devices, etc., and can use or not use any type of operating system. In some cases, source device 12 and destination device 14 can be configured for wireless communication. Thus, source device 12 and destination device 14 can be wireless communication devices.

[0076] In some cases, FIG. 1A FIG. 1 is merely one example. The techniques of this disclosure can be applied in video coding settings in which data is not necessarily communicated between an encoding device and a decoding device (e.g., video encoding or video decoding). In other examples, data is retrieved from local storage, streamed over a network, etc. A video encoding device can encode data and store the data to storage, and / or a video decoding device can retrieve data from storage and decode the data. In some examples, encoding and decoding are performed by devices that do not communicate with one another, but rather only encode data to storage and / or retrieve data from storage and decode the data.

[0077] For ease of description, this document (for example) refers to reference software developed by the Joint Collaboration Team on Video Coding (JCT-VC) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG), namely High-Efficiency Video Coding (HEVC) or Versatile Video Coding (VVC), a next-generation video coding standard, to describe embodiments of this application. Those skilled in the art will understand that embodiments of this application are not limited to HEVC or VVC.

[0078] Encoders and Encoding Methods

[0079] FIG. 2 This is a schematic block diagram of an exemplary video encoder 20 used to implement the technology in this application. FIG. 2 In the example, the video encoder 20 includes an input terminal 201 (or input interface 201), a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a loop filter unit 220, a decoded picture buffer (DPB) 230, a mode selection unit 260, an entropy coding unit 270, and an output terminal 272 (or output interface 272). The mode selection unit 260 may include an inter-frame prediction unit 244, an intra-frame prediction unit 254, and a segmentation unit 262. The inter-frame prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). FIG. 2 The video encoder 20 shown can also be called a hybrid video encoder or a video encoder based on a hybrid video codec.

[0080] The residual calculation unit 204, transform processing unit 206, quantization unit 208, and mode selection unit 260 can form the forward signal path of the encoder 20, while the inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, buffer 216, loop filter 220, decoded picture buffer (DPB) 230, inter-frame prediction unit 244, and intra-frame prediction unit 254 can form the backward signal path of the video encoder 20. The backward signal path of the video encoder 20 corresponds to the decoder (see [link to decoder]). FIG. 3The signal path of the video decoder 30 in FIG. 1. The inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the loop filter 220, the decoded picture buffer (DPB) 230, the inter prediction unit 244 and the intra prediction unit 254 also constitute an "in-built decoder" of the video encoder 20.

[0081] Pictures and picture partitioning (pictures and blocks)

[0082] The encoder 20 can be configured to receive a picture 17 (or picture data 17) via an input 201 or the like. The picture 17 can be a picture of a sequence of pictures that constitutes a video or video sequence. The received picture or picture data can also be a pre-processed picture 19 (or pre-processed picture data 19). For simplicity, the picture 17 is used in the following description. The picture 17 can also be referred to as current picture or picture to be coded (especially in video coding to distinguish the current picture from other pictures in the same video sequence, i.e. the video sequence that also comprises the current picture, such as previously encoded and / or decoded pictures).

[0083] A (digital) picture is or can be considered as a two-dimensional array or matrix of samples with intensity values. A sample in the array can also be referred to as a pixel (short for picture element). The number of samples in the array or picture in horizontal and vertical direction (or axis) defines the size and / or resolution of the picture. For representing colors, usually 3 color components are employed, i.e. a picture can be represented as or can comprise 3 sample arrays. In RGB format or color space, a picture comprises corresponding arrays of red, green and blue samples. However, in video coding, each pixel is usually represented in a luma and chroma format or color space, such as YCbCr, comprising a luma component (sometimes also denoted as L) and 2 chroma components denoted as Cb and Cr. The luma component Y represents the luminance or gray scale intensity (which is the same in a gray scale picture, for example), while the 2 chroma components Cb and Cr represent the chrominance or color information components. Thus, a picture in YCbCr format comprises a luma sample array composed of luma sample values (Y) and 2 chroma sample arrays composed of chroma values (Cb and Cr). An RGB format picture can be converted or transformed into YCbCr format and vice versa. This process is also referred to as color transform or conversion. If the picture is black and white, the picture can comprise only a luma sample array. Accordingly, for example, a picture can be a luma sample array in black and white format or a luma sample array and 2 corresponding chroma sample arrays in 4:2:0, 4:2:2 and 4:4:4 color format.

[0084] Embodiments of the video encoder 20 can comprise an image partitioning unit (not shown in the figure) for partitioning the image 17 into a plurality of (typically non-overlapping) image blocks 203. These blocks can also be referred to as root blocks, macroblocks (in H.264 / AVC), or coding tree blocks (CTB) or coding tree units (CTU) (in H.265 / HEVC and VVC). The image partitioning unit can be configured to use the same block size for all images in a video sequence and a corresponding grid of defined block sizes, or to vary the block size between images or subsets or groups of images and to partition each image into a plurality of corresponding blocks. FIG. 2

[0085] In other embodiments, the video encoder can be configured to directly receive the blocks 203 in the image 17, e.g. one, several or all blocks constituting the image 17. The image blocks 203 can also be referred to as current image blocks or to-be-coded image blocks.

[0086] Similar to the image 17, the image blocks 203 are or can be regarded as two-dimensional arrays or matrices of samples consisting of intensity values (sample values), but the size of the image blocks 203 is smaller than the size of the image 17. In other words, depending on the applied color format, the blocks 203 can comprise, for example, one array of samples (e.g. an array of luma values in case of a monochrome image 17 or an array of luma values or two arrays of chroma values in case of a color image), or three arrays of samples (e.g. one array of luma values and two arrays of chroma values in case of a color image 17), or any other number and / or type of arrays. The number of samples in a block 203 in horizontal and vertical direction (or axis) defines the size of the block 203. Accordingly, a block can be an M x N (M columns x N rows) array of samples, or an M x N array of transform coefficients, etc.

[0087] FIG. 2 The illustrated embodiments of the video encoder 20 can be configured to encode the image 17 block-wise, e.g. encoding and prediction are performed on a block 203 basis.

[0088] FIG. 2 The illustrated embodiments of the video encoder 20 can also be configured to partition and / or encode the image by slices (also referred to as video slices). An image can be partitioned into or encoded by one or more (typically non-overlapping) slices, each of which can comprise one or more blocks (e.g. CTUs).

[0089] FIG. 2 ​The illustrated embodiment of video encoder 20 can also be configured to segment and / or encode a picture by tile groups (also referred to as video tile groups) and / or tiles (also referred to as video tiles). A picture can be segmented into and / or encoded by one or more (typically non-overlapping) tile groups; each tile group can comprise one or more blocks (e.g., CTUs) or one or more tiles, etc.; each tile can be rectangular, etc., and can comprise one or more complete or partial blocks (e.g., CTUs), etc.

[0090] Residual calculation

[0091] Residual calculation unit 204 can be configured to calculate a residual block 205 (also referred to as residual 205) from picture block 203 and prediction block 265 (more details of prediction block 265 are provided later) by, for example, subtracting sample values of prediction block 265 from sample values of picture block 203 on a sample-by-sample (pixel-by-pixel) basis.

[0092] Transform

[0093] Transform processing unit 206 can be configured to apply a transform, such as a discrete cosine transform (DCT) or a discrete sine transform (DST), to sample values of residual block 205 to obtain transform coefficients 207 in a transform domain. Transform coefficients 207 can also be referred to as transform residual coefficients and represent residual block 205 in the transform domain.

[0094] Transform processing unit 206 can be configured to apply an integer approximation of a DCT / DST, such as the transform specified for H.265 / HEVC. In contrast to an orthogonal DCT transform, such an integer approximation is typically scaled by a certain factor. In order to maintain the norm of a residual block that is processed by the forward and inverse transform, other scaling factors are applied as part of the transform process. The scaling factors are typically chosen according to certain constraints, such as being a power of two for shift operations, the bit depth of the transform coefficients, a tradeoff between precision and implementation cost, etc. For example, a specific scaling factor is specified for the inverse transform by inverse transform processing unit 212 (and a corresponding inverse transform by video decoder 30 by inverse transform processing unit 312); accordingly, encoder 20 can specify a corresponding scaling factor for the forward transform by transform processing unit 206.

[0095] Embodiments of video encoder 20 (respectively, transform processing unit 206) can be used to output transform parameters (e.g., type of one or more transforms) directly or after encoding or compression by entropy encoding unit 270, such that video decoder 30 can receive and use the transform parameters for decoding, and so forth.

[0096] Quantization

[0097] Quantization unit 208 can be used to quantize transform coefficients 207 by applying scalar quantization or vector quantization, and so forth, to obtain quantized coefficients 209. Quantized coefficients 209 can also be referred to as quantized transform coefficients 209 or quantized residual coefficients 209.

[0098] A quantization process can reduce the bit depth associated with some or all of transform coefficients 207. For example, n-bit transform coefficients can be rounded down to m-bit transform coefficients during quantization, where n is greater than m. The degree of quantization can be modified by adjusting a quantization parameter (QP). For example, for scalar quantization, different degrees of scaling can be performed to achieve finer or coarser quantization. A smaller quantization step size corresponds to finer quantization, while a larger quantization step size corresponds to coarser quantization. A suitable quantization step size can be represented by a quantization parameter (QP). For example, the quantization parameter can be an index to a set of predefined suitable quantization step sizes. For example, a smaller quantization parameter can correspond to fine quantization (smaller quantization step size), a larger quantization parameter can correspond to coarse quantization (larger quantization step size), and vice versa. Quantization can include division by the quantization step size, while corresponding dequantization and / or inverse quantization by inverse quantization unit 210, and so forth, can include multiplication by the quantization step size. Embodiments in accordance with some standards, such as HEVC, can be used to use a quantization parameter to determine a quantization step size. Generally, the quantization step size can be calculated from the quantization parameter by a fixed-point approximation of an equation that includes a division. An additional scaling factor can be introduced for quantization and dequantization to recover a norm of a residual block that can have been modified due to scaling used in the fixed-point approximation of the equation for the quantization step size and the quantization parameter. In an example implementation, scaling for inverse transform and dequantization can be combined. Alternatively, a custom quantization table can be used and signaled by the encoder to the decoder in a bitstream, and so forth. Quantization is a lossy operation, where the larger the quantization step size, the greater the loss.

[0099] Embodiments of video encoder 20 (respectively, quantization unit 208) can be used to output quantization parameters (QP) directly or after encoding by entropy encoding unit 270, such that video decoder 30 can receive and use the quantization parameters for decoding, and so forth.

[0100] Dequantization

[0101] The inverse quantization unit 210 is configured to apply inverse quantization of the quantized coefficients by the quantization unit 208 to obtain dequantized coefficients 211, e.g., by applying an inverse quantization scheme opposite to the quantization scheme applied by the quantization unit 208, e.g., according to or using the same quantization step size as the quantization unit 208. The dequantized coefficients 211 can also be referred to as dequantized residual coefficients 211, corresponding to the transform coefficients 207, but typically not identical to the transform coefficients due to quantization losses.

[0102] inverse transform

[0103] The inverse transform processing unit 212 is configured to apply an inverse transform opposite to the transform applied by the transform processing unit 206, e.g., an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST) or another inverse transform, to obtain a reconstructed residual block 213 (or corresponding dequantized coefficients 213) in the sample domain. The reconstructed residual block 213 can also be referred to as a transform block 213.

[0104] reconstruction

[0105] The reconstruction unit 214, e.g., a summer or adder 214, is configured to add the transform block 213, i.e., the reconstructed residual block 213, to the prediction block 265 to obtain a reconstructed block 215 in the sample domain, e.g., by adding sample values of the reconstructed residual block 213 to sample values of the prediction block 265 on a sample-by-sample basis.

[0106] filtering

[0107] The in-loop filter unit 220 (or simply “in-loop filter” 220) is configured to filter the reconstructed block 215 to obtain a filtered block 221, or generally to filter reconstructed samples to obtain filtered samples. For example, the in-loop filter unit is configured to smooth pixel transitions or otherwise improve video quality. The in-loop filter unit 220 can include one or more in-loop filters, e.g., a deblocking filter, a sample adaptive offset (SAO) filter, or one or more other filters, e.g., a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, a collaborative filter, or any combination thereof. Although the in-loop filter unit 220 is shown in FIG. 2 as an in-loop filter, in other configurations, the in-loop filter unit 220 can be implemented as a post-loop filter. The filtered block 221 can also be referred to as a filtered reconstructed block 221. FIG. 2

[0108] ​Embodiments of video encoder 20 (respectively, in-loop filter unit 220) can be configured to output loop filter parameters (e.g., sample adaptive offset information) directly or after encoding by entropy encoding unit 270, such that decoder 30 can receive and use the same loop filter parameters or corresponding loop filters for decoding, and so on.

[0109] decoded picture buffer

[0110] Decoded picture buffer (DPB) 230 can be a memory that stores reference pictures or generally stores reference picture data used for encoding video data by video encoder 20. DPB 230 can be formed of any of a plurality of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. Decoded picture buffer (DPB) 230 can be used to store one or more filtered blocks 221. Decoded picture buffer 230 can also be used to store other previously filtered blocks (e.g., previously filtered reconstructed blocks 221) in the same current picture or different pictures (e.g., previous reconstructed pictures), and can provide previously fully reconstructed (i.e., decoded) pictures (and corresponding reference blocks and samples) and / or partially reconstructed current pictures (and corresponding reference blocks and samples) for inter prediction, and so on. Decoded picture buffer (DPB) 230 can also be used to store one or more unfiltered reconstructed blocks 215 or generally unfiltered reconstructed samples, if any, if reconstructed blocks 215 are not filtered by in-loop filter unit 220, and so on, or to store any other blocks or samples resulting from further processing of reconstructed blocks or samples.

[0111] Mode selection (partitioning and prediction)

[0112] The mode selection unit 260 comprises a partitioning unit 262, an inter prediction unit 244 and an intra prediction unit 254 and is configured to receive or obtain original picture data, such as original blocks 203 (current blocks 203 in a current picture 17) and reconstructed picture data, such as filtered and / or unfiltered reconstructed samples or blocks in the same (current) picture and / or one or more previously decoded pictures, from a decoded picture buffer 230 or other buffers (e.g. line buffers, not shown in the figure) and the like. The reconstructed picture data is used as reference picture data for prediction, such as inter or intra prediction, to obtain prediction blocks 265 or prediction values 265.

[0113] The mode selection unit 260 can be configured to determine or select one partitioning and to determine or select one prediction mode (e.g. intra or inter prediction mode) for the current block prediction mode (including no partitioning) and to generate a corresponding prediction block 265 which is used for the calculation of the residual block 205 and for the reconstruction of the reconstructed block 215.

[0114] Embodiments of the mode selection unit 260 can be configured to select the partitioning and the prediction mode (e.g. from those modes supported or available for the mode selection unit 260). The partitioning and the prediction mode provide the best match or minimum residual (meaning better compression in transmission or storage) or the minimum signaling overhead (meaning better compression in transmission or storage) or a compromise or balance between both. The mode selection unit 260 can be configured to determine the partitioning and the prediction mode according to a rate distortion optimization (RDO), i.e. to select the prediction mode which provides the minimum rate distortion. The terms “best”, “minimum”, “optimal” and the like in the present text do not necessarily refer to an overall “best”, “minimum”, “optimal” and the like, but can also refer to a situation where a termination or selection criterion is met, e.g. a value which exceeds or is below a threshold or other constraints can lead to a “suboptimal selection”, but reduce complexity and processing time.

[0115] In other words, the partitioning unit 262 can be configured to partition a block 203 into smaller block partitions or sub-blocks (forming blocks again) by iteratively using a quad-tree (QT) partitioning, a binary-tree (BT) partitioning or a triple-tree (TT) partitioning or any combination thereof, for example, and to perform a prediction for each block partition or sub-block, wherein the mode selection comprises selecting the tree structure which partitions the block 203 and the prediction mode is applied to each block partition or sub-block.

[0116] Partitioning (e.g., performed by partitioning unit 260) and prediction processing (performed by inter prediction unit 244 and intra prediction unit 254) performed by example video encoder 20 are described in detail below.

[0117] Partitioning

[0118] Partitioning unit 262 can partition (or divide) current block 203 into smaller partitions, e.g., square or rectangular sub-blocks. These sub-blocks (also referred to as sub-blocks) can be further partitioned into even smaller partitions. This is also referred to as tree partitioning or hierarchical tree partitioning. A root block at root tree level 0 (hierarchy level 0, depth 0) can be recursively partitioned into two or more blocks of a next lower tree level, e.g., nodes of tree level 1 (hierarchy level 1, depth 1). These blocks can be again partitioned into two or more blocks of a next lower level, e.g., tree level 2 (hierarchy level 2, depth 2) and so on until the partitioning ends (as end criteria are met, e.g., a maximum tree depth is reached or a minimum block size is reached). Blocks that are not further partitioned are also referred to as leaf blocks or leaf nodes of the tree. A tree that is partitioned into two partitions is referred to as a binary-tree (BT), a tree that is partitioned into 3 partitions is referred to as a ternary-tree (TT), and a tree that is partitioned into 4 partitions is referred to as a quad-tree (QT).

[0119] As described above, the term “block” as used herein can be a portion of an image, in particular a square or rectangular portion. With reference to HEVC and VVC, a block can be or can correspond to a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), and a transform unit (TU), and / or to a plurality of corresponding blocks, e.g., a coding tree block (CTB), a coding block (CB), a transform block (TB), or a prediction block (PB).

[0120] For example, a coding tree unit (CTU) can be or include 1 CTB of luma samples in a picture with 3 sample arrays and 2 corresponding CTBs of chroma samples in the picture, or can be or include 1 CTB of samples in a monochrome picture or a picture coded using 3 separate color planes and syntax structures for coding the samples. The syntax structures are used to code the samples. Correspondingly, a coding tree block (CTB) can be a block of NxN samples, where N can be set to a value such that one component is divided into multiple CTBs, which is one way of partitioning. A coding unit (CU) can be or include 1 coding block of luma samples in a picture with 3 sample arrays and 2 corresponding coding blocks of chroma samples in the picture, or can be or include 1 coding block of samples in a monochrome picture or a picture coded using 3 separate color planes and syntax structures for coding the samples. The syntax structures are used to code the samples. Correspondingly, a coding block (CB) can be a block of MxN samples, where M and N can be set to a value such that one CTB is divided into multiple coding blocks, which is one way of partitioning.

[0121] In an embodiment, for example according to HEVC, a coding tree unit (CTU) can be divided into CUs by a quad-tree structure denoted as a coding tree. Whether to code a picture region using inter (temporal) prediction or intra (spatial) prediction is decided at the CU level. Each CU can be further divided into 1, 2, or 4 PUs according to the PU partition type. The same prediction process is performed within one PU, and information related to the PU is sent to the decoder in units of PUs. After a residual block is obtained by the prediction process according to the PU partition type, the CU can be partitioned into transform units (TUs) according to other quad-tree structures similar to the coding tree of the CU.

[0122] In an embodiment, a combined quad-tree and binary tree (QTBT) partitioning is used for partitioning coding blocks, e.g., in accordance with the latest video coding standard under development, referred to as Versatile Video Coding (VVC), etc. In the QTBT block structure, a CU can be square or rectangular. For example, a coding tree unit (CTU) is first partitioned by a quad-tree structure. Quad-tree leaf nodes are further partitioned by a binary or ternary tree structure. The partitioned tree leaf nodes are referred to as coding units (CUs), and this segmentation is used for prediction and transform processing without any further partitioning. This means that the CU, PU, and TU have the same block size in the QTBT coding block structure. Meanwhile, multiple partitioning such as ternary tree partitioning can be used with the QTBT block structure.

[0123] In one example, the mode selection unit 260 in video encoder 20 can be configured to perform any combination of the partitioning techniques described herein.

[0124] As described above, video encoder 20 is configured to determine or select a best or optimal prediction mode from a set of (e.g., predetermined) prediction modes. The set of prediction modes can include intra-prediction modes and / or inter-prediction modes, etc.

[0125] Intra-prediction

[0126] The set of intra-prediction modes can include 35 different intra-prediction modes, e.g., the DC (or mean) mode and the planar mode defined in HEVC and other non-directional modes or directional modes, or can include 67 different intra-prediction modes, e.g., the DC (or mean) mode and the planar mode defined in VVC and other non-directional modes or directional modes.

[0127] The intra-prediction unit 254 is configured to generate the intra-predicted block 265 using reconstructed samples of neighboring blocks in the same current picture according to an intra-prediction mode from the set of intra-prediction modes.

[0128] The intra-prediction unit 254 (or generally the mode selection unit 260) is also configured to output the intra-prediction parameters (or generally information representative of the selected intra-prediction mode for the block) in the form of syntax elements 266 to the entropy encoding unit 270 for inclusion in the encoded picture data 21 such that the video decoder 30 can receive and use the prediction parameters for decoding, etc.

[0129] Inter-prediction

[0130] The (possible) set of inter prediction modes depends on the available reference pictures (i.e. at least some of the decoded pictures stored in the DPB 230 or the like as described above) and other inter prediction parameters, e.g. on whether the entire reference picture or only a part of the reference picture (e.g. a search window region around the region of the current block) is used for searching for the best matching reference block, and / or e.g. on whether or not pixel interpolation is performed, e.g. half / pel interpolation and / or quarter pel interpolation.

[0131] In addition to the above-mentioned prediction modes, a skip mode and / or a direct mode can be used.

[0132] The inter prediction unit 244 can comprise a motion estimation (ME) unit and a motion compensation (MC) unit (both not shown in Fig. 2). The motion estimation unit can be configured to receive or obtain the picture block 203 (current picture block 203 in the current picture 17) and the decoded picture 231 or at least one or more previous reconstructed blocks (e.g. reconstructed blocks in one or more other / different previous decoded pictures 231) for motion estimation. For example, the video sequence can comprise the current picture and the previous decoded picture 231 or, in other words, the current picture and the previous decoded picture 231 can be part of or constitute a picture sequence which constitutes the video sequence. FIG. 3 For example, the encoder 20 can be configured to select one reference block from a plurality of reference blocks of the same or different pictures of a plurality of other pictures and provide the reference picture (or reference picture index) and / or an offset (spatial offset) between the position of the reference block and the position of the current block as inter prediction parameters to the motion estimation unit. Such an offset is also referred to as motion vector (MV).

[0133] The motion compensation unit is configured to obtain (e.g. receive) the inter prediction parameters and perform inter prediction in accordance with or using the inter prediction parameters to obtain the inter predicted block 265. The motion compensation performed by the motion compensation unit can comprise extracting or generating the prediction block in accordance with the motion / block vectors determined by the motion estimation and can further comprise performing interpolation to reach sub-pixel accuracy. Interpolation filtering can generate further pixel samples from known pixel samples, thereby potentially increasing the number of candidate prediction blocks which can be used for coding the picture block. Upon receiving the motion vector for the PU of the current picture block, the motion compensation unit can locate the prediction block pointed to by the motion vector in one of the reference picture lists.

[0134]

[0135] ​The motion compensation unit can also generate syntax elements related to the blocks and the video slice for use by the video decoder 30 in decoding the blocks of the picture in the video slice. In addition to or instead of slices and corresponding syntax elements, tile groups and / or tiles and corresponding syntax elements can be generated or used.

[0136] Entropy encoding

[0137] The entropy encoding unit 270 is configured to apply an entropy encoding algorithm or scheme, e.g. variable length coding (VLC) scheme, context adaptive VLC (CAVLC) scheme, arithmetic coding scheme, binarization, context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy encoding methods or techniques, or none (no compression) to the quantized coefficients 209, inter prediction parameters, intra prediction parameters, loop filter parameters, and / or other syntax elements, to obtain encoded picture data 21 that can be output via output terminal 272 in the form of an encoded bitstream 21, such that the video decoder 30 can receive and use these parameters for decoding, etc. The encoded bitstream 21 can be transmitted to the video decoder 30 or stored in a memory for later transmission or retrieval by the video decoder 30.

[0138] Other structural variations of the video encoder 20 can be used to encode a video stream. For example, a non-transform-based encoder 20 can directly quantize the residual signal for certain blocks or frames without the transform processing unit 206. In another implementation, the encoder 20 can include the quantization unit 208 and the inverse quantization unit 210 combined into a single unit.

[0139] Decoder and decoding method

[0140] FIG. 3An example of a video decoder 30 for implementing the techniques in this application. The video decoder 30 is configured to receive, e.g., encoded picture data 21 (e.g., an encoded bitstream 21) encoded by the encoder 20, to obtain decoded pictures 331. The encoded picture data or bitstream includes information for decoding the encoded picture data, e.g., data representing image blocks of encoded video slices (and / or tile groups or tiles) and associated syntax elements.

[0141] In FIG. 2 In an example, the decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., a summer 314), an in-loop filter 320, a decoded picture buffer (DPB) 330, a mode application unit 360, an inter prediction unit 344, and an intra prediction unit 354. The inter prediction unit 344 can be or include a motion compensation unit. In some examples, the video decoder 30 can perform a decoding process generally reciprocal to the encoding process described with reference to the video encoder 100 in FIG. 3

[0142] As described with reference to the encoder 20, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the in-loop filter 220, the decoded picture buffer (DPB) 230, the inter prediction unit 344, and the intra prediction unit 354 also constitute a “built-in decoder” of the video encoder 20. Accordingly, the inverse quantization unit 310 can be identical in functionality to the inverse quantization unit 110, the inverse transform processing unit 312 can be identical in functionality to the inverse transform processing unit 212, the reconstruction unit 314 can be identical in functionality to the reconstruction unit 214, the in-loop filter 320 can be identical in functionality to the in-loop filter 220, and the decoded picture buffer 330 can be identical in functionality to the decoded picture buffer 230. Thus, explanations made with respect to the corresponding units and functionalities of the video encoder 20 apply correspondingly to the corresponding units and functionalities of the video decoder 30.

[0143] Entropy decoding

[0144] The entropy decoding unit 304 is configured to parse the bitstream 21 (or generally referred to as the encoded picture data 21) and perform entropy decoding and the like on the encoded picture data 21 to obtain quantized coefficients 309 and / or decoded encoding parameters (e.g., motion vectors, block shapes, prediction modes, etc.). FIG. 3 ​inter prediction parameters (e.g., reference picture indices and motion vectors), intra prediction parameters (e.g., intra prediction modes or indices), transform parameters, quantization parameters, loop filter parameters, and / or any or all of the other syntax elements. Entropy decoding unit 304 can be configured to apply a decoding algorithm or scheme corresponding to the encoding schemes described with respect to entropy encoding unit 270 in encoder 20. Entropy decoding unit 304 can also be configured to provide inter prediction parameters, intra prediction parameters, and / or other syntax elements to mode application unit 360, and other parameters to other units in decoder 30. Video decoder 30 can receive syntax elements at the video slice level and / or the video block level. In addition to, or instead of, slices and corresponding syntax elements, tile groups and / or tiles and corresponding syntax elements can be received and / or used.

[0145] dequantization

[0146] Dequantization unit 310 can be configured to receive quantization parameters (QPs) (or generally information related to dequantization) and quantized coefficients from encoded image data 21 (e.g., parsed and / or decoded by entropy decoding unit 304, etc.), and dequantize the decoded quantized coefficients 309 according to these quantization parameters to obtain dequantized coefficients 311. Dequantized coefficients 311 can also be referred to as transform coefficients 311. The dequantization process can include determining a degree of quantization using quantization parameters determined by video encoder 20 for each video block in a video slice (or tile or tile group), and likewise determining a degree of dequantization to apply.

[0147] inverse transform

[0148] Inverse transform processing unit 312 can be configured to receive dequantized coefficients 311 (also referred to as transform coefficients 311), and apply a transform to dequantized coefficients 311 to obtain a reconstructed residual block 313 in the sample domain. Reconstructed residual block 313 can also be referred to as transform block 313. The transform can be an inverse transform, such as an inverse DCT, an inverse DST, an inverse integer transform, or a conceptually similar inverse transform process. Inverse transform processing unit 312 can also be configured to receive transform parameters or corresponding information from encoded image data 21 (e.g., parsed and / or decoded by entropy decoding unit 304, etc.) to determine the transform to apply to dequantized coefficients 311.

[0149] reconstruction

[0150] Reconstruction unit 314 (e.g., adder or summer 314) can be configured to add reconstructed residual block 313 to prediction block 365 to obtain a reconstructed block 315 in the sample domain by, for example, adding sample values of reconstructed residual block 313 to sample values of prediction block 365.

[0151] filtering

[0152] The in-loop filter unit 320 (in the coding loop or after) is configured to filter the reconstructed block 315 to obtain a filtered block 321 to facilitate smooth pixel transitions or otherwise improve video quality, etc. The in-loop filter unit 320 can include one or more in-loop filters, e.g., a deblocking filter, a sample adaptive offset (SAO) filter, or one or more other filters, e.g., a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, a collaborative filter, or any combination thereof. Although the in-loop filter unit 320 is shown as an in-loop filter in FIG. 3 , in other configurations, the in-loop filter unit 320 can be implemented as a post-loop filter.

[0153] decoded picture buffer

[0154] The decoded video blocks 321 in one picture are then stored in the decoded picture buffer 330, which stores decoded pictures 331 as reference pictures for subsequent motion compensation of other pictures and / or for output or display.

[0155] The decoder 30 is configured to output the decoded pictures 331 via an output terminal 332 and / or the like for presentation to and / or viewing by a user.

[0156] prediction

[0157] The inter prediction unit 344 can be identical in functionality to the inter prediction unit 244 (in particular to the motion compensation unit), and the intra prediction unit 354 can be identical in functionality to the intra prediction unit 254, and perform partitioning or segmentation decisions and perform predictions in accordance with partitioning and / or prediction parameters or respective information received from the encoded picture data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304 or the like). The mode application unit 360 can be configured to perform predictions (intra or inter predictions) on a block-by-block basis in accordance with reconstructed pictures, blocks, or respective samples (filtered or unfiltered) to obtain predicted blocks 365.

[0158] When a video slice is coded as an intra coded (I) slice, intra prediction unit 354 in mode application unit 360 is used to generate prediction blocks 365 for blocks of the current video slice from the indicated intra prediction mode and data from previously decoded blocks in the current image. When a video slice is coded as an inter coded (i.e., B or P) slice, inter prediction unit 344 (e.g., a motion compensation unit) in mode application unit 360 is used to generate prediction blocks 365 for video blocks in the current video slice from the motion vectors and other syntax elements received from entropy decoding unit 304. For inter prediction, these prediction blocks can be produced from one of the reference pictures in one of the reference picture lists. Video decoder 30 can construct reference picture list 0 and list 1 using default construction techniques from reference pictures stored in DPB 330. In addition to or instead of slices (e.g., video slices), the same or similar processes can be applied to or by embodiments that use tile groups (e.g., video tile groups) and / or tiles (e.g., video tiles), e.g., a video can be coded using I, P, or B tile groups and / or tiles.

[0159] Mode application unit 360 is used to determine prediction information for video blocks in the current video slice by parsing the motion vectors or related information and other syntax elements, and use the prediction information to generate prediction blocks for the current video blocks being decoded. For example, mode application unit 360 uses some of the syntax elements received to determine the prediction mode (e.g., intra or inter prediction) used to code the video blocks in the video slice, the inter prediction slice type (e.g., B slice, P slice, or GPB slice), the construction information for one or more reference picture lists for the slice, the motion vectors for each inter coded video block of the slice, the inter prediction status for each inter coded video block of the slice, and other information to decode the video blocks in the current video slice. In addition to or instead of slices (e.g., video slices), the same or similar processes can be applied to or by embodiments that use tile groups (e.g., video tile groups) and / or tiles (e.g., video tiles), e.g., a video can be coded using I, P, or B tile groups and / or tiles.

[0160] FIG. 3 The illustrated embodiment of video decoder 30 can be used to partition and / or decode an image by slices (also referred to as video slices). One image can be partitioned into or decoded by one or more (typically non-overlapping) slices, each of which can include one or more blocks (e.g., CTUs).

[0161] FIG. 4Embodiments of the illustrated video decoder 30 can be used to segment and / or decode an image by tile groups (also referred to as video tile groups) and / or tiles (also referred to as video tiles). One image can be segmented into and / or decoded by one or more (typically non-overlapping) tile groups; each tile group can include one or more blocks (e.g., CTUs) or one or more tiles, etc.; each tile can be rectangular, etc., and can include one or more complete or partial blocks (e.g., CTUs), etc.

[0162] Other variations of the video decoder 30 can be used to decode the encoded image data 21. For example, the decoder 30 can generate the output video stream without the loop filter unit 320. For example, a non-transform-based decoder 30 can directly dequantize the residual signal for certain blocks or frames without the inverse transform processing unit 312. In another implementation, the video decoder 30 can include the inverse quantization unit 310 and the inverse transform processing unit 312 combined into a single unit.

[0163] It should be understood that in the encoder 20 and the decoder 30, the processing result of the current step can be further processed and then output to the next step. For example, after the interpolation filtering, the motion vector derivation, or the loop filtering, more operations such as clipping or shifting can be performed on the processing result of the interpolation filtering, the motion vector derivation, or the loop filtering.

[0164] It should be noted that more operations can be applied to the derived motion vectors (including but not limited to the control point motion vectors of the affine mode, the sub-block motion vectors of the affine mode, the planar mode, the ATMVP mode, the temporal motion vector, etc.) of the current block. For example, the value of the motion vector is constrained within a predefined range according to the representation bits of the motion vector. If the representation bits of the motion vector are bitDepth, the range is –2^(bitDepth–1) ~ 2^(bitDepth–1) – 1, where “^” represents the power. For example, if the bitDepth is set to 16, the range is –32768 ~ 32767; if the bitDepth is set to 18, the range is –131072 ~ 131071. For example, the value of the derived motion vector (e.g., the MVs of 4 4x4 sub-blocks in one 8x8 block) is constrained such that the maximum difference between the integer parts of the 4 4x4 sub-block MVs does not exceed N pixels, e.g., does not exceed 1 pixel. Two methods of constraining the motion vector according to the bitDepth are provided herein.

[0165] Method 1: Remove the most significant bits (MSBs) by smoothing operation

[0166] ux = (mvx + 2 bitDepth ) % 2 bitDepth (1)

[0167] mvx = (ux >= 2 bitDepth-1 )? (ux - 2 bitDepth ) : ux (2)

[0168] uy = (mvy + 2 bitDepth ) % 2 bitDepth (3)

[0169] mvy = (uy >= 2 bitDepth-1 )? (uy - 2 bitDepth ) : uy (4)

[0170] where mvx is the horizontal component of the motion vector of a picture block or sub-block, mvy is the vertical component of the motion vector of a picture block or sub-block, and ux and uy represent intermediate values.

[0171] For example, if the value of mvx is -32769, the value obtained after using equations (1) and (2) is 32767. In computer systems, decimal numbers are stored in the form of two's complement. The two's complement of -32769 is 1,0111,1111,1111,1111 (17 bits), and then the MSB is discarded, thus obtaining 0111,1111,1111,1111 (the decimal number is 32767), which is the same as the output obtained after using equations (1) and (2).

[0172] ux = (mvpx + mvdx + 2 bitDepth ) % 2 bitDepth (5)

[0173] mvx = (ux >= 2 bitDepth-1 )? (ux - 2 bitDepth ) : ux (6)

[0174] uy = (mvpy + mvdy + 2 bitDepth ) % 2 bitDepth (7)

[0175] mvy = (uy >= 2 bitDepth-1 )? (uy - 2 bitDepth ) : uy (8)

[0176] These operations can be performed in the process of summing mvp and mvd, as shown in equations (5) to (8).

[0177] Method 2: Removing the overflow MSB by clipping the value

[0178] vx = Clip3(-2 bitDepth-1 , 2 bitDepth-1 -1, vx)

[0179] vy = Clip3(-2 bitDepth-1 , 2 bitDepth-1 -1, vy)

[0180] where vx is the horizontal component of a motion vector of a picture block or sub-block, vy is the vertical component of the motion vector of the picture block or sub-block, x, y and z correspond to the 3 input values of the MV clipping process respectively, and the function Clip3 is defined as follows:

[0181]

[0182] FIG. 1A A schematic diagram of a video coding device 400 is provided for an embodiment of the present application. The video coding device 400 is suitable for implementing the disclosed embodiments described herein. In one embodiment, the video coding device 400 can be a decoder (e.g., the video decoder 30 in FIG. 1) or an encoder (e.g., the video encoder 20 in FIG. 1). FIG. 1A FIG. 5

[0183] The video coding device 400 includes an ingress port 410 (or input port 410) and a receive unit (Rx) 420 for receiving data, a processor, logic unit or central processing unit (CPU) 430 for processing the data, a transmit unit (Tx) 440 and an egress port 450 (or output port 450) for transmitting the data, and a memory 460 for storing the data. The video coding device 400 can further include optical-to-electrical (OE) components and electrical-to-optical (EO) components coupled to the ingress port 410, the receive unit 420, the transmit unit 440 and the egress port 450 for use as an egress or an ingress of optical or electrical signals.

[0184] ​​The processor 430 is implemented by hardware and software. The processor 430 can be implemented as one or more CPU chips, one or more cores (e.g., a multi-core processor), one or more FPGAs, one or more ASICs, and one or more DSPs. The processor 430 is in communication with the ingress port 410, the receiving unit 420, the sending unit 440, the egress port 450, and the memory 460. The processor 430 includes a coding module 470. The coding module 470 implements the disclosed embodiments described above. For example, the coding module 470 performs, processes, prepares, or provides various coding operations. Thus, inclusion of the coding module 470 provides substantial improvement to the functionality of the video coding device 400 and affects the video coding device 400’s ability to transition between states. The coding module 470 is optionally implemented in instructions stored in the memory 460 and executed by the processor 430.

[0185] The memory 460 can include one or more disks, one or more tape drives, and one or more solid-state drives, and can serve as an over-flow data storage device, to store programs when such programs are selected for execution, and to store instructions and data read from a program as such program is executed. The memory 460 can be volatile and / or non-volatile, and can be read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and / or static random-access memory (SRAM), for example.

[0186] FIG. 1A A simplified block diagram of an apparatus 500 is provided for an example embodiment. The apparatus 500 can be used as a source device 12 and / or a destination device 14 in a system 10. http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 17_Brussels /

[0187] The processor 502 in the apparatus 500 can be a central processing unit. Alternatively, the processor 502 can be any other type of device, either existing or developed in the future, capable of manipulating or processing information. While the disclosed implementations can be practiced with a single processor, such as the processor 502 as shown, the use of multiple processors can increase speed and efficiency.

[0188] ​In one implementation, the memory 504 in the apparatus 500 can be a read only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of memory device can be used as the memory 504. The memory 504 can include code and data 506 that is accessed by the processor 502 via the bus 512. The memory 504 can also include an operating system 508 and an application program 510 including at least one program that, when executed by the processor 502, enables the processor 502 to perform the methods described herein. For example, the application program 510 can include an application 1 through an application N, which includes a video coding application that performs the methods described herein.

[0189] The apparatus 500 can also include one or more output devices, such as a display 518. In one example, the display 518 can be a touch-sensitive display that combines a display with a touch-sensitive element that can be used to sense touch inputs. The display 518 can be coupled to the processor 502 via the bus 512.

[0190] Although the bus 512 in the apparatus 500 is described herein as a single bus, the bus 512 can include multiple buses. In addition, the secondary storage 514 can be directly coupled to the other components in the apparatus 500 or can be accessed via a network, and can include a single integrated unit (such as one memory card) or multiple units (such as multiple memory cards). Thus, the apparatus 500 can be implemented in a variety of configurations.

[0191] Parameter Set

[0192] The parameter sets are basically similar and have the same basic design goal, which is to improve the code rate, enhance error resilience, and provide a system layer interface. The parameter sets in HEVC (H.265) are hierarchical, including a video parameter set (VPS), a sequence parameter set (SPS), and a picture parameter set (PPS), which are similar to the corresponding parameter sets in AVC and VVC. Each slice refers to an activated PPS, SPS, and VPS to access the information used to decode the slice. The PPS includes information applicable to all slices in a picture, so all slices in a picture must refer to the same PPS. Slices in different pictures can also refer to the same PPS. Similarly, the SPS includes information applicable to all pictures in the same coded video sequence (CVS).

[0193] While the PPS corresponding to different pictures can be different, many or all pictures in a coded video sequence typically refer to the same PPS. Reuse of parameter sets improves the coding rate because it avoids sending common information multiple times. It is also robust because the parameter set content can be carried over a more reliable out-of-band communication link or it can be repeated frequently in the bitstream to ensure that it is not lost.

[0194] Parameter Set

[0195] The parameter sets are largely similar and have the same basic design goals, namely to improve coding rate, to enhance error resilience, and to provide a system level interface. The parameter sets in HEVC (H.265) are hierarchical and include a video parameter set (VPS), a sequence parameter set (SPS), and a picture parameter set (PPS), which are similar to the corresponding parameter sets in AVC and VVC. Each slice refers to an active PPS, SPS, and VPS to access information used to decode the slice. A PPS includes information that applies to all slices in a picture and thus all slices in a picture must refer to the same PPS. Slices in different pictures can also refer to the same PPS. Similarly, an SPS includes information that applies to all pictures in a coded video sequence.

[0196] While the PPS corresponding to different pictures can be different, many or all pictures in a coded video sequence typically refer to the same PPS. Reuse of parameter sets improves the coding rate because it avoids sending common information multiple times. It is also robust because the parameter set content can be carried over a more reliable out-of-band communication link or it can be repeated frequently in the bitstream to ensure that it is not lost.

[0197] Sequence Parameter Set (SPS)

[0198] An SPS includes parameters that apply to one or more layers in a coded video sequence and does not change as pictures in the coded video sequence change. In particular, an SPS includes information for subpicture indication.

[0199] Some parts of the following table show a snapshot of a part of the subpicture indication in the SPS in ITU JVET-Q2001-vl l, which can be downloaded at the following link:

[0200] FIG. 6 FIG. 7In the remainder of this application, for simplicity, the prior art document is referred to as VVC Draft 8.

[0201]

[0202]

[0203] Some syntax elements in SPS indicate the position information and control flags of each subpicture. The position information of the i-th subpicture includes:

[0204] • subpic_ctu_top_left_x[ i ], which indicates the horizontal component of the top-left coordinate of the subpicture i in the picture; or

[0205] • subpic_ctu_top_left_y[ i ], which indicates the vertical component of the top-left coordinate of the subpicture i in the picture; or

[0206] • subpic_width_minus1[ i ], which indicates the width of the subpicture i in the picture; or

[0207] • subpic_height_minus1[ i ], which indicates the height of the subpicture i in the picture.

[0208] Some syntax elements indicate the number of subpictures within a picture, such as sps_num_subpics_minus1.

[0209] A picture is divided into one or more tile rows and one or more tile columns. A tile is a sequence of CTUs that covers a rectangular region of the picture. The CTUs in a tile are scanned in the tile in the raster scan order.

[0210] A slice includes an integer number of complete tiles in the picture or an integer number of consecutive complete CTU rows within a tile in the picture. Therefore, each vertical slice boundary is always also a vertical tile boundary. The horizontal boundaries of a slice can not be tile boundaries, but include horizontal CTU boundaries within a tile; this occurs when a tile is divided into multiple rectangular slices, each of which includes an integer number of consecutive complete CTU rows within the tile.

[0211] Two slice modes are supported, namely, the raster-scan slice mode and the rectangular slice mode. In the raster-scan slice mode, a slice includes a series of complete tiles in the picture in the tile raster scan order. In the rectangular slice mode, a slice includes multiple complete tiles that collectively form a rectangular region in the picture; or a slice includes multiple consecutive complete CTU rows in a tile that collectively form a rectangular region in the picture. The tiles within a rectangular slice are scanned in the rectangular region corresponding to the slice in the tile raster scan order.

[0212] A sub-picture comprises one or more slices that together cover a rectangular region of the picture. Hence, each sub-picture boundary is always also a slice boundary, and each vertical sub-picture boundary is always also a vertical tile boundary.

[0213] Each sub-picture and tile shall satisfy one or both of the following conditions:

[0214] - all CTUs in a sub-picture belong to the same tile;

[0215] - all CTUs in a tile belong to the same sub-picture.

[0216] Splitting the picture into CTUs, slices, tiles and sub-pictures.

[0217] Splitting the picture into CTUs:

[0218] The picture is split into a series of coding tree units (CTUs). The term coding tree block (CTB) is sometimes used interchangeably. The concept of CTU is the same as in HEVC. For a picture with 3 sample arrays, 1 CTU includes 1 NxN block of luma samples and 2 corresponding blocks of chroma samples. FIG. 7 An example of splitting a picture into CTUs is shown. The size of the CTUs must be the same within a frame, except for CTUs located at picture boundaries (there can be incomplete CTUs).

[0219] Splitting the picture into tiles:

[0220] When tiles are used, the picture is split into rectangular groups of CTUs separated by vertical and / or horizontal boundaries. The vertical and horizontal tile boundaries intersect the picture from top to bottom and from the left picture boundary to the right picture boundary, respectively. Indications related to the position of said horizontal and vertical tile boundaries are included in the bitstream.

[0221] FIG. 7 Splitting a picture into 9 tiles is illustrated by way of example. In this example, the tile boundaries are marked with bold dashed lines. In other words, FIG. 8 A tile-based raster scan order of the CTUs of a picture with 9 tiles of different sizes is shown. Note that the tile boundaries are marked with bold dashed lines.

[0222] When there is more than 1 tile within a picture, the scan order of the CTUs is changed. The rules to scan the CTUs are as follows:

[0223] 1. The sub-blocks are scanned from left to right, from top to bottom in raster scan order, referred to as sub-block scan order in this application. This means that starting from the top-left sub-block, all sub-blocks in the same sub-block row are scanned from left to right first. Then, starting from the first sub-block in the second sub-block row (the next sub-block row), all sub-blocks in the second sub-block row are scanned from left to right. The above process is repeated until all sub-blocks are scanned.

[0224] 2. Within a sub-block, the CTUs are scanned in raster scan order. Within a CTU row, the CTUs are scanned from left to right, and the CTU rows are scanned from top to bottom. FIG. 8 An example is shown to illustrate the scan order of CTUs when sub-blocks exist. The numbers within the CTU represent the scan order.

[0225] Sub-blocks can enable image partitioning such that each sub-block can be decoded independently from other sub-blocks in the same image, where decoding refers to entropy decoding, residual decoding and prediction decoding. In addition, using sub-blocks can partition an image into regions of similar size. Thus, sub-blocks in an image can be processed in parallel with each other, which is suitable for multi-core processing environments where each processing core is identical.

[0226] The terms "processing order" and "scan order" are used in this application as follows:

[0227] Processing refers to encoding or decoding a CTU in an encoder or decoder. Scan order refers to indexing a particular partition within an image. The CTU scan order in a sub-block refers to how the CTUs within the sub-block are indexed, which can be different from their processing order.

[0228] An image is partitioned into slices:

[0229] Slices can enable image partitioning such that each slice can be decoded independently from other slices in the same image, where decoding refers to entropy decoding, residual decoding and prediction decoding. The difference from sub-blocks is that slices can have more arbitrary shapes (there are more partitioning possibilities), and the purpose of slice partitioning is not parallel processing, but rather to enhance error resilience and improve data packet size matching in transmission environments.

[0230] A slice can include a complete image and parts of it. In HEVC, a slice includes a number of consecutive CTUs in an image arranged in processing order. A slice is identified by its starting CTU address, which is indicated in the slice header or in the picture parameter set or other units.

[0231] In VVC Draft 8, a slice includes an integer number of complete tiles in a picture or an integer number of consecutive CTU rows within a tile in a picture. Therefore, each vertical slice boundary is always also a vertical tile boundary. A horizontal slice boundary can not be a tile boundary, but includes horizontal CTU boundaries within a tile; this happens when a tile is divided into multiple rectangular slices, each of which includes an integer number of consecutive complete CTU rows within the tile.

[0232] In some examples, there are two slice modes, i.e., raster scan slice mode and rectangular slice mode. In the raster scan slice mode, a slice includes a series of tiles in a picture in tile raster scan order. In the rectangular slice mode, a slice includes multiple tiles that together form a rectangular region in a picture; or, a slice includes multiple consecutive CTU rows in a tile that together form a rectangular region in a picture. The tiles within a rectangular slice are scanned in tile raster scan order within the rectangular region corresponding to the slice.

[0233] All slices in a picture together form the entire picture, i.e., all CTUs in a picture must be included in one of the slices in the picture. Similar rules apply to tiles and sub-pictures.

[0234] A picture is partitioned into sub-pictures:

[0235] A sub-picture is a rectangular partition in a picture. A sub-picture can be the entire picture or a part of the picture. Sub-pictures enable picture partitioning such that each sub-picture can be independently decodable from other sub-pictures in the entire video sequence. In VVC Draft 8, this also applies when indicated in the bitstream. That is, when subpic_treated_as_pic_flag[i] is indicated as true for sub-picture i, the sub-picture i can be independently decodable from other sub-pictures in the entire video sequence.

[0236] The difference between a sub-picture and a tile or a slice is that a sub-picture results in an independently decodable video sequence within a video sequence. For tiles and slices, independent decoding is only guaranteed within a single picture of a video sequence.

[0237] In VVC Draft 8, a sub-picture includes one or more slices that together cover a rectangular region in a picture. Therefore, each sub-picture boundary is always a slice boundary, and each vertical sub-picture boundary is always a vertical tile boundary.

[0238] FIG. 8 An example of tiles, slices, and sub-pictures is provided. In other words, FIG. 9 An example is shown in which a picture includes 4 tiles, i.e., 2 tile columns and 2 tile rows, 4 rectangular slices, and 3 sub-pictures. Sub-picture 1 includes 2 slices.

[0239] In FIG. 9 In the example shown, the picture is partitioned into 216 CTUs, 4 tiles, 4 slices, and 3 subpictures. The value of sps num subpics minusl is 2, and the position-dependent syntax elements have the following values:

[0240] For subpicture 0,

[0241] • subpic ctu top left x[0] is not signaled but inferred to be 0;

[0242] • subpic ctu top left y[0] is not signaled but inferred to be 0;

[0243] • The value of subpic width minusl [0] is 8;

[0244] • The value of subpic height minusl [0] is 11.

[0245] For subpicture 1,

[0246] • The value of subpic ctu top left x[1] is 9;

[0247] • The value of subpic ctu top left y[1] is 0;

[0248] • The value of subpic width minusl [1] is 8;

[0249] • The value of subpic height minusl [1] is 5.

[0250] For subpicture 2,

[0251] • The value of subpic ctu top left x[2] is 9;

[0252] • The value of subpic ctu top left y[2] is 6;

[0253] • subpic width minusl [2] is not signaled but inferred to be 8;

[0254] • subpic height minusl [2] is not signaled but inferred to be 5.

[0255] The tile

[0256] The following table illustrates the indication of tile size and coordinates within a tile, from the picture parameter set RBSP syntax table in VVC Draft 8.

[0257]

[0258] The partitioning information of tiles (address and size of each tile) is usually included in the parameter set. In the example above, an indication (no_pic_partition_flag) is first included in the bitstream to indicate whether the picture is partitioned into slices and tiles. If the indication is true (meaning the picture is not partitioned into slices or tiles), it is inferred that the picture is partitioned into only one slice and only one tile, whose boundaries are aligned with the picture boundaries. Otherwise (no_pic_partition_flag is false), the tile partitioning information is included in the bitstream.

[0259] The syntax element tile_column_width_minus1[ i ] specifies the width of the i-th tile column. The syntax element tile_row_height_minus1[ i ] specifies the height of the i-th tile row.

[0260] The tile row height and tile column width can be explicitly indicated in the bitstream or inferred. The syntax elements num_exp_tile_columns_minus1 and num_exp_tile_rows_minus1 specify the number of tile columns and tile rows whose width and height are explicitly indicated, respectively. The width and height of the remaining tile columns and tile rows are inferred according to the function

[0261] The index of a tile is in the “tile scan order in picture”. The tiles in a picture are ordered (scanned) in a raster scan order, with the first tile in the top-left corner of the picture being the 0-th tile, and the index is incremented from left to right in each tile row, and after the last tile in a tile row is scanned, the scanning continues with the left-most tile in the next tile row (the tile row after the current tile row).

[0262] The indication of slices:

[0263] The following table illustrates the indication of tile size and coordinates within a tile, from the picture parameter set RBSP syntax table in VVC Draft 8.

[0264]

[0265]

[0266] In VVC Draft 8, the following relationship exists between slice and tile. One slice includes one or more complete tiles, or one tile includes one or more complete slices. Therefore, slice coordinates and size are indicated according to tile partitioning. In VVC Draft 8, tile partitioning is first indicated in the picture parameter set. After that, slice partitioning information is indicated by tile mapping information.

[0267] In the above table, the syntax element num_slices_in_pic_minus1 indicates the number of slices within a picture. Tile_idx_delta[i] indicates the difference between the tile index of the first tile in the (i+1)th slice and the first tile in the ith slice. For example, the index of the first tile of the first slice within a picture is 0. If the tile index of the first tile of the second slice within a picture is 5, then Tile_idx_delta[0] is equal to 5. In this case, the tile index is used as the address of the slice, i.e., the index of the first tile in a slice is the start address of the slice.

[0268] slice_width_in_tiles_minus1[i] and slice_height_in_tiles_minus1[i] indicate the width and height of the ith slice within a picture in units of tiles.

[0269] In the above table, if both slice_width_in_tiles_minus1[i] and slice_height_in_tiles_minus1[i] are equal to 0 (indicating that the height and width of the ith slice are at most 1 tile), the syntax element num_exp_slices_in_tile[i] can be included in the bitstream. This syntax element indicates the number of slices within a tile.

[0270] As described above, according to VVC Draft 8, one slice can include multiple complete tiles, or one tile can include multiple complete slices, and other alternatives are prohibited. According to the syntax table above, the number of tiles within a slice is first indicated (by including slice_width_in_tiles_minus1[i] and slice_height_in_tiles_minus1[i]). In addition, if the number of tiles within a slice is equal to 1 according to the indication, the number of slices within the tile is indicated (by num_exp_slices_in_tile[i]). Therefore, if both slice_width_in_tiles_minus1[i] and slice_height_in_tiles_minus1[i] are equal to 1, the actual size of the slice can be equal to or less than 1 tile.

[0271] The syntax element single_slice_per_subpic_flag, when true, specifies that there is one and only one slice per subpicture for each subpicture of a slice (i.e., a subpicture cannot be split into more than one slice).

[0272] According to an alternative indication method, the slice mapping (slice start address and slice size) is indicated according to the following steps in VVC Draft 8.

[0273] 1. First, the tile partitioning mapping is indicated in the bitstream, where an index (which can be referred to as tileldx) is used to index all tiles in the picture (in the tile scan order in the picture). After this step, the index, coordinates and size of each tile are known.

[0274] 2. The number of slices in the picture is indicated. In one example, the number of slices can be indicated by the syntax element num_slices_in_pic_minus1.

[0275] 3. For the first slice in the picture, only the width and height of the slice in the number of tiles are indicated. The start address of the first slice is not explicitly indicated but inferred to be tileldx 0 (the first tile in the picture is the first tile in the first slice in the picture).

[0276] 4. If the width and height of the first slice are both equal to 1 tile, and if there is more than 1 CTU row within the tile included in the first slice, the n syntax element num_exp_slices_in_tile[0] is indicated, which indicates the number of slices (referred to as numSlicesInTile[0]) included within the tile.

[0277] 5. For each slice in the picture from the second slice to the last slice (including the second slice but not including the last slice), the width and height of the slice in the number of tiles are explicitly indicated. The start address of the slice can be explicitly indicated by the syntax element tile_idx_delta[i], where i is the index of the slice. If the start address is not explicitly indicated (e.g., if the indication order of the slices makes it possible to infer the start position of the next slice from the start position, width and height of the current slice), the start address of the slice is inferred according to the function

[0278] 6. If the width and height of the nth slice (n is between 2 and the number of slices in the picture minus 1) are both equal to 1 tile, and if there is more than 1 CTU row within the tile included in the first slice, the syntax element num_exp_slices_in_tile[n] is indicated, which indicates the number of slices included within the tile.

[0279] 7. For the last slice in the picture, the width and height of the slice are not explicitly signaled, but are inferred from the number of tiles in picture width in units of picture width, the number of tiles in picture height in units of picture height, and the start address of the last slice. The start address of the last slice can be explicitly signaled or inferred. The width and height of the last slice in the picture can be inferred from the following two equations, which are from Section 6.5.1 in VVC Draft 8.

[0280] slice_width_in_tiles_minus1[i] = NumTileColumns - 1 - tileX

[0281] slice_height_in_tiles_minus1[i] = NumTileRows - 1 - tileY

[0282] As can be seen from the steps explained above, the width and height of the last slice are not signaled. It is desirable to not include the width and height of the last slice in the bitstream, because if the start address of the slice is known, it is easy to infer the width and height of the slice. Thus, by not including redundant information in the bitstream, efficient compression is achieved.

[0283] The variables tileX, tileY, NumTileColumns, and NumTileRows in the above equations are explained later.

[0284] Section 6.5.1 in VVC Draft 8

[0285] 6.5.1 CTB raster scan, tile scan, and subpicture scan processes

[0286] For rectangular slices, the list NumCtusInSlice[ i ] (i in the range of 0 to num_slices_in_pic_minus1, inclusive) represents the number of CTUs in the i-th slice, the list SliceTopLeftTileIdx[ i ] (i in the range of 0 to num_slices_in_pic_minus1, inclusive) represents the index of the top-left tile in the slice, and the matrix CtbAddrInSlice[ i ][ j ] (i in the range of 0 to num_slices_in_pic_minus1, inclusive, j in the range of 0 to NumCtusInSlice[ i ] - 1, inclusive) represents the picture raster scan address of the j-th CTB within the i-th slice, which are derived as follows:

[0287]

[0288]

[0289] where the function AddCtbsToSlice(sliceIdx, startX, stopX, startY, stopY) is as follows:

[0290]

[0291] Again, for completeness, the almost identical content is referenced from the Generic Video Coding according to Recommendation ITU-T H.266 (ISO / IEC 23090-3:2020) as passed on 29 August 2020 via http: / / handle.itu.int / 11.1002 / 1000 / 14336.

[0292] When rect_slice_flag is equal to 1, the list NumCtusInSlice[ i ] (i in the range of 0 to num_slices_in_pic_minus1, inclusive) indicates the number of CTUs in the i-th slice, the list SliceTopLeftTileIdx[ i ] (i in the range of 0 to num_slices_in_pic_minus1, inclusive) indicates the tile index of the tile that includes the first CTU in the slice, the matrix CtbAddrInSlice[ i ][ j ] (i in the range of 0 to num_slices_in_pic_minus1, inclusive, j in the range of 0 to NumCtusInSlice[ i ] - 1, inclusive) indicates the picture raster scan address of the j-th CTB within the i-th slice, and the variable NumSlicesInTile[ i ] indicates the number of slices in the tile that includes the i-th slice, are derived as follows:

[0293]

[0294]

[0295]

[0296] Reference is now made to the text in the above VVC Draft 8.

[0297] The step-by-step explanation of the strip mapping indication within a picture above is one example indicated in VVC Draft 8. More specifically, the above explanation describes the case where rectangular strips are used, the number of strips in each subpicture is not indicated as 1, there is more than 1 tile in the picture, and the number of CTU rows in a tile is greater than 1. Other modes of strip mapping indication can be used if some of the parameters mentioned above are changed. For example, if it is indicated that there is only 1 strip in each subpicture, the width and height of the strip are not explicitly indicated in the bitstream, but are inferred to be the corresponding subpicture width and height.

[0298] Section 6.5.1 in VVC Draft 8 details the scan order of CTUs within a slice i, where i is the slice index. The matrix CtbAddrInSlice[i][n] is the output of this section, which represents the scan order of CTUs within slice i, where n is the CTU index between 0 and the number of CTUs in slice i. The value of CtbAddrInSlice[i][n] represents the address (in the raster scan order in the picture) of the nth CTU in slice i.

[0299] FIG. 11 An example is shown to illustrate the raster scan order in a picture including CTUs (raster scan order of CTUs in a picture) and one slice in this picture (slice 5, i.e., the 5th slice in the picture). In other words, FIG. 11 An example is shown to illustrate the raster scan order of CTUs within a picture, where the picture includes 1 tile and 1 subpicture.

[0300] According to this example, the values of CtbAddrInSlice are shown as follows:

[0301] CtbAddrInSlice[4][0] = 27,

[0302] CtbAddrInSlice[4][1] = 28,

[0303] CtbAddrInSlice[4][2] = 29,

[0304] CtbAddrInSlice[4][3] = 30,

[0305] CtbAddrInSlice[4][4] = 37,

[0306] CtbAddrInSlice[4][5] = 38,

[0307] CtbAddrInSlice[4][6] = 39,

[0308] CtbAddrInSlice[4][7] = 40.

[0309] The terms used in this application:

[0310] • "Tile scan order in a picture" explained in this application

[0311] • "CTU scan order within a tile" explained in this application

[0312] • "CTU scan order within a slice" explained in this application

[0313] • "CTU raster scan order in a picture" explained in this application

[0314] • "Tile-based scan order of CTUs within a picture"

[0315] • "Scan order" refers to the order of indexing X in Y according to increasing index.

[0316] • "Processing" refers to decoding or encoding in an encoder or decoder. Thus, processing order refers to the order in which X (e.g., CTU) is processed in an encoder or decoder.

[0317] In VVC Draft 8, when each picture has more than 1 tile, the slices are indicated as follows:

[0318] 1. Determine the starting tile address of a slice in units of tiles using explicit indication or inference.

[0319] 2. For each slice except the last slice, indicate the number of tiles included by the slice.

[0320] a. If it is determined that the slice includes only 1 tile, indicate the number of slices included within the tile.

[0321] 3. For the last slice in a picture, if it is determined that the slice includes at least 1 complete tile, infer the number of tiles in the slice.

[0322] In other words, in VVC Draft 8, if the width and height of the last slice in a picture are both greater than or equal to 1 tile, the size of the last slice is inferred instead of being indicated.

[0323] This can be seen from Table 1, where slice_width_in_tiles_minus1[i] and slice_height_in_tiles_minus1[i] (representing the width and height in tile number of the i-th slice, respectively) are included in the bitstream if they are smaller than num_slices_in_pic_minus1 (because of the for loop "for(i=0; i

[0324] Luma mapping with chroma scaling (LMCS):

[0325] In VVC, a coding tool called luma mapping with chroma scaling (LMCS) is added before the in-loop filter as a new processing block. LMCS mainly consists of two parts: (1) in-loop mapping of the luma component based on an adaptive piecewise linear model, (2) for the chroma components, applying luma-dependent chroma residual scaling. FIG. 11 The LMCS architecture from the decoder perspective is shown. FIG. 11 The light blue shaded blocks in Fig. 1 represent processing in the mapped domain; these blocks include dequantization, inverse transform, luma intra prediction, and luma prediction plus luma residual. FIG. 10 The non-shaded blocks in Fig. 1 represent processing in the original (i.e., non-mapped) domain; these blocks include in-loop filters (e.g., deblocking filter, ALF, and SAO), motion-compensated prediction, chroma intra prediction, chroma prediction plus chroma residual, and decoded picture storage as reference picture. FIG. 10 The light yellow shaded blocks in Fig. 1 are the new LMCS functional blocks, including forward and inverse mapping of the luma signal and luma-dependent chroma scaling process. Like most other tools in VVC, LMCS can be enabled / disabled at the sequence level by an SPS flag.

[0326] Slice header: A part of the coded slice that includes data elements related to all tiles or to the CTU rows within one tile represented in the slice.

[0327] Slice header:

[0328] Table 3

[0329]

[0330]

[0331] Table 3 illustrates an example of a part of the slice header syntax structure in VVC Draft 8. The lines including "..." indicate that some lines in the table are omitted.

[0332] In the slice header, the syntax elements represent the following:

[0333] picture_header_in_slice_header_flag indicates whether the picture header syntax structure is present in the slice header. If the picture header syntax structure is not present in the slice header, the picture header syntax structure must be included in the picture header, and the picture header must be included in the bitstream.

[0334] slice_address indicates the tile index of the first tile in the slice.

[0335] num_tiles_in_slice_minus1 indicates the number of tiles included in the slice.

[0336] FIG. 10 An example is shown where a picture is partitioned into 12 tiles and 3 slices. Or, in other words, FIG. 14 An example is shown where a picture including 18x12 luma CTUs is partitioned into 12 tiles and 3 raster-scan slices.

[0337] In the example shown in FIG. 14 In this example shown in

[0338] • Slice 1

[0339] o slice_address = 0, the slice starts at tile index 0.

[0340] o num_tiles_in_slice_minus1 = 1, the slice includes 2 tiles.

[0341] • Slice 2

[0342] o slice_address = 2, the slice starts at tile index 2.

[0343] o num_tiles_in_slice_minus1 = 5, the slice includes 5 tiles.

[0344] • Slice 3

[0345] o slice_address = 7, the slice starts at tile index 7.

[0346] o nnum_tiles_in_slice_minus1 = 4, the slice includes 5 tiles.

[0347] slice_lmcs_enabled_flag equal to 1 specifies that luma mapping with chroma scaling is enabled for the current slice. slice_lmcs_enabled_flag equal to 0 specifies that luma mapping with chroma scaling is not enabled for the current slice. When slice_lmcs_enabled_flag is not present, slice_lmcs_enabled_flag is inferred to be 0.

[0348] The starting tile in the slice (address of the slice in the picture) and the number of tiles within the picture can be indicated in 2 ways. When rect_slice_flag is equal to 1, it specifies that the slice in the picture is rectangular, then the indication mechanism in Table 1 is used. Table 1 represents a part of the picture parameter set. In this mechanism, the address and size of all slices in the picture are indicated in the picture parameter set before the first slice of the picture in the bitstream. It is noted that the order in which information (picture parameter set, slice in the picture and syntax elements within the syntax structure, etc.) is included in (or parsed from) the bitstream.

[0349] If rect_slice_flag is equal to 0, it specifies that the slice in the picture does not need to be rectangular, then slice_address and the syntax element num_tiles_in_slice_minus1 in the slice header specify the address and size of the slice.

[0350] Picture header:

[0351] 7.3.2.6 Picture header RBSP syntax

[0352]

[0353] The above table shows the picture header syntax according to VVC Draft 8. The above table includes the picture header structure and rbsp_trailing_bits(), which are trailing bits to make the number of bits in the picture header equal to a multiple of 8.

[0354] Picture header structure:

[0355] 7.3.2.7 Picture header structure syntax

[0356]

[0357]

[0358] The picture header structure includes syntax elements that are applicable to all slices in the picture. Some of the syntax elements included in the picture header structure are shown in the table above. For example, ph_lmcs_enabled_flag indicates whether the luma mapping with chroma scaling (LMCS) coding tool is enabled for slices in the picture.

[0359] ph_lmcs_enabled_flag equal to 1 indicates that luma mapping with chroma scaling is enabled for all slices associated with the PH. ph_lmcs_enabled_flag equal to 0 indicates that luma mapping with chroma scaling is disabled for one or more or all slices associated with the PH. When ph_lmcs_enabled_flag is not present, then the value of ph_lmcs_enabled_flag is inferred to be 0.

[0360] As mentioned above, the picture header structure can be present in the slice header or in the picture header. According to VVC draft 8, the picture header must be in the slice header or in the picture header of the picture. If the picture header structure is in the picture header, then all slices in the picture referring to the picture header can not include the picture header structure. The same is true in reverse, if the picture header structure is not in the picture header, thus the picture header is not included in the bitstream corresponding to a particular picture, then the picture header structure must be in the slice header of the slices in the picture.

[0361] In addition, there is another restriction condition in VVC draft 8, that is, if the picture header structure is in the slice header, then the picture must only include one slice (i.e. the picture cannot be divided into multiple slices).

[0362] Since slice_address and num_tiles_in_slice_minus1 are included in the bitstream redundantly in some cases, the current VVC draft 8 is not efficient. The inclusion of slice_address and num_tils_in_slice_minus1 in the bitstream redundantly increases the bitstream, because each slice header of the picture can include the syntax elements, thus the compression efficiency is reduced.

[0363] Embodiment 1:

[0364] According to an embodiment, whether slice_address and the syntax element num_tils_in_slice_minus1 are present in the slice header is controlled according to whether the picture header structure is present in the slice header.

[0365]

[0366]

[0367] The application can be implemented as shown in the above table. According to the application, slice_address is included in the slice header when the condition in row 6 is true. In other words, slice_address is included in the slice header in the following cases:

[0368] • The number of tiles in the picture is greater than 1, non-rectangular slices are used (rect_slice_flag = 0) and the picture header structure is not present in the slice header;

[0369] Or,

[0370] • (rect_slice_flag = 1) the number of slices in the current subpicture is greater than 1.

[0371] Otherwise, slice_address is not included in the slice header, its value can be inferred to be 0.

[0372] Additionally or alternatively, the presence of the syntax element num_tiles_in_slice_minus1 in the slice header can be controlled by whether the picture header structure is present in the slice header or not. For example, if the picture header structure is present in the slice header, num_tiles_in_slice_minus1 is not included in the slice header.

[0373] Row 10 in the above table shows one implementation of the application, where num_tiles_in_slice_minus1 is included in the slice header in the following cases:

[0374] • rect_slice_flag is equal to 0, the number of tiles in the picture is greater than 1 and picture_header_in_slice_header_flag is equal to 0.

[0375] Otherwise, num_tiles_in_slice_minus1 is not included in the slice header, its value can be inferred to be the number of tiles in the picture minus 1.

[0376] As mentioned above, there is a bitstream conformance requirement in VVC Draft 8 that restricts the inclusion of the picture header structure in the slice header. According to VVC Draft 8, the picture header structure can be included in the slice header if there is one slice per picture.

[0377] According to the application, whether the picture header structure is present in the slice header is used to control whether slice_address and the number of tiles are present in the slice indication, because if there is a single slice in the picture, the slice address must be equal to the first tile in the picture and the number of tiles in the slice must be equal to the number of tiles in the picture.

[0378] Embodiment 2:

[0379]

[0380] Additionally or alternatively, whether num_tiles_in_slice_minus1 is present in the slice header is controlled by the difference between the number of tiles in the picture (e.g., NumTilesInPic in the table above) and the slice address.

[0381] More specifically, if the difference between the number of tiles in the picture and the slice address is less than a threshold, num_tiles_in_slice_minus1 is not included in the slice header, and its value is inferred to be a predefined number. For example, if the difference between NumTilesInPic and slice_address is less than or equal to 1, num_tiles_in_slice_minus1 is not included in the bitstream, and its value is inferred to be 0 (indicating that there is one tile in the current slice).

[0382] slice_address specifies the slice address of the slice. When slice_address is not present, the value of slice_address is inferred to be 0.

[0383] If rect_slice_flag is equal to 0, the following applies:

[0384] slice_address specifies the slice address of the slice. When slice_address is not present, the value of slice_address is inferred to be 0.

[0385] slice_address has a length of Ceil( Log2( NumTilesInPic ) ) bits.

[0386] The value of slice_address shall be in the range of 0 to NumTilesInPic - 1, inclusive.

[0387] Otherwise (rect_slice_flag is equal to 1), the following applies:

[0388] slice_address specifies the subpicture level slice index of the current slice, i.e., SubpicLevelSliceIdx[ j ], where j is the picture level slice index of the current slice.

[0389] slice_address has a length of Ceil( Log2( NumSlicesInSubpic[ CurrSubpicIdx ] ) ) bits.

[0390] The value of slice_address shall be in the range of 0 to NumSlicesInSubpic[ CurrSubpicldx ] - 1, inclusive.

[0391] The requirement of bitstream conformance is that the following constraints apply:

[0392] If rect_slice_flag is equal to 0 or sps_subpic_info_present_flag is equal to 0, the value of slice_address shall not be equal to the value of slice_address of any other coded slice NAL unit of the same coded picture.

[0393] Otherwise, the pair of values of subpic_id and slice_address shall not be equal to the pair of values of subpic_id and slice_address of any other coded slice NAL unit of the same coded picture.

[0394] The shape of the slice in the picture shall be such that each CTU, when decoded, shall have its entire left boundary and its entire top boundary consist of picture boundaries or consist of the boundaries of previously decoded CTU or CTUs.

[0395] num_tiles_in_slice_minus1 + 1, when present, specifies the number of tiles in the slice. The value of num_tiles_in_slice_minus1 shall be in the range of 0 to NumTilesInPic - 1, inclusive. When num_tiles_in_slice_minus1 is not present, the value of num_tiles_in_slice_minus1 is inferred to be 0.

[0396] The variable NumCtusInCurrSlice specifies the number of CTUs in the current slice, and the list CtbAddrInCurrSlice[ i ], i in the range of 0 to NumCtusInCurrSlice - 1, inclusive, specifies the picture raster scan address of the i-th CTB within the slice, which are derived as follows:

[0397]

[0398] The variables SubpicLeftBoundaryPos, SubpicTopBoundaryPos, SubpicRightBoundaryPos, and SubpicBotBoundaryPos are derived as follows:

[0399]

[0400] Embodiment 3:

[0401]

[0402] Additionally or alternatively, slice_lmcs_enabled_flag is present in the slice header depending on whether the picture header structure is present in the slice header. An example implementation is included in line 15 of the above table.

[0403] More specifically, if the picture header structure is included in the slice header, slice_lmcs_enabled_flag is not included in the slice. Additionally, when slice_lmcs_enabled_flag is not included in the slice, the value of slice_lmcs_enabled_flag can be inferred according to the following rule:

[0404] The value of slice_lmcs_enabled_flag is inferred to be ph_lmcs_enabled_flag.

[0405] Alternatively or additionally, when slice_lmcs_enabled_flag is not present in the slice header, the value of slice_lmcs_enabled_flag can be inferred according to the following rule:

[0406] When picture_header_in_slice_header_flag is equal to 1 (the picture header structure is included in the slice header), the value of slice_lmcs_enabled_flag is inferred to be ph_lmcs_enabled_flag.

[0407] Alternatively or additionally, when slice_lmcs_enabled_flag is not present in the slice header, the value of slice_lmcs_enabled_flag can be inferred according to the following rule:

[0408] When picture_header_in_slice_header_flag is equal to 0, the value of picture_header_in_slice_header_flag is inferred to be 0.

[0409] The above embodiments can be implemented by replacing the condition "!rect_slice_flag && NumTilesInPic > 1" in the 6th row and the 10th row with "!rect_slice_flag". In some example implementations, if the value of rect_slice_flag is equal to 0, indicating that the slices in the picture are not necessarily rectangular, the value of the NumTilesInPic syntax element must be greater than 0 (e.g., the number of tiles in the slice must be greater than 1). In other words, if the number of tiles in the picture is greater than 1, the value of rect_slice_flag can only be equal to 0. In such implementations, the condition "!rect_slice_flag && NumTilesInPic > 1" and the condition "!rect_slice_flag" will produce the same result. Therefore, the condition (in the 6th row and the 10th row in all the above embodiments), including the partial condition "!rect_slice_flag && NumTilesInPic > 1", can be replaced with "!rect_slice_flag".

[0410] The above embodiments can be implemented by replacing the condition "!rect_slice_flag && NumTilesInPic > 1" in the 6th row and the 10th row with "!rect_slice_flag". In some example implementations, if the value of rect_slice_flag is equal to 0, indicating that each slice in the picture includes one or more tiles, and if picture_header_in_slice_header_flag is equal to 0, indicating that the number of slices in the picture is greater than 1, therefore if picture_header_in_slice_header_flag is equal to 0 and rect_slice_flag is equal to 0, NumTilesInPic must be greater than 1.

[0411] The application of the encoding method and the decoding method shown in the above embodiments and the system using the applications are explained and described below.

[0412] FIG. 15 A flowchart of a method for decoding a video bitstream is provided for an embodiment of the present application. FIG. 15The method shown is a method of decoding an image from a video bitstream by a decoding device. The bitstream includes a slice header of a current slice and data representing the current slice, and the method includes: if a condition is met, obtaining (step 1601) a parameter used for deriving a number of tiles in the current slice from the slice header, wherein the condition includes: a slice address of the current slice is not an address of a last tile in an image where the current slice is located; and reconstructing (step 1603) the current slice by the number of tiles in the current slice and the data representing the current slice.

[0413] FIG. 16 Another flowchart of a method of encoding a video bitstream is provided for an embodiment of the application. FIG. 16 The method shown is a method of encoding a video bitstream by an encoding device. The bitstream includes a slice header of a current slice and data representing the current slice, and the method includes: if a condition is met, encoding (step 1701) a parameter used for deriving a number of tiles in the current slice in the slice header, wherein the condition includes: a slice address of the current slice is not an address of a last tile in an image where the current slice is located; and reconstructing (step 1703) the current slice by the number of tiles in the current slice and the data representing the current slice.

[0414] FIG. 17 An apparatus for decoding a video bitstream, i.e., a decoder (30), is shown for an embodiment of the application. FIG. 17 The apparatus shown is an apparatus (30) for decoding an image from a video bitstream. The bitstream includes a slice header of a current slice and data representing the current slice, and the apparatus includes: an obtaining unit (3001) configured to: if a condition is met, obtain a parameter used for deriving a number of tiles in the current slice from the slice header, wherein the condition includes: a slice address of the current slice is not an address of a last tile in an image where the current slice is located; and a reconstructing unit (3003) configured to reconstruct the current slice by the number of tiles in the current slice and the data representing the current slice.

[0415] FIG. 16 An apparatus for encoding a video bitstream, i.e., an encoder (20), is shown for an embodiment of the application. FIG. 1AThe apparatus shown is an apparatus (20) for encoding a coded video bitstream. The bitstream includes a slice header of a current slice and data representing the current slice, and the apparatus includes: an encoding unit (2001) configured to encode, if a condition is satisfied, a parameter in the slice header for deriving a number of tiles in the current slice, wherein the condition includes that a slice address of the current slice is not an address of a last tile in a picture in which the current slice is located; and a reconstruction unit (2003) configured to reconstruct the current slice based on the number of tiles in the current slice and the data representing the current slice.

[0416] FIG. 1B The video decoding apparatus shown can be FIG. 3 、 FIG. 13 and FIG. 4 The decoder 30 and FIG. 5 The video decoder 3206 shown can also be included therein. Furthermore, the decoding apparatus can be included in FIG. 12 The video coding device 400 shown, FIG. 17 The apparatus 500 shown, and FIG. 1A The terminal device 3106 shown. FIG. 1B The encoding apparatus shown can be FIG. 3 、 FIG. 4 and FIG. 5 The encoder 20 shown can also be included therein. Furthermore, the encoding apparatus can be included in FIG. 12 The video coding device 400 shown, FIG. 12 The apparatus 500 shown, and FIG. 13 The capture device 3102 shown.

[0417] The present application discloses the following more features. FIG. 13 A block diagram of a content supply system 3100 for implementing a content distribution service. The content supply system 3100 includes a capture device 3102, a terminal device 3106, and optionally a display 3126. The capture device 3102 communicates with the terminal device 3106 through a communication link 3104. The communication link can include the communication channel 13 described above. The communication link 3104 includes, but is not limited to, Wi-Fi, Ethernet, cable, wireless (3G / 4G / 5G), USB, or any kind of combination thereof, etc.

[0418] The capture device 3102 is configured to generate data and can encode the data by the encoding method shown in the above embodiments. Alternatively, the capture device 3102 can distribute the data to a streaming server (not shown in the figure), which encodes the data and sends the encoded data to the terminal device 3106. The capture device 3102 includes, but is not limited to, a camera, a smartphone or a tablet, a computer or a notebook, a video conference system, a PDA, a vehicle-mounted device, or any combination thereof, etc. For example, the capture device 3102 can include the source device 12 described above. When the data includes video, the video encoder 20 included in the capture device 3102 can actually perform the video encoding process. When the data includes audio (i.e., sound), the audio encoder included in the capture device 3102 can actually perform the audio encoding process. For some practical scenarios, the capture device 3102 distributes the encoded video data and the encoded audio data by multiplexing the encoded video data and the encoded audio data together. For other practical scenarios, such as in a video conference system, the encoded audio data and the encoded video data are not multiplexed. The capture device 3102 distributes the encoded audio data and the encoded video data to the terminal device 3106.

[0419] In the content supply system 3100, the terminal device 310 receives and regenerates the encoded data. The terminal device 3106 can be a device with data receiving and restoring capability, such as a smartphone or a tablet 3108, a computer or a notebook 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a television 3114, a set top box (STB) 3116, a video conference system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, a vehicle-mounted device 3124, or a combination thereof, or a device capable of decoding the above encoded data. For example, the terminal device 3106 can include the destination device 14 described above. When the encoded data includes video, the video decoder 30 included in the terminal device performs video decoding preferentially. When the encoded data includes audio, the audio decoder included in the terminal device performs audio decoding process preferentially.

[0420] For terminal devices with displays, such as a smartphone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a television 3114, a personal digital assistant (PDA) 3122, or a car device 3124, the terminal device can feed the decoded data to its display. For terminal devices without displays, such as a STB 3116, a video conference system 3118, or a video surveillance system 3120, an external display 3126 is connected to receive and display the decoded data.

[0421] When each device in the system performs encoding or decoding, an image encoding device or an image decoding device as shown in the above-described embodiments can be used.

[0422] FIG. 13 A schematic diagram of an exemplary structure of the terminal device 3106. After the terminal device 3106 receives the stream from the capture device 3102, a protocol processing unit 3202 analyzes the transmission protocol of the stream. The protocol includes, but is not limited to, Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live streaming protocol (HLS), MPEG-DASH, Real-time Transport protocol (RTP), Real Time Messaging Protocol (RTMP), or any kind of combination thereof, etc.

[0423] After the protocol processing unit 3202 processes the stream, a stream file is generated. The file is output to a demultiplexing unit 3204. The demultiplexing unit 3204 can separate the multiplexed data into encoded audio data and encoded video data. As described above, for other practical scenarios, such as in a video conference system, the encoded audio data and the encoded video data are not multiplexed. In this case, the encoded data is sent to a video decoder 3206 and an audio decoder 3208 without passing through the demultiplexing unit 3204.

[0424] By demultiplexing, video elementary streams (ES), audio ES, and optional subtitles are generated. Video decoder 3206 comprises video decoder 30 as explained in the above embodiments, which decodes the video ES by the decoding method as shown in the above embodiments to generate video frames, and feeds this data to synchronization unit 3212. Audio decoder 3208 decodes the audio ES to generate audio frames, and feeds this data to synchronization unit 3212. Optionally, the video frames can be stored in a buffer (not shown in ​ ​

[0425] Synchronization unit 3212 synchronizes the video frames and the audio frames, and provides video / audio to video / audio display 3214. For example, synchronization unit 3212 synchronizes the presentation of the video information and the audio information. The information can be coded in syntax using timestamps related to the presentation of the coded audio and visual data, and timestamps related to the transmission of the data stream itself.

[0426] If subtitles are included in the stream, subtitle decoder 3210 decodes the subtitles, synchronizes the subtitles with the video frames and the audio frames, and provides video / audio / subtitles to video / audio / subtitles display 3216.

[0427] The present application is not limited to the above-described system, and the image encoding apparatus or the image decoding apparatus in the above embodiments can be included in other systems such as a car system.

[0428] Mathematical operators

[0429] The mathematical operators used in the present application are similar to those used in the C programming language. However, the present application defines the results of integer division and arithmetic shift operations precisely, and also defines other operations such as power and real division. The numbering and counting convention is usually from 0, e.g., "first" corresponds to 0th, "second" corresponds to 1st, and so on.

[0430] Arithmetic operators

[0431] The arithmetic operators are defined as follows:

[0432] + addition

[0433] - subtraction (binary operator) or negation (unary prefix operator)

[0434] * multiplication, including matrix multiplication

[0435] x y ​​Power, representing x to the power of y. Depending on the context, this notation is used as a superscript, rather than as a power operation.

[0436] / Integer division, truncated in the direction of the value 0. For example, 7 / 4 and -7 / -4 are truncated to 1, and -7 / 4 and 7 / -4 are truncated to -1.

[0437] ÷ Division, without truncation or rounding.

[0438] Division, without truncation or rounding.

[0439] The cumulative sum of the function f(i) for all integer values of i from x to y, inclusive.

[0440] x % y Modulo, representing the remainder of x divided by y, where x and y are integers and x >= 0 and y > 0.

[0441] Logical operators

[0442] Logical operators are defined as follows:

[0443] x && y Boolean logical "and" operation of x and y

[0444] x || y Boolean logical "or" operation of x and y

[0445] ! Boolean logical "not" operation

[0446] x? y : z If x is true or not equal to 0, then the value of y is sought; otherwise, the value of z is sought.

[0447] Relational operators

[0448] Relational operators are defined as follows:

[0449] > Greater than

[0450] >= Greater than or equal to

[0451] < Less than

[0452] <= Less than or equal to

[0453] == Equal to

[0454] != Not equal to

[0455] When a relational operator is applied to a syntax element or variable that has been assigned the value "na" (not applicable), the value "na" is considered to be a distinct value from the syntax element or variable. The value "na" is considered not equal to any other value.

[0456] Bitwise operators

[0457] The bitwise operators are defined as follows:

[0458] & Bitwise "and". When operating on integer arguments, the binary complement representation of the integer values is operated on. When operating on binary arguments, the shorter argument is extended with more bits equal to zero at the left, if it is shorter than the other argument.

[0459] | Bitwise "or". When operating on integer arguments, the binary complement representation of the integer values is operated on. When operating on binary arguments, the shorter argument is extended with more bits equal to zero at the left, if it is shorter than the other argument.

[0460] ^ Bitwise "exclusive or". When operating on integer arguments, the binary complement representation of the integer values is operated on. When operating on binary arguments, the shorter argument is extended with more bits equal to zero at the left, if it is shorter than the other argument.

[0461] x » y shifts x, which is in two's complement integer representation, arithmetically right by y binary digits. This function is only defined for non-negative integer values of y. The result of the right shift is that the bits shifted into the most significant bit (MSB) are equal to the MSB of x before the shift operation.

[0462] x « y shifts x, which is in two's complement integer representation, arithmetically left by y binary digits. This function is only defined for non-negative integer values of y. The result of the left shift is that the bits shifted into the least significant bit (LSB) are equal to zero.

[0463] Assignment operator

[0464] The arithmetic operators are defined as follows:

[0465] = Assignment operator

[0466] ++ Increment, i.e. x++ is equivalent to x = x + 1; when used as an array subscript, the variable value is evaluated before the increment operation.

[0467] –– Decrement, i.e. x–– is equivalent to x = x - 1; when used as an array subscript, the variable value is evaluated before the decrement operation.

[0468] + = Increment by a specified value, i.e. x + = 3 is equivalent to x = x + 3, x + = ( - 3) is equivalent to x = x + ( - 3).

[0469] – = Decrement by a specified value, i.e. x – = 3 is equivalent to x = x - 3, x – = ( - 3) is equivalent to x = x - ( - 3).

[0470] Range Notation

[0471] The following notation is used to specify ranges of values:

[0472] x = y..z x takes on the integer values from y, including y, to z, including z, where x, y, and z are integers, and z is greater than y.

[0473] Mathematical Functions

[0474] Mathematical functions are defined as follows:

[0475]

[0476] Asin(x) inverse sine function operating on the argument x, in the range –1.0 to 1.0, inclusive, with the output in the range –π÷2 to π÷2, inclusive, in radians.

[0477] Atan(x) inverse tangent function operating on the argument x, with the output in the range –π÷2 to π÷2, inclusive, in radians.

[0478]

[0479] Ceil(x) smallest integer greater than or equal to x.

[0480] Clip1 Y (x) = Clip3(0, (1 << BitDepth Y ) - 1, x)

[0481] Clip1 C (x) = Clip3(0, (1 << BitDepth C ) - 1, x)

[0482]

[0483] Cos(x) cosine function operating on the argument x, in radians.

[0484] Floor(x) largest integer less than or equal to x.

[0485]

[0486] Ln(x) returns the natural logarithm (logarithm to the base e, where e is the natural logarithm base constant 2.718281828… ) of x.

[0487] Log2(x) logarithm to the base 2 of x.

[0488] Log10(x) logarithm to the base 10 of x.

[0489]

[0490] Round(x) = Sign(x) * Floor(Abs(x) + 0.5)

[0491]

[0492] Sin(x) trigonometric sine function, operating on the argument x in radians.

[0493]

[0494] Swap(x, y) = (y, x)

[0495] Tan(x) trigonometric tangent function, operating on the argument x in radians.

[0496] Order of operations

[0497] When no parentheses are used to explicitly indicate the order of precedence in an expression, the following rules apply:

[0498] - High-priority operations are calculated before any lower-priority operations.

[0499] - Operations of the same priority are calculated from left to right.

[0500] The following table indicates the precedence of operations, from highest to lowest, with higher position in the table indicating higher precedence.

[0501] For operators also used in the C programming language, the order of precedence of operators in this specification is the same as in the C programming language.

[0502] Table: Order of precedence of operations, from highest (top of table) to lowest (bottom of table)

[0503]

[0504] Textual description of logical operations

[0505] In text, logical operation statements are described in mathematical form as follows:

[0506]

[0507]

[0508] This can be described as follows:

[0509] … as follows / … the following applies:

[0510] - If condition 0, then statement 0

[0511] - Else if condition 1, then statement 1

[0512] -...

[0513] - Else (provisional statement for remaining conditions), then statement n

[0514] Each "if... else if... else..." statement in the text is introduced with "... as follows" or "... the following applies", followed by "if...". The last condition of an "if... else if... else..." always has an "else...". Statements in between "if... else if... else..." can be identified by matching "... as follows" or "... the following applies" with the end "else...".

[0515] In the text, the following logical operation statement is described in mathematical form:

[0516]

[0517] This can be described as follows:

[0518] ... as follows / ... the following applies:

[0519] - If all of the following conditions are true, then statement 0:

[0520] - Condition 0a

[0521] - Condition 0b

[0522] - Else if one or more of the following conditions are met, then statement 1:

[0523] - Condition 1a

[0524] - Condition 1b

[0525] -...

[0526] - Else, statement n

[0527] In the text, the following logical operation statement is described in mathematical form:

[0528] if (condition 0)

[0529] statement 0

[0530] if (condition 1)

[0531] statement 1

[0532] This can be described as follows:

[0533] When condition 0, statement 0

[0534] When condition 1, statement 1.

[0535] While the embodiments of the present application are primarily described in terms of video coding, it is noted that the embodiments of the coding system 10, the encoder 20 and the decoder 30 (respectively, the system 10) and other embodiments described herein can also be used for still image processing or coding, i.e. processing or coding of a single image independent of any previous or consecutive image. Generally, if the image processing coding is limited to a single image 17, only the inter prediction units 244 (encoder) and 344 (decoder) are not available. All other functions (also referred to as tools or techniques) of the video encoder 20 and the video decoder 30 can likewise be used for still image processing, e.g. residual calculation 204 / 304, transform 206, quantization 208, inverse quantization 210 / 310, (inverse) transform 212 / 312, partitioning 262 / 362, intra prediction 254 / 354 and / or loop filtering 220 / 320, entropy encoding 270 and entropy decoding 304.

[0536] The embodiments of the encoder 20 and the decoder 30 and the functions described herein with respect to the encoder 20 and the decoder 30 can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media can include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates

[0537] By way of example, and not limitation, such computer-readable storage media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other storage medium that can be used to store desired program code means in the form of instructions or data structures and that can be accessed by a computer. Also, any

[0538] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term "processor," as used herein can refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, the various functions described herein can be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated in a combined codec. Also, the techniques could be fully implemented in one or more circuits or logic elements.

[0539] The techniques of this disclosure can be implemented in a wide variety of devices or apparatuses including a wireless handset, an integrated circuit (IC) or a set of ICs (e.g., a chip set). Various components, modules, or units are described herein as being part of a device, apparatus, or system to emphasize that the techniques disclosed herein can be implemented in a variety of devices, apparatuses, or systems. Whether these components, modules, or units are implemented as hardware or software depends on the particular application and design constraints. It is noted that for purposes of clarity, not all components, modules, or units are shown or described in every embodiment. For example, the techniques are not limited to implementations that include all of the components, modules, or units shown and described herein. Further, the techniques are not limited to implementations that include only a subset of the components, modules, or units shown and described herein. For example, one or more components, modules, or units can be omitted in some embodiments. Further, some embodiments can include additional components, modules, or units beyond those shown and described herein.

[0540] The following 21 aspects are disclosed.

[0541] 1. An aspect of a method of decoding a video or image bitstream implemented by a decoding device, wherein the bitstream comprises data representative of a current slice, the method comprising: obtaining, from a slice header of the bitstream, a slice address of the current slice if a presence condition is satisfied, wherein the presence condition comprises that a picture header syntax structure is not present in the slice header; and reconstructing the current slice according to the slice address of the current slice.

[0542] 2. An aspect of the method of aspect 1, wherein the picture header syntax structure not being present in the slice header comprises a syntax element being false, the syntax element being false indicating that the picture header syntax structure is not present in the slice header.

[0543] 3. An aspect of the method of aspect 1 or 2, wherein a value of the slice address of the current slice is inferred to be 0 when the presence condition is not satisfied.

[0544] 4. An aspect of a method of decoding a video or image bitstream implemented by a decoding device, wherein the bitstream comprises data representative of a current slice, the method comprising: obtaining, from a slice header of the bitstream, a parameter for deriving a number of tiles in the current slice if a presence condition is satisfied, wherein the presence condition comprises that a picture header syntax structure is not present in the slice header; and reconstructing the current slice according to the number of tiles in the current slice.

[0545] 5. An aspect of the method of aspect 4, wherein the picture header syntax structure not being present in the slice header comprises a syntax element being false, the syntax element being false indicating that the picture header syntax structure is not present in the slice header.

[0546] 6. An aspect of the method of aspect 4 or 5, wherein a value of the parameter of the current slice is inferred to be a total number of tiles in a picture in which the current slice is located minus 1 when the presence condition is not satisfied.

[0547] 7. An aspect of a method for decoding a video or image stream implemented by a decoding device, wherein the stream includes data representing a current slice, the method comprising: if a condition is satisfied, obtaining a parameter from the slice header of the stream for deriving the number of blocks in the current slice, wherein the condition includes that the slice address of the current slice is not the address of the last block in the image in which the current slice is located; and reconstructing the current slice based on the number of blocks in the current slice.

[0548] 8. According to one aspect of the method of aspect 7, wherein the strip address of the current stripe is the address of the last block in the image, including the number of blocks in the image minus the strip address of the current stripe, which equals 1.

[0549] 9. According to one aspect of the method of aspect 7 or 8, wherein when the existence condition is not met, the value of the parameter of the current strip is inferred to be a default value.

[0550] 10. According to one aspect of the method of aspect 9, wherein the default value is equal to 0.

[0551] 11. An aspect of a method for decoding a video or image stream implemented by a decoding device, wherein the stream includes data representing a current slice, the method comprising: if a condition is satisfied, obtaining a parameter from the slice header of the stream for deriving the number of blocks in the current slice, wherein the condition includes that the slice address of the current slice is not the address of the last block in the image in which the current slice is located and that the image header syntax structure does not exist in the slice header; and reconstructing the current slice based on the number of blocks in the current slice.

[0552] 12. According to one aspect of the method of aspect 11, wherein when the strip address of the current strip is the address of the last block in the image, the value of the parameter is inferred to be a first default value, or when the image header syntax structure exists in the strip header, the value of the parameter is inferred to be a second default value.

[0553] 13. An aspect of a method for decoding a video or image stream implemented by a decoding device, wherein the stream includes data representing a current slice, the method comprising: obtaining, if a condition is satisfied, a parameter (e.g., slice_lmcs_enabled_flag) from the slice header of the stream for indicating whether luminance mapping with chroma scaling is enabled for the current slice, wherein the condition includes that an image header syntax structure is not present in the slice header; and reconstructing the current slice based on the number of blocks in the parameter.

[0554] 14. An aspect of the method according to aspect 13, wherein the picture header syntax structure is not present in the slice header comprises being equal to a syntax element being false, the syntax element being false indicating that the picture header syntax structure is not present in the slice header.

[0555] 15. An aspect of a method implemented by an encoding device for encoding a video or image into a bitstream, wherein the bitstream comprises data representing a current slice, the method comprising: including, according to a slice header of the bitstream, a slice address of the current slice into the bitstream if a presence condition is satisfied, wherein the presence condition comprises a picture header syntax structure not being present in the slice header; and reconstructing the current slice according to the slice address of the current slice.

[0556] 16. An aspect of a decoder (30), wherein the decoder (30) comprises processing circuitry configured to perform the method according to any one of aspects 1 to 15.

[0557] 17. An aspect of a computer program product comprising program code configured to perform the method according to any one of the above aspects when the program code is executed on a computer or processor.

[0558] 18. An aspect of a decoder, wherein the decoder comprises:

[0559] one or more processors; and a non-transitory computer-readable storage medium coupled to the one or more processors and storing programming for execution by the one or more processors, wherein the programming, when executed by the one or more processors, is configured to cause the decoder to perform the method according to any one of aspects 1 to 15.

[0560] 19. An aspect of a non-transitory computer-readable medium comprising program code, wherein the program code, when executed by a computer device, causes the computer device to perform the method according to any one of aspects 1 to 15.

[0561] 20. An aspect of an encoded bitstream for a video signal comprising a plurality of syntax elements, wherein the plurality of syntax elements comprises a picture_header_in_slice_header_flag, a flag (e.g., slice_lmcs_enabled_flag) is indicated in a slice header according to a condition based on a value of the picture_header_in_slice_header_flag.

[0562] 21. An aspect of a non-transitory recording medium, wherein the non-transitory recording medium includes an encoded bitstream decoded by an image decoding device, the bitstream is generated by dividing a frame in a video signal or an image signal into a plurality of blocks, the bitstream includes a plurality of syntax elements, the plurality of syntax elements includes rect_slice_flag or sps_num_subpics_minusl, a flag (e.g., slice_lmcs_enabled_flag) is indicated in a slice header based on a value of picture_header_in_slice_header_flag according to a condition.

Claims

1. A method for decoding an image from a video bitstream using a decoding device, characterized in that, The bitstream includes a stripe header for the current stripe and data representing the current stripe, and the method includes: If the condition is met, the parameter used to deduce the number of blocks in the current strip is obtained from the strip header. The strip address of the current strip is the raster scan block index of the first block of the current strip. The condition includes: the strip address of the current strip is not the address of the last block in the image where the current strip is located. The current stripe is reconstructed using the number of blocks in the current stripe and the data representing the current stripe.

2. The method according to claim 1, characterized in that, The slice address of the current slice is the address of the last block in the image, including determining that the number of blocks in the image minus the slice address of the current slice equals 1.

3. The method according to claim 1, characterized in that, The fact that the address of the current stripe is not the address of the last block in the image includes determining that the number of blocks in the image minus the address of the current stripe is greater than 1.

4. The method according to any one of claims 1 to 3, characterized in that, When the condition is not met, the value of the parameter of the current stripe is inferred to be the default value.

5. The method according to claim 4, characterized in that, The default value is 0.

6. The method according to claim 1, characterized in that, The striped addresses are in blocks.

7. The method according to claim 1, characterized in that, The step of reconstructing the current stripe based on the number of blocks in the current stripe includes: determining the scanning order of the coding tree units in the current stripe based on the number of blocks in the current stripe; and reconstructing the coding tree units in the current stripe based on the scanning order.

8. An apparatus for decoding an image from a video stream, characterized in that, The bitstream includes a stripe header for the current stripe and data representing the current stripe; the device includes: The acquisition unit is configured to: if a condition is met, acquire a parameter from the strip header for deriving the number of blocks in the current strip, wherein the strip address of the current strip is the raster scan block index of the first block of the current strip, and the condition includes: the strip address of the current strip is not the address of the last block in the image where the current strip is located; A reconstruction unit is configured to reconstruct the current stripe using the number of blocks in the current stripe and the data representing the current stripe.

9. A decoder (30), characterized in that, The decoder (30) includes processing circuitry for performing the method according to any one of claims 1 to 7.

10. A computer program product including program code, characterized in that, When the program code is executed on a computer or processor, the program code is used to perform the method according to any one of claims 1 to 7.

11. A decoder, characterized in that, The decoder includes: One or more processors; A non-transitory computer-readable storage medium coupled to and storing a program executed by the one or more processors, wherein, when the program is executed by the one or more processors, the decoder is configured to perform the method according to any one of claims 1 to 7.

12. A non-transitory computer-readable medium comprising program code, characterized in that, When the program code is executed by a computer device, the computer device performs the method according to any one of claims 1 to 7.

Images