Encoder, decoder, and corresponding method

Abstract

To provide an improved compression and decompression technique which improves compressibility with little or no sacrifice of picture quality.SOLUTION: Provided is a method of decoding a video or picture bitstream implemented by a decoding device, wherein the bitstream includes data representative of a current video coding layer. The method includes a step of using an identifier of an NAL unit of the current video coding layer as an identifier of the current video coding layer when it is determined that the current video coding layer is the only layer in the bitstream.SELECTED DRAWING: Figure 1A

Description

[Technical Field]

[0001] Cross-reference of related applications This patent application claims priority under the international patent application PCT / EP2020 / 057229, filed on 17 March 2020. The disclosure of the aforementioned patent application is incorporated herein by reference in its entirety.

[0002] Embodiments of this application relate to the field of picture processing in general, and more specifically to video coding. [Background technology]

[0003] Video coding (video encoding and decoding) is used in a wide range of digital video applications, such as digital television broadcasting, video transmission over the internet and mobile networks, real-time conversational applications like video chat, video conferencing, DVD and Blu-ray® discs, video content acquisition and editing systems, and camcorders for security applications. [Overview of the project] [Problems that the invention aims to solve]

[0004] Even relatively short videos can require a considerable amount of video data to depict, which can pose difficulties when streaming or otherwise communicating data over communication networks with limited bandwidth. Therefore, video data is generally compressed before being transmitted over modern telecommunications networks. Video size can also be a concern when storing video on storage devices, as memory resources may be limited. Video compression devices often use software and / or hardware at the source to encode video data before transmission or storage, thereby reducing the amount of data required to represent the digital video image. The compressed data is then received at the destination by a video decompression device that decodes the video data. With limited network resources and increasing demands for higher video quality, improved compression and decompression techniques that increase compression ratios with little to no sacrifice of picture quality are desired. [Means for solving the problem]

[0005] Embodiments of this application provide apparatus and methods for encoding and decoding according to independent claims.

[0006] The above and other objectives are achieved by the subject matter of the independent claims. Further embodiments are evident from the dependent claims, specification and drawings.

[0007] According to a first aspect, the present invention relates to a method for decoding a bitstream of a video or picture implemented by a decoding device, wherein the bitstream includes data representing the current video coding layer, and the method includes the steps of determining whether the current video coding layer is the only layer in the bitstream, and if it is determined that the current video coding layer is the only layer in the bitstream (for example, sps_video_parameter_set_id is equal to 0) (for example, only if it is determined), using the identifier of the NAL unit of the current video coding layer (for example, nuh_layer_id) as the identifier of the current video coding layer (for example, vps_layer_id).

[0008] VPS is an arbitrary set of parameters. For example, in the case of a single-layer bitstream, there may be no VPS in the bitstream, and therefore the identifier of the current video coding layer that may exist in VPS may not exist in the bitstream, and using the identifier of the NAL unit of the current video coding layer as the identifier of the current video coding layer for a single-layer bitstream guarantees the form of video coding implementation.

[0009] In one implementation, the method may further include the step of predicting the current slice of the current video coding layer using the identifier of the current video coding layer.

[0010] In one implementation, the method may further include the step of obtaining the identifier of the current video coding layer by parsing the bitstream, if it is determined that the bitstream contains multiple layers (for example, sps_video_parameter_set_id is greater than 0).

[0011] The method provided for decoding the video bitstream ensures efficient signaling of identifiers in the current video coding layer and saves bitrate.

[0012] In one implementation, the identifier of the current video coding layer may be used to construct one or more reference picture lists of the current slice of the current video coding layer, and predicting the current slice of the current video coding layer using the identifier of the current video coding layer may include predicting the current slice using one or more reference picture lists.

[0013] According to one implementation, the NAL unit of the current video coding layer may be the VCL NAL unit of the current video coding layer that exists within the bitstream.

[0014] When a single-layer bitstream is obtained from sub-bitstream extraction, non-VCL units within the single-layer bitstream may have different layer identifiers. Therefore, it is more accurate to use the identifier of the VCL NAL unit in the current video coding layer as the identifier of the current video coding layer for the single-layer bitstream.

[0015] According to one implementation, the VCL NAL units of the current video coding layer can be any one (or all) VCL NAL units of the current video coding layer present in the bitstream.

[0016] According to a second aspect, the present invention relates to a method for encoding a bitstream of video or picture implemented by an encoding device, wherein the bitstream includes data representing the current video coding layer, and the method includes the steps of determining whether the current video coding layer is the only layer in the bitstream, and if it is determined that the current video coding layer is the only layer in the bitstream, using the identifier of the NAL unit of the current video coding layer as the identifier of the current video coding layer.

[0017] According to one implementation form, the method may further include the step of predicting the current slice of the current video coding layer using the identifier of the current video coding layer.

[0018] According to one implementation form, the method may further include the step of encoding the identifier of the current video coding layer into the bitstream when it is determined that the bitstream includes multiple layers (for example, sps_video_parameter_set_id is greater than 0).

[0019] According to one implementation form, the identifier of the current video coding layer may be used to construct one or more reference picture lists of the current slice of the current video coding layer, and predicting the current slice of the current video coding layer using the identifier of the current video coding layer may include predicting the current slice using one or more reference picture lists.

[0020] According to one implementation form, the NAL unit of the current video coding layer may be the VCL NAL unit of the current video coding layer existing in the bitstream.

[0021] According to one implementation form, the VCL NAL unit of the current video coding layer may be any one (or all) of the VCL NAL units of the current video coding layer existing in the bitstream.

[0022] According to a third aspect, the present invention relates to an apparatus for decoding a (encoded) video bitstream, the apparatus comprising: a determination unit configured to determine whether the current video coding layer is the only layer in the bitstream; and a prediction unit configured to use the identifier of the NAL unit of the current video coding layer (e.g., nuh_layer_id) as the identifier of the current video coding layer (e.g., vps_layer_id) if it is determined that the current video coding layer is the only layer in the bitstream (e.g., sps_video_parameter_set_id is equal to 0) (e.g., only if it is determined).

[0023] In one implementation, the prediction unit may be further configured to predict the current slice of the current video coding layer using the identifier of the current video coding layer.

[0024] In one implementation, if it is determined that the bitstream contains multiple layers (for example, sps_video_parameter_set_id is greater than 0), the device may further include an acquisition unit configured to obtain the identifier of the current video coding layer by parsing the bitstream.

[0025] According to one implementation, the identifier of the current video coding layer may be used to construct one or more reference picture lists of the current slice of the current video coding layer, and the prediction unit is configured to predict the current slice of the current video coding layer using the identifier of the current video coding layer, and to predict the current slice using one or more reference picture lists. It can be configured in such a way .

[0026] According to one implementation, the NAL unit of the current video coding layer may be the VCL NAL unit of the current video coding layer that exists within the bitstream.

[0027] According to one implementation, the VCL NAL units of the current video coding layer can be any one (or all) VCL NAL units of the current video coding layer present in the bitstream.

[0028] According to a fourth aspect of the present invention, the present invention relates to an apparatus for encoding a video bitstream, the apparatus comprising: a determination unit configured to determine whether the current video coding layer is the only layer in the bitstream; and a prediction unit configured to use the identifier of the NAL unit of the current video coding layer as the identifier of the current video coding layer, if it is determined that the current video coding layer is the only layer in the bitstream.

[0029] In one implementation, the prediction unit may be further configured to predict the current slice of the current video coding layer using the identifier of the current video coding layer.

[0030] In one implementation, if it is determined that the bitstream contains multiple layers (for example, sps_video_parameter_set_id is greater than 0), the device may further include an encoding unit configured to encode the identifier of the current video coding layer into the bitstream.

[0031] In one implementation, the identifier of the current video coding layer may be used to construct one or more reference picture lists of the current slice of the current video coding layer, and the prediction unit may be configured to predict the current slice of the current video coding layer using the identifier of the current video coding layer, and may also be configured to predict the current slice using one or more reference picture lists.

[0032] In one implementation, the NAL unit of the current video coding layer is the VCL NAL unit of the current video coding layer that exists within the bitstream.

[0033] According to one implementation, the VCL NAL units of the current video coding layer are any one (or all) VCL NAL units of the current video coding layer present in the bitstream.

[0034] The method according to the first aspect of the present invention can be carried out by the apparatus according to the third aspect of the present invention. Further features and implementation forms of the apparatus according to the third aspect of the present invention correspond to the features and implementation forms of the method according to the first aspect of the present invention.

[0035] The method according to the second aspect of the present invention can be carried out by the apparatus according to the fourth aspect of the present invention. Further features and implementation forms of the apparatus according to the fourth aspect of the present invention correspond to the features and implementation forms of the method according to the second aspect of the present invention.

[0036] Each of the above methods can be implemented in a decoding device or an encoding device, and thus an encoder is provided that includes processing circuitry for performing a method of encoding a video bitstream according to any one of the above examples. Furthermore, an encoder is provided that includes one or more processors and a non-temporary computer-readable storage medium coupled to the processors and storing a program for execution by the processors, wherein the program, when executed by the processors, configures the encoder to perform a method of encoding a video bitstream according to any one of the above examples. Similarly, a decoder is provided that includes processing circuitry for performing a method of decoding a video bitstream according to any one of the above examples, and a decoder is provided that includes one or more processors and a non-temporary computer-readable storage medium coupled to the processors and storing a program for execution by the processors, wherein the program, when executed by the processors, configures the decoder to perform a method of decoding a video bitstream according to any one of the above examples.

[0037] Furthermore, a computer program product is provided that, when executed on a computer or processor, contains program code for performing one of the methods described in the examples above. Similarly, a non-temporary computer-readable medium is provided that, when executed by a computer device, carries program code that causes the computer device to perform one of the methods described in the examples above.

[0038] Details of one or more embodiments are described in the accompanying drawings and the following description. Other features, purposes, and advantages will become apparent from the specification, drawings, and claims.

[0039] Embodiments of the present invention will be described in more detail below with reference to the accompanying figures and drawings. [Brief explanation of the drawing]

[0040] [Figure 1A] This is a block diagram showing an example of a video coding system configured to implement embodiments of the present invention. [Figure 1B] This is a block diagram showing another example of a video coding system configured to implement embodiments of the present invention. [Figure 2] This is a block diagram showing an example of a video encoder configured to implement an embodiment of the present invention. [Figure 3] This is a block diagram illustrating an exemplary structure of a video decoder configured to implement embodiments of the present invention. [Figure 4] A block diagram showing an example of an encoding or decoding device. [Figure 5] This is a block diagram showing other examples of encoding or decoding devices. [Figure 6] This diagram shows an example of the raster scan order. [Figure 7] This figure shows examples of tiles, slices, and subpictures. [Figure 8] This figure shows an example of layers and sub-layers in scalable video coding. [Figure 9] This figure shows a method for decoding a video bitstream according to one embodiment. [Figure 10] This figure shows a method for encoding a video bitstream according to one embodiment. [Figure 11] This figure shows a method for decoding a video bitstream according to one embodiment. [Figure 12] This figure shows a method for encoding a video bitstream according to one embodiment. [Figure 13] This is a block diagram showing an exemplary structure of a content supply system 3100 that realizes a content distribution service. [Figure 14] A carbon image showing the structure of an example terminal device. [Modes for carrying out the invention]

[0041] The following description refers to the accompanying drawings, which form part of this disclosure and illustrate specific embodiments of the invention or specific ways in which embodiments of the invention may be used. It is understood that embodiments of the invention may be used in other ways, including structural or logical modifications not shown in the drawings. Therefore, the following detailed description should not be construed as restrictive, and the scope of the invention is defined by the appended claims.

[0042] For example, disclosures relating to a described method also apply to a corresponding device or system configured to perform that method, and vice versa. For example, where one or more specific method steps are described, the corresponding device may include one or more units (e.g., one unit that performs one or more steps, or multiple units, each performing one or more of the steps), such as functional units, to perform the described method step, even if such one or more units are not explicitly described or illustrated. On the other hand, for example, where a particular device is described based on one or more units, such as functional units, the corresponding method may include one step (e.g., one step that performs the function of one or more units, or multiple steps, each performing one or more of the functions of multiple units), even if such one or more steps are not explicitly described or illustrated. Furthermore, unless otherwise noted, it should be understood that the features of the various exemplary embodiments and / or aspects described herein may be combined with each other.

[0043] Video coding typically refers to the processing of a sequence of pictures that make up a video or video sequence. Instead of the term "picture," the terms "frame" or "image" may be used synonymously in the field of video coding. Video coding (or coding in general) consists of two parts: video encoding and video decoding. Video encoding is performed on the source side and typically involves processing the original video picture (e.g., by compression) to reduce the amount of data required to represent the video picture (for more efficient storage and / or transmission). Video decoding is performed on the destination side and typically involves the reverse processing of the encoder, reconstructing the video picture. Embodiments referring to "coding" a video picture (or picture in general) should be understood as relating to the "encoding" or "decoding" of the video picture or each video sequence. The combination of the encoding and decoding parts is also called a CODEC (Coding and Decoding).

[0044] In lossless video coding, the original video picture can be reconstructed, meaning the reconstructed video picture is of the same quality as the original (assuming there is no transmission loss or other data loss during storage or transmission). In lossy video coding, further compression is performed, for example, by quantization, to reduce the amount of data representing the video picture, which cannot be fully reconstructed in the decoder, meaning the quality of the reconstructed video picture is lower or inferior to that of the original video picture.

[0045] Some video coding standards belong to the group of “lossy hybrid video codecs” (i.e., they perform 2D transform coding to combine spatial and temporal prediction in the sample domain and apply quantization in the transform domain). Each picture in a video sequence is typically partitioned into a set of non-overlapping blocks, and coding is typically performed at the block level. In other words, in an encoder, video is typically processed, i.e., encoded, at the block (video block) level by, for example, using spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to generate prediction blocks, subtracting the prediction blocks from the current block (the block currently being processed / planned to be processed) to obtain residual blocks, and transforming the residual blocks to reduce (compress) the amount of data being transmitted by quantizing the residual blocks in the transform domain. On the other hand, in a decoder, the reverse processing of the encoder is applied to the encoded or compressed block to reconstruct the current block for representation. Furthermore, the encoder duplicates the decoder's processing loop so that both generate the same predictions (e.g., intra-prediction and inter-prediction) and / or processing of subsequent blocks, i.e., reconstruction for coding.

[0046] In the following embodiments of the video coding system 10, the video encoder 20 and the video decoder 30 are described with reference to Figures 1 to 3.

[0047] Figure 1A is a schematic block diagram showing an exemplary coding system 10, for example, a video coding system 10 (or short coding system 10), which may utilize the technique of the present application. The video encoder 20 (or short encoder 20) and video decoder 30 (or short decoder 30) of the video coding system 10 represent examples of devices that may be configured to perform the technique according to the various examples described in this application.

[0048] As shown in Figure 1A, the coding system 10 encodes the picture data 21, for example, the encoded picture data 21 Includes a source device 12 configured to provide to a destination device 14 for decoding.

[0049] The source device 12 includes an encoder 20 and may additionally, i.e., optionally, include a picture source 16, a preprocessor (or preprocessing unit) 18, for example, a picture preprocessor 18, and a communication interface or communication unit 22.

[0050] The picture source 16 includes, or may include, any kind of picture capture device, e.g., a camera for capturing real-world pictures, and / or any kind of picture generation device, e.g., a computer graphics processor for generating computer-animated pictures, or any kind of other device for acquiring / providing real-world pictures, computer-generated pictures (e.g., screen content, virtual reality (VR) pictures), and / or any combination thereof (e.g., augmented reality (AR) pictures). The picture source may be any kind of memory or storage for storing any of the aforementioned pictures.

[0051] To distinguish it from the processing performed by the preprocessor 18 and the preprocessing unit 18, the picture or picture data 17 may also be called the raw picture or raw picture data 17.

[0052] The preprocessor 18 is configured to receive (raw) picture data 17 and perform preprocessing on the picture data 17 in order to obtain a preprocessed picture 19 or preprocessed picture data 19. Preprocessing performed by the preprocessor 18 may include, for example, cropping, color format conversion (e.g., RGB to YCbCr), color correction, or denoising. It should be understood that the preprocessing unit 18 may be an optional component.

[0053] The video encoder 20 is configured to receive pre-processed picture data 19 and provide encoded picture data 21 (further details are described below, for example, based on Figure 2).

[0054] The communication interface 22 of the source device 12 may be configured to receive the encoded picture data 21 and transmit the encoded picture data 21 (or a further processed version thereof) via the communication channel 13 to another device, for example, the destination device 14 or any other device for storage or direct reconstruction.

[0055] The destination device 14 includes a decoder 30 (for example, a video decoder 30), and may also include, optionally, a communication interface or communication unit 28, a post-processor 32 (or post-processing unit 32), and a display device 34.

[0056] The communication interface 28 of the destination device 14 is configured to receive encoded picture data 21 (or a further processed version thereof) for example directly from the source device 12 or from any other source, such as a storage device, such as an encoded picture data storage device, and to provide the encoded picture data 21 to the decoder 30.

[0057] Communication interfaces 22 and 28 transmit encoded picture data 21 or encoded data via a direct communication link between source device 12 and destination device 14, for example, via a direct wired or wireless connection, or via any type of network, for example, a wired or wireless network or any combination thereof, or any type of private and public network or any combination thereof. Ta It can be configured to transmit or receive.

[0058] The communication interface 22 may be configured to process the encoded picture data, for example, by packaging the encoded picture data 21 into an appropriate format, such as a packet, and / or by using any kind of transmission encoding or processing for transmission over a communication link or communication network.

[0059] A communication interface 28 forming a counterpart to communication interface 22 may be configured, for example, to process the transmitted data using any kind of corresponding transmission decoding or processing and / or depackaging in order to receive the transmitted data and obtain the encoded picture data 21.

[0060] Both communication interfaces 22 and 28 may be configured as unidirectional or bidirectional communication interfaces, as indicated by the arrows on communication channel 13 in Figure 1A pointing from source device 12 to destination device 14, and may be configured to set up connections, for example, to send and receive messages, in order to confirm and exchange any other information relating to the communication link and / or data transmission, such as encoded picture data transmission.

[0061] The decoder 30 is configured to receive encoded picture data 21 and provide decoded picture data 31 or decoded picture 31 (further details are described below, for example, based on Figure 3 or Figure 5).

[0062] The post-processor 32 of the destination device 14 is configured to post-process the decoded picture data 31 (also called reconstructed picture data), for example, the decoded picture 31, in order to obtain post-processed picture data 33, for example, the post-processed picture 33. The post-processing performed by the post-processing unit 32 may include, for example, color format conversion (e.g., YCbCr to RGB), color correction, cropping, or resampling, or any other processing, to prepare the decoded picture data 31 for display by the display device 34.

[0063] The display device 34 of the destination device 14 is configured to receive post-processed picture data 33 for displaying the picture to, for example, a user or viewer. The display device 34 may include any type of display for representing the reconstructed picture, such as an internal or external display or monitor. The display may include, for example, a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, a plasma display, a projector, a microLED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display.

[0064] Although Figure 1A shows the source device 12 and the destination device 14 as separate devices, the device embodiments may also include the functions of both or both of the source device 12 or its corresponding functions and the destination device 14 or its corresponding functions. In such embodiments, the source device 12 or its corresponding functions and the destination device 14 or its corresponding functions may be implemented using the same hardware and / or software, or by separate hardware and / or software, or any combination thereof.

[0065] As will be apparent to those skilled in the art based on this description, the presence and (exact) split of different units or functions within the source device 12 and / or destination device 14 may vary depending on the actual device and application, as shown in Figure 1A.

[0066] An encoder 20 (for example, a video encoder 20) or a decoder 30 (for example, a video decoder 30), or both an encoder 20 and a decoder 30, may be implemented via a processing circuit shown in Figure 1B, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, dedicated to video coding, or any combination thereof. The encoder 20 may be implemented via a processing circuit 46 to embody various modules described with respect to the encoder 20 in Figure 2 and / or any other encoder systems or subsystems described herein. The decoder 30 may be implemented via a processing circuit 46 to embody various modules described with respect to the decoder 30 in Figure 3 and / or any other decoder systems or subsystems described herein. The processing circuits may be configured to perform various operations, as described later. If the technique is partially implemented in software, as shown in Figure 5, the device may store software instructions in a suitable non-temporary computer-readable storage medium and execute the instructions in hardware using one or more processors to perform the technique of this disclosure. Either the video encoder 20 or the video decoder 30 can be integrated as part of a composite encoder / decoder (CODEC) within a single device, for example, as shown in Figure 1B.

[0067] The source device 12 and destination device 14 may include any type of handheld or fixed device, such as a notebook or laptop computer, mobile phone, smartphone, tablet or tablet computer, camera, desktop computer, set-top box, television, display device, digital media player, video game console, video streaming device (such as a content service server or content distribution server), broadcast receiving device, broadcast transmitter device, etc., and may or may not use an operating system. In some cases, the source device 12 and destination device 14 may be equipped for wireless communication. Thus, the source device 12 and destination device 14 may be wireless communication devices.

[0068] In some cases, the video coding system 10 shown in Figure 1A is merely an example, and the techniques of this application may be applicable to video coding configurations (e.g., video encoding or video decoding) that do not necessarily involve data communication between an encoding device and a decoding device. In other examples, data may be retrieved from local memory or streamed over a network. A video encoding device may encode data and store it in memory, and / or a video decoding device may retrieve and decode data from memory. In some examples, encoding and decoding are performed by devices that do not communicate with each other but simply encode data into memory and / or retrieve and decode data from memory.

[0069] For illustrative purposes, embodiments of the present invention are described herein by reference to, for example, High-Efficiency Video Coding (HEVC) or Versatile Video Coding (VVC), a next-generation video coding standard developed by the ITU-T Video Coding Experts Group (VCEG)'s Joint Collaboration Team on Video Coding (JCT-VC) and ISO / IEC Motion Picture Experts Group (MPEG). Those skilled in the art will understand that embodiments of the present invention are not limited to HEVC or VVC.

[0070] Encoder and encoding method Figure 2 shows a schematic block diagram of an exemplary video encoder 20 configured to implement the technique of the present application. In the example of Figure 2, the video encoder 20 includes an input 201 (or input interface 201), a residual calculation unit 204, a transformation processing unit 206, a quantization unit 208, an inverse quantization unit 210, and an inverse transformation processing unit 212, a reconstruction unit 214, a loop filter unit 220, a decoded picture buffer (DPB) 230, a mode selection unit 260, an entropy encoding unit 270, and an output 272 (or output interface 272). The mode selection unit 260 may include an inter-prediction unit 244, an intra-prediction unit 254, and a partitioning unit 262. The inter-prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). The video encoder 20 shown in Figure 2 may also be called a hybrid video encoder or a video encoder with a hybrid video codec.

[0071] The residual calculation unit 204, the conversion processing unit 206, the quantization unit 208, and the mode selection unit 260 are sometimes said to form the forward signal path of the encoder 20, while the inverse quantization unit 210, the inverse conversion processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the decoded picture buffer (DPB) 230, the inter-prediction unit 244, and the intra-prediction unit 254 are sometimes said to form the reverse signal path of the video encoder 20, with the reverse signal path of the video encoder 20 corresponding to the signal path of the decoder (see video decoder 30 in Figure 3). The inverse quantization unit 210, the inverse conversion processing unit 212, the reconstruction unit 214, the loop filter 220, the decoded picture buffer (DPB) 230, the inter-prediction unit 244, and the intra-prediction unit 254 are also sometimes said to form the "built-in decoder" of the video encoder 20.

[0072] Pictures and picture partitioning (pictures and blocks) The encoder 20 may be configured to receive a picture 17 (or picture data 17) via input 201, for example, a picture of a sequence of pictures that form a video or video sequence. The received picture or picture data may also be a pre-processed picture 19 (or pre-processed picture data 19). For simplicity, the following description will refer to the picture 17. The picture 17 may also be called the current picture or the picture being coded (particularly in video coding to distinguish the current picture from other pictures, for example, previously encoded and / or decoded pictures of the same video sequence, i.e., video sequences that also contain the current picture).

[0073] A (digital) picture is, or can be considered as, a two-dimensional array or matrix of samples with intensity values. Samples within the array may also be called pixels (a shortened form of picture element) or pels. The number of samples in the horizontal and vertical (or axis) directions of the array or picture defines the size and / or resolution of the picture. Three color components are typically used to represent color; that is, a picture may be represented by a three-sample array, or it may contain a three-sample array. In RGB format or color space, a picture contains corresponding red, green, and blue sample arrays. However, in video coding, each pixel is typically represented in luminance and chrominance format or color space, for example, YCbCr, which contains a luminance component represented by Y (sometimes L is used instead) and two chrominance components represented by Cb and Cr. The luminance (or short luminance) component Y represents the intensity of the brightness or gray level (for example, as in a grayscale picture), and the two chrominance (or short saturation) components Cb and Cr represent the chromaticity or color information components. Thus, a picture in YCbCr format contains a luminance sample array of luminance sample values ​​(Y) and two chrominance sample arrays of chrominance values ​​(Cb and Cr). An RGB format picture may be converted to or transformed into YCbCr format, and vice versa; this process is also known as color conversion or transformation. If the picture is monochrome, the picture may contain only a luminance sample array. Thus, a picture could be, for example, an array of luminance samples in monochrome format, or an array of luminance samples and two corresponding arrays of saturation samples in 4:2:0, 4:2:2, and 4:4:4 color formats.

[0074] Embodiments of the video encoder 20 may include a picture partitioning unit (not shown in Figure 2) configured to partition a picture 17 into multiple (typically non-overlapping) picture blocks 203. These blocks may also be called root blocks, macroblocks (H.264 / AVC), coding tree blocks (CTB), or coding tree units (CTU) (H.265 / HEVC and VVC). The picture partitioning unit may be configured to use the same block size for all pictures in a video sequence and for the corresponding grid that defines the block size, or to change the block size between pictures or subsets or groups of pictures, and to partition each picture into the corresponding block.

[0075] In a further embodiment, the video encoder may be configured to directly receive blocks 203 of picture 17, for example, one, more, or all of the blocks that make up picture 17. Picture blocks 203 may also be referred to as the current picture block or the coded picture block.

[0076] Similar to picture 17, picture block 203 is, or can be considered as, a two-dimensional array or matrix of samples having intensity values ​​(sample values), although it has fewer dimensions than picture 17. In other words, block 203 can contain, for example, one sample array (e.g., a luminance array for monochrome picture 17, or a luminance or saturation array for color picture) or three sample arrays (e.g., one luminance array and two saturation arrays for color picture 17), or any other number and / or type of arrays depending on the color format applied. The number of samples in the horizontal and vertical (or axis) directions of block 203 defines the size of block 203. Thus, a block can be, for example, an M×N (M columns × N rows) array of samples, or an M×N array of conversion coefficients.

[0077] The embodiment of the video encoder 20 shown in Figure 2 may be configured to encode the picture 17 block by block, for example, encoding and prediction may be performed for each block 203.

[0078] The embodiment of the video encoder 20 shown in Figure 2 may be further configured to partition and / or encode a picture by using slices (also called video slices), the picture may be partitioned or encoded using one or more slices (usually non-overlapping), each slice may contain one or more blocks (e.g., CTUs).

[0079] The embodiment of the video encoder 20 shown in Figure 2 may be further configured to partition and / or encode a picture by using tile groups (also called video tile groups) and / or tiles (also called video tiles), the picture may be partitioned or encoded using one or more tile groups (usually non-overlapping), each tile group may, for example, contain one or more blocks (e.g., CTUs) or one or more tiles, each tile may be, for example, a rectangle, may contain one or more blocks (e.g., CTUs), and may be, for example, a complete block or a partial block.

[0080] Residual calculation The residual calculation unit 204 may be configured to calculate the residual block 205 (also called residual 205) based on the picture block 203 and the prediction block 265, for example, by subtracting the sample values ​​of the prediction block 265 from the sample values ​​of the picture block 203 sample by sample (pixel by pixel) in order to obtain the residual block 205 in the sample region (further details about the prediction block 265 will be provided later).

[0081] conversion The transformation processing unit 206 may be configured to apply a transformation, for example, a discrete cosine transform (DCT) or discrete sine transform (DST) to the sample values ​​of the residual block 205, in order to obtain the transformation coefficients 207 in the transformation domain. The transformation coefficients 207 are also called transformation residual coefficients and represent the residual block 205 in the transformation domain.

[0082] The conversion processing unit 206 may be configured to apply an integer approximation of the DCT / DST, such as the conversion specified for H.265 / HEVC. Compared to the orthogonal DCT conversion, such an integer approximation is typically scaled by a specific coefficient. An additional scaling coefficient is applied as part of the conversion process to preserve the norm of the residual blocks processed by the forward and inverse conversions. The scaling coefficient is typically selected based on specific constraints, such as a scaling coefficient that is a power of 2 of the shift operation, the bit depth of the conversion coefficient, and a trade-off between precision and implementation cost. A specific scaling coefficient may be specified, for example, for the inverse conversion, by the inverse conversion processing unit 212 (and the corresponding inverse conversion, for example, by the inverse conversion processing unit 312 in the video decoder 30), and the corresponding scaling coefficient for the forward conversion in the encoder 20, for example, by the conversion processing unit 206, may be specified accordingly.

[0083] Embodiments of the video encoder 20 (each a conversion processing unit 206) may be configured to output conversion parameters, such as one or more types of conversion, encoded or compressed, for example, directly or via the entropy encoding unit 270, so that the video decoder 30 can receive and use the conversion parameters for decoding.

[0084] quantization The quantization unit 208 may be configured to quantize the transformation coefficient 207 in order to obtain the quantized coefficient 209, for example by applying scalar quantization or vector quantization. The quantized coefficient 209 may also be called the quantized transformation coefficient 209 or the quantized residual coefficient 209.

[0085] The quantization process can reduce the bit depth associated with some or all of the 207 conversion coefficients. For example, n-bit conversion coefficients may be rounded to m-bit conversion coefficients during quantization, where n is greater than m. The degree of quantization can be modified by adjusting the quantization parameter (QP). For example, in the case of scalar quantization, different scalings may be applied to achieve finer or coarser quantization. Smaller quantization step sizes correspond to finer quantization, and larger quantization step sizes correspond to coarser quantization. Applicable quantization step sizes can be indicated by the quantization parameter (QP). The quantization parameter may be, for example, an index to a predetermined set of applicable quantization step sizes. For example, a small quantization parameter may correspond to finer quantization (smaller quantization step size), a large quantization parameter may correspond to coarser quantization (larger quantization step size), and vice versa. Quantization may involve division by the quantization step size, and the corresponding dequantization and / or inverse dequantization by, for example, the inverse quantization unit 210 may involve multiplication by the quantization step size. Embodiments by several standards, such as HEVC, may be configured to use a quantization parameter to determine the quantization step size. In general, the quantization step size may be calculated based on the quantization parameter using a fixed-point approximation of the expression involving division. Additional scaling factors may be introduced into quantization and dequantization to restore the norm of the residual block, which may be modified for the scaling used in the fixed-point approximation of the quantization step size and quantization parameter expression. In one exemplary implementation, scaling and dequantization of the inverse transform can be combined. Alternatively, a customized quantization table may be used and signaled, for example, from encoder to decoder in a bitstream. Quantization is an irreversible operation, and the loss increases with increasing quantization step size.

[0086] Embodiments of the video encoder 20 (each a quantization unit 208) may be configured to output quantization parameters (QP) that are encoded, for example, directly or via the entropy encoding unit 270, so that, for example, the video decoder 30 can receive and apply the quantization parameters for decoding.

[0087] inverse quantization The inverse quantization unit 210 is configured to apply the inverse quantization of the quantization unit 208 to the quantized coefficients in order to obtain the unquantized coefficients 211, for example, by applying the inverse of the quantization scheme applied by the quantization unit 208, based on or using the same quantization step size as the quantization unit 208. The unquantized coefficients 211 are also called unquantized residual coefficients 211 and may correspond to the transformation coefficients 207, although they are not typically identical to the transformation coefficients due to losses due to quantization.

[0088] Inverse Transform The inverse transform processing unit 212 is configured to apply the inverse transform of the transform applied by the transform processing unit 206, such as the inverse discrete cosine transform (DCT) or the inverse discrete sine transform (DST) or other inverse transform, in order to obtain the reconstructed residual block 213 (or the corresponding unquantized coefficient 213) in the sample region. The reconstructed residual block 213 is sometimes referred to as the transform block 213.

[0089] Rebuild The reconstruction unit 214 (for example, an adder or analog adder 214) is configured to add the transformed block 213 (i.e., the reconstructed residual block 213) to the predicted block 265 by adding, for example, the sample value of the reconstructed residual block 213 to the sample value of the predicted block 265, sample by sample, in order to obtain the reconstructed block 215 in the sample domain.

[0090] filtering The loop filter unit 220 (or, for short, the "loop filter" 220) is configured to filter the reconstructed block 215 to obtain the filtered block 221, or generally to filter the reconstructed sample to obtain the filtered sample. The loop filter unit is configured, for example, to smooth pixel transitions or otherwise improve video quality. The loop filter unit 220 may include a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as one or more loop filters, or any combination thereof, including a bilateral filter, an adaptive loop filter (ALF), a sharpening filter, a smoothing filter, or a co-filter. In Figure 2, the loop filter unit 220 is shown as an in-loop filter, but in other configurations, the loop filter unit 220 may be implemented as a post-loop filter. The filtered block 221 is sometimes referred to as the filtered reconstructed block 221.

[0091] Embodiments of the video encoder 20 (each a loop filter unit 220) may be configured to output loop filter parameters (such as sample-adaptive offset information) that are encoded, for example, directly or via the entropy encoding unit 270, so that a decoder 30, for example, can receive and apply the same loop filter parameters or the respective loop filters for decoding.

[0092] Decoded picture buffer The decoded picture buffer (DPB) 230 may be a memory that stores a reference picture, or more generally, reference picture data, for encoding video data by the video encoder 20. The DPB 230 may be formed by any of various memory devices, such as dynamic random access memory (DRAM) including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The decoded picture buffer (DPB) 230 may be configured to store one or more filtered blocks 221. The decoded picture buffer 230 may be further configured to store other previously filtered blocks, for example, previously reconstructed and filtered blocks 221 of the same current picture or a different picture, for example, a previously reconstructed picture, for example, to provide, for example, a complete previously reconstructed, i.e., decoded picture (and the corresponding reference block and sample) and / or a partially reconstructed current picture (and the corresponding reference block and sample) for interpretation. The decoded picture buffer (DPB) 230 may also be configured to store, for example, one or more unfiltered reconstructed blocks 215, or generally unfiltered reconstructed samples, or any other further processed versions of the reconstructed blocks or samples, if the reconstructed blocks 215 are not filtered by the loop filter unit 220.

[0093] Mode selection (partitioning and prediction) The mode selection unit 260 includes a partitioning unit 262, an inter-prediction unit 244, and an intra-prediction unit 254, and is configured to receive or acquire original picture data, for example, the original block 203 (the current block 203 of the current picture 17), and reconstructed picture data, for example, filtered and / or unfiltered reconstructed samples or blocks of the same (current) picture, and / or filtered and / or unfiltered reconstructed samples or blocks from one or more previously decoded pictures, for example, from a decoded picture buffer 230 or other buffers (for example, a line buffer, not shown). The reconstructed picture data is used as reference picture data for prediction, for example, inter-prediction or intra-prediction, to acquire prediction blocks 265 or predictors 265.

[0094] The mode selection unit 260 may be configured to determine or select the partitioning for the current block prediction mode (including no partitioning) and prediction mode (e.g., intra or inter-prediction mode), and to generate the corresponding prediction block 265, which is used for calculating the residual block 205 and reconstructing the reconstructed block 215.

[0095] Embodiments of the mode selection unit 260 may be configured to select a partitioning and prediction mode (for example, from those supported by or available to the mode selection unit 260) that provides an optimal match, or in other words, the smallest residual (smallest residual means better compression of transmission or storage), or the smallest signaling overhead (smallest signaling overhead means better compression of transmission or storage), or consider both, or strike a balance. The mode selection unit 260 may be configured to determine the partitioning and prediction mode based on rate distortion optimization (RDO), i.e., to select a prediction mode that provides the smallest rate distortion. In this context, terms such as “best,” “smallest,” and “optimal” do not necessarily refer to an overall “best,” “smallest,” and “optimal,” but may also refer to meeting termination or selection criteria, such as a value being above or below a threshold, or other constraints that may lead to a “suboptimal choice” but reduce complexity and processing time.

[0096] In other words, the partitioning unit 262 may be configured to partition block 203 into smaller block partitions or subblocks (forming blocks again) in order to make predictions for each of the block partitions or subblocks, for example, by repeatedly using quadtree partitioning (QT), binary tree partitioning (BT), or ternary tree partitioning (TT), or any combination thereof, and the mode selection includes selecting the tree structure of the partitioning block 203, and the prediction mode is applied to each of the block partitions or subblocks.

[0097] The following describes partitioning performed by the example video encoder 20 (for example, partitioning unit 262 The prediction process (by the inter-prediction unit 244 and the intra-prediction unit 254) will be explained in more detail.

[0098] Partitioning The partitioning unit 262 may partition (or split) the current block 203 into smaller partitions, for example, smaller blocks of square or rectangular size. These smaller blocks (also called subblocks) may be further partitioned into even smaller partitions. This is also called tree partitioning or hierarchical tree partitioning. For example, the root block at root tree level 0 (hierarchical level 0, depth 0) may be recursively partitioned, for example, into two or more blocks of nodes at the next lower tree level, for example, tree level 1 (hierarchical level 1, depth 1), and these blocks may be further partitioned into two or more blocks at the next lower level, for example, tree level 2 (hierarchical level 2, depth 2), and so on, until partitioning ends, for example, because a termination criterion is met, for example, because the maximum depth of the tree or the minimum block size has been reached. Blocks that are not further partitioned are also called leaf blocks or leaf nodes of the tree. A tree that uses partitioning into two partitions is called a binary tree (BT), a tree that uses partitioning into three partitions is called a ternary tree (TT), and a tree that uses partitioning into four partitions is called a quadary tree (QT).

[0099] As stated herein, the term “block” can refer to a portion of a picture, particularly a square or rectangular portion. For example, with reference to HEVC and VVC, a block may be a coding tree unit (CTU), coding unit (CU), prediction unit (PU), and transformation unit (TU), and / or a corresponding block, such as a coding tree block (CTB), coding block (CB), transformation block (TB), or prediction block (PB).

[0100] For example, a coding tree unit (CTU) may be a CTB of luminance samples, two corresponding CTBs of saturation samples of a picture having three sample arrays, or a CTB of samples of a picture coded using three separate color planes and syntactic structures used to code a monochrome picture or sample, or may include these. Correspondingly, just as the division of a component into CTBs is partitioning, a coding tree block (CTB) may be an N×N block of samples for a certain value N. A coding unit (CT) may be a coding block of luminance samples, two corresponding coding blocks of saturation samples of a picture having three sample arrays, or a coding block of samples of a picture coded using three separate color planes and syntactic structures used to code a monochrome picture or sample, or may include these. Correspondingly, just as the division of a CTB into coding blocks is partitioning, a coding block (CB) may be an M×N block of samples for a certain value M and N.

[0101] In embodiments, for example, according to HEVC, a coding tree unit (CTU) can be split into CUs by using a quadtree structure, which is shown as a coding tree. The decision of whether to code the picture region using interpicture (time) or intrapicture (spatial) prediction is made at the CU level. Each CU can be further split into one, two, or four PUs, according to the PU split type. Within a single PU, the same prediction process is applied, and relevant information is transmitted to the decoder on a PU basis. After obtaining residual blocks by applying the prediction process based on the PU split type, the CUs can be partitioned into translation units (TUs) according to other quadtree structures similar to the coding tree of the CU.

[0102] In embodiments, for example, according to the latest video coding standard currently under development called Versatile Video Coding (VVC), combined quadtree and binary tree (QTBT) partitioning is used, for example, to partition coding blocks. In a QTBT block structure, CUs can have either a square or rectangular shape. For example, a coding tree unit (CTU) is first partitioned by a quadtree structure. The quadtree leaf nodes are further partitioned by a binary tree or ternary (or triple) tree structure. The partitioned tree leaf nodes are called coding units (CUs), and their segmentation is used for prediction and transformation processing without further partitioning. This means that CUs, PUs, and TUs have the same block size in a QTBT coding block structure. In parallel, multiple partitions, such as ternary tree partitions, can be used with the QTBT block structure.

[0103] In one example, the mode selection unit 260 of the video encoder 20 may be configured to perform any combination of the partitioning techniques described herein.

[0104] As described above, the video encoder 20 is configured to determine or select the best or most optimal prediction mode from a set of prediction modes (for example, a predetermined set). The set of prediction modes may include, for example, an intra-prediction mode and / or an inter-prediction mode.

[0105] Intra Prediction The set of intra-prediction modes may include 35 different intra-prediction modes, such as omnidirectional modes including DC (or average) mode and planar mode, or directional modes as defined in HEVC, for example, or 67 different intra-prediction modes, such as omnidirectional modes including DC (or average) mode and planar mode, or directional modes as defined in VVC, for example.

[0106] The intra-prediction unit 254 is configured to use reconstructed samples of adjacent blocks of the same current picture to generate an intra-prediction block 265 according to the intra-prediction mode of a set of intra-prediction modes.

[0107] The intra-prediction unit 254 (or generally the mode selection unit 260) is further configured to output intra-prediction parameters (or generally information indicating the selected intra-prediction mode for a block) to the entropy encoding unit 270 in the form of syntax elements 266 to be included in the encoded picture data 21 so that the video decoder 30 can receive and use the prediction parameters for decoding.

[0108] Interpretation The set of interpretation modes (or possible interpretation modes) depends on the available reference picture (i.e., a previously at least partially decoded picture, e.g., one stored in DBP 230) and other interpretation parameters, e.g., whether the entire reference picture or only a portion of it is used to search for the most matching reference block, e.g., whether the search window area around the current block region of the reference picture is used to search for the most matching reference block, and / or whether pixel interpolation is applied, e.g., half / semi-perpetual and / or quarter-perpetual interpolation.

[0109] In addition to the prediction modes described above, skip mode and / or direct mode may be applied.

[0110] The interpretation unit 244 may include a motion estimation (ME) unit and a motion compensation (MC) unit (neither of which are shown in Figure 2). The motion estimation unit may be configured to receive or acquire, for motion estimation, a picture block 203 (the current picture block 203 of the current picture 17) and a decoded picture 231, or at least one or more previously reconstructed blocks, for example, one or more reconstructed blocks of one or more other / different previously decoded pictures 231. For example, a video sequence may include the current picture and a previously decoded picture 231, or in other words, the current picture and a previously decoded picture 231 may be part of a sequence of pictures that make up the video sequence, or may make up the video sequence.

[0111] The encoder 20 may be configured to, for example, select a reference block from multiple reference blocks of the same or different pictures of multiple other pictures, and also to provide, for example, the reference picture (or reference picture index) and / or offset (spatial offset) between the position (x, y coordinates) of the reference block and the position of the current block as interprediction parameters to the motion estimation unit. This offset is also called the motion vector (MV).

[0112] The motion compensation unit is configured to obtain, for example, receive, and perform interprediction based on or using the interprediction parameters in order to obtain interprediction block 265. Motion compensation performed by the motion compensation unit may include fetching or generating prediction blocks based on the motion / block vector determined by motion estimation, and possibly interpolating to sub-pixel precision. Interpolation filtering may generate additional pixel samples from known pixel samples, thus increasing the number of candidate prediction blocks that can be used to code picture blocks. Upon receiving the motion vector of the current picture block, the motion compensation unit can pinpoint the prediction block that the motion vector points to in one of the reference picture lists.

[0113] The motion compensation unit may also generate blocks and syntactic elements associated with video slices that are used by the video decoder 30 when decoding picture blocks of video slices. In addition to slices and their respective syntactic elements, or as alternatives thereto, tile groups and / or tiles, as well as their respective syntactic elements, may be generated or used.

[0114] Entropy coding The entropy encoding unit 270 is configured to apply, for example, an entropy encoding algorithm or scheme (e.g., variable-length coding (VLC) scheme, context-adaptive VLC scheme (CAVLC), arithmetic coding scheme, binarization, context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding or other entropy encoding methodology or technique), or quantized coefficient bypass (uncompressed) 209, inter-prediction parameters, intra-prediction parameters, loop filter parameters and / or other syntactic elements, in order to obtain encoded picture data 21 that can be output via output 272 in the form of an encoded bitstream 21, for example, so that the video decoder 30 can receive parameters and use them for decoding. The encoded bitstream 21 may be transmitted to the video decoder 30 or stored in memory for later transmission or retrieval by the video decoder 30.

[0115] Other structural variations of the video encoder 20 can be used to encode a video stream. For example, a non-conversion-based encoder 20 can directly quantize the residual signal for a particular block or frame without a conversion processing unit 206. In other implementations, the encoder 20 can combine the quantization unit 208 and the inverse quantization unit 210 into a single unit.

[0116] Decoder and decoding method Figure 3 shows an example of a video decoder 30 configured to implement the technique of the present application. The video decoder 30 is configured to receive encoded picture data 21 (e.g., encoded bitstream 21), which has been encoded by, for example, the encoder 20, in order to obtain a decoded picture 331. The encoded picture data or bitstream contains information for decoding the encoded picture data, for example, data representing a picture block of encoded video slices (and / or tile groups or tiles) and associated syntactic elements.

[0117] In the example in Figure 3, the decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transformation unit 312, a reconstruction unit 314 (e.g., an analog adder 314), a loop filter 320, a decoded picture buffer (DBP) 330, a mode application unit 360, an interpretation unit 344, and an intraprediction unit 354. The interpretation unit 344 may be a motion compensation unit, or may include a motion compensation unit. In some examples, the video decoder 30 may perform a decoding path that is roughly the reverse of the encoding path described with respect to the video encoder 100 from Figure 2.

[0118] As described with respect to encoder 20, the inverse quantization unit 210, inverse transformation processing unit 212, reconstruction unit 214, loop filter 220, decoded picture buffer (DPB) 230, inter-prediction unit 344, and intra-prediction unit 354 are also referred to as the “built-in decoder” of video encoder 20. Thus, the inverse quantization unit 310 is the inverse quantization unit 210The inverse processing unit 312 may be functionally identical to the inverse processing unit 212, the reconstruction unit 314 may be functionally identical to the reconstruction unit 214, the loop filter 320 may be functionally identical to the loop filter 220, and the decoded picture buffer 330 may be functionally identical to the decoded picture buffer 230. Therefore, the video Ugh ncoda 20 The descriptions provided for each unit and function apply to each unit and function of the video decoder 30.

[0119] Entropy decoding The entropy decoding unit 304 is configured to parse the bitstream 21 (or generally encoded picture data 21) to obtain, for example, some or all of the quantized coefficients 309 and / or decoded coding parameters (not shown in Figure 3), such as inter-prediction parameters (e.g., reference picture index and motion vector), intra-prediction parameters (e.g., intra-prediction mode or index), transformation parameters, quantization parameters, loop filter parameters, and / or other syntactic elements, and to perform entropy decoding on the encoded picture data 21, for example. The entropy decoding unit 304 may be configured to apply a decoding algorithm or scheme corresponding to an encoding scheme, as described with respect to the entropy encoding unit 270 of the encoder 20. The entropy decoding unit 304 may be further configured to provide the inter-prediction parameters, intra-prediction parameters, and / or other syntactic elements to the mode application unit 360 and other parameters to other units of the decoder 30. The video decoder 30 may receive syntactic elements at the video slice level and / or video block level. In addition to slices and their respective syntactic elements, or as alternatives thereto, tile groups and / or tiles, and their respective syntactic elements may be received and / or used.

[0120] inverse quantization The inverse quantization unit 310 may be configured to receive quantization parameters (QP) (or information generally related to inverse quantization) and quantized coefficients from the encoded picture data 21 (for example, by parsing and / or decoding by the entropy decoding unit 304), and to apply inverse quantization to the decoded quantization coefficients 309 based on the quantization parameters to obtain unquantized coefficients 311, also called conversion coefficients 311. The inverse quantization process may include using the quantization parameters determined by the video encoder 20 for each video block in the video slice (or tile or tile group) to determine the degree of quantization and, similarly, the degree of inverse quantization to be applied.

[0121] Inverse Transform The inverse transformation processing unit 312 may be configured to receive non-quantized coefficients 311, also called transformation coefficients 311, and to apply a transformation to the non-quantized coefficients 311 in order to obtain a reconstructed residual block 213 in the sample region. The reconstructed residual block 213 may also be called a transformation block 313. The transformation may be an inverse transformation, such as an inverse DCT, inverse DST, inverse integer transformation, or a conceptually similar inverse transformation process. The inverse transformation processing unit 312 may be further configured to receive transformation parameters or corresponding information from the encoded picture data 21 (for example, by an entropy decoding unit 304, for example, by parsing and / or decoding) in order to determine the transformation to be applied to the non-quantized coefficients 311.

[0122] Rebuild The reconstruction unit 314 (for example, an adder or analog adder 314) may be configured to add the reconstructed residual block 313 to the predicted block 365, for example, by adding the sample values ​​of the reconstructed residual block 313 to the sample values ​​of the predicted block 365, in order to obtain the reconstructed block 315 in the sample domain.

[0123] filtering The loop filter unit 320 (either within or after the coding loop) is configured to filter the reconstructed block 315 to obtain the filtered block 321, for example, to smooth pixel transitions or otherwise improve video quality. The loop filter unit 320 may include a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening filter, a smoothing filter, or a co-filter, or any combination thereof. In Figure 3, the loop filter unit 320 is shown as an in-loop filter, but in other configurations, the loop filter unit 320 may be implemented as a post-loop filter.

[0124] Decoded picture buffer Next, the decoded video block 321 of the picture is stored in the decoded picture buffer 330, which stores the decoded picture 331 as a reference picture for subsequent motion compensation of other pictures and / or for their respective output displays.

[0125] Decoder 30 processes the decoded picture 331 It is configured to output, for example, via output 312, for presentation or viewing by the user.

[0126] prediction The inter-prediction unit 344 may be identical to the inter-prediction unit 244 (in particular, the motion compensation unit), and the intra-prediction unit 354 may be functionally identical to the inter-prediction unit 254, and perform split or partitioning decisions and predictions based on partitioning and / or prediction parameters received from the encoded picture data 21 (for example, by parsing and / or decoding by the entropy decoding unit 304), or based on the respective information. The mode application unit 360 may be configured to perform block-by-block predictions (intra-predictions or inter-predictions) based on the reconstructed picture, block, or each sample (filtered or unfiltered) in order to obtain prediction blocks 365.

[0127] If a video slice is coded as an intra-coded (I) slice, the intra-prediction unit 354 of the mode-applying unit 360 is configured to generate a prediction block 365 for the picture block of the current video slice based on the signaled intra-prediction mode and data from previously decoded blocks of the current picture. If a video picture is coded as an inter-coded (i.e., B or P) slice, the inter-prediction unit 344 (e.g., a motion compensation unit) of the mode-applying unit 360 is configured to generate a prediction block 365 for the video block of the current video slice based on motion vectors and other syntactic elements received from the entropy decoding unit 304. In the case of inter-prediction, the prediction block may be generated from one of the reference pictures contained in one of the reference picture lists. The video decoder 30 may construct the reference frame list, list 0 and list 1 using a default construction technique based on the reference pictures stored in the DPB 330. For embodiments in which tile groups (e.g., video tile groups) and / or tiles (e.g., video tiles) are used in addition to or as a substitute for slices (e.g., video slices), or depending on the embodiment, the same or similar may apply, for example, video may be coded using I, P, or B tile groups and / or tiles.

[0128] The mode-applying unit 360 is configured to determine predictive information for video blocks in the current video slice by parsing motion vectors or related information and other syntactic elements, and uses the predictive information to generate predictive blocks for the current video block being decoded. For example, the mode-applying unit 360 uses some of the received syntactic elements to determine the predictive mode (e.g., intra or inter-predictive) used to code the video blocks in the video slice, the inter-predictive slice type (e.g., B-slice, P-slice, or GPB-slice), one or more construction pieces of the slice's reference picture list, the motion vector for each inter-encoded video block in the slice, the inter-predictive status for each intercoded video block in the slice, and other information for decoding the video blocks in the current video slice. The same or similar may apply to embodiments that use tile groups (e.g., video tile groups) and / or tiles (e.g., video tiles) in addition to or as a substitute for slices (e.g., video slices), for example, video may be coded using I, P, or B tile groups and / or tiles.

[0129] An embodiment of the video decoder 30 shown in Figure 3 may be configured to partition and / or decode a picture using slices (also called video slices), the picture may be partitioned or decoded using one or more slices (usually non-overlapping), each slice may contain one or more blocks (e.g., CTUs).

[0130] Embodiments of the video decoder 30 shown in Figure 3 may be configured to partition and / or decode a picture by using tile groups (also called video tile groups) and / or tiles (also called video tiles), the picture may be partitioned or decoded using one or more tile groups (usually non-overlapping), each tile group may, for example, contain one or more blocks (e.g., CTUs) or one or more tiles, each tile may, for example, be a rectangle, may contain one or more blocks (e.g., CTUs), and may be, for example, a complete block or a partial block.

[0131] Other variations of the video decoder 30 can be used to decode the encoded picture data 21. For example, the decoder 30 can generate an output video stream without a loop filtering unit 320. For example, a non-transformation-based decoder 30 can directly dequantize the residual signal for a particular block or frame without an inverse transformation unit 312. In other implementations, the video decoder 30 can combine the inverse quantization unit 310 and the inverse transformation unit 312 into a single unit.

[0132] It should be understood that in encoder 20 and decoder 30, the processing result of the current step may be further processed and output to the next step. For example, after interpolation filtering, motion vector derivation, or loop filtering, further operations such as clipping or shifting may be performed on the processing result of interpolation filtering, motion vector derivation, or loop filtering.

[0133] Note that further operations (including, but not limited to, control point motion vectors in affine mode, and subblock motion vectors in affine, planar, ATMVP mode, and time motion vectors) may be applied to the derived motion vector of the current block. For example, the value of a motion vector is restricted to a predefined range according to its representation bits. If the representative bit of the motion vector is bitDepth, the range is 2^(bitDepth-1) to 2^(bitDepth-1)-1, where "^" means exponentiation. For example, if bitDepth is set to equal 16, the range is -32768 to 32767, and if bitDepth is set to equal 18, the range is -131072 to 131071. For example, the value of a derived motion vector (e.g., the MV of four 4x4 subblocks in one 8x8 block) is constrained such that the maximum difference between the integer parts of the four 4x4 subblock MVs is less than or equal to N pixels, such as 1 pixel or less. Here, we show two methods for restricting the motion vector according to bitDepth.

[0134] Method 1: Remove the overflow MSB (most significant bit) using flow calculation. ux=(mvx+2 bitDepth ) % 2 bitDepth (1) mvx=( ux >= 2 bitDepth-1 ) ? (ux - 2 bitDepth ) : ux (2) uy=(mvy+2 bitDepth ) % 2 bitDepth (3) mvy=( uy >= 2 bitDepth-1 ) ? (uy - 2 bitDepth ) : uy (4) In the above equation, mvx is the horizontal component of the motion vector of an image block or subblock, mvy is the vertical component of the motion vector of an image block or subblock, and ux and uy represent the intermediate values.

[0135] For example, when the value of mvx is -32769, after applying equations (1) and (2), the resulting value is 32767. In a computer system, decimal numbers are stored as two's complements. The two's complement of -32769 is 1,0111,1111,1111,1111 (17 bits), and since the MSB is discarded, the resulting two's complement is 0111,1111,1111,1111 (decimal 32767), which is the same as the output by applying equations (1) and (2). ux=(mvpx + mvdx +2 bitDepth ) % 2 bitDepth (5) mvx=( ux >= 2 bitDepth-1 ) ? (ux - 2 bitDepth ) : ux (6) uy=(mvpy + mvdy +2 bitDepth ) % 2 bitDepth (7) mvy=( uy >= 2 bitDepth-1 ) ? (uy - 2 bitDepth ) : uy (8)

[0136] The operation can be applied between the sum of mvp and mvd as shown in equations (5) to (8).

[0137] Method 2: Removing the overflow MSB by clipping the value vx=Clip3(-2 bitDepth-1 , 2 bitDepth-1 -1, vx) vy=Clip3(-2 bitDepth-1 , 2 bitDepth-1 -1, vy) In the above equation, vx is the horizontal component of the motion vector of an image block or sub-block, vy is the vertical component of the motion vector of an image block or sub-block, x, y, and z respectively correspond to the three input values of the MV clipping process, and the definition of the function Clip3 is as follows.

[0138]

Number

[0139] Figure 4 is a schematic diagram of a video coding device 400 according to one embodiment of the present disclosure. The video coding device 400 is suitable for implementing the embodiments disclosed herein. In one embodiment, the video coding device 400 may be a decoder, such as the video decoder 30 in Figure 1A, or an encoder, such as the video encoder 20 in Figure 1A.

[0140] The video coding device 400 includes an input port 410 (or input port 410) and a receiver unit (Rx) 420 for receiving data, a processor, logic unit, or central processing unit (CPU) 430 for processing data, a transmitter unit (Tx) 440 and an output port 450 (or output port 450) for transmitting data, and memory 460 for storing data. The video coding device 400 may also include optoelectronic (OE) and electrooptic (EO) components coupled to the input port 410, receiver unit 420, transmitter unit 440, and output port 450 for the input and output of optical or electrical signals.

[0141] The processor 430 is implemented by hardware and software. The processor 430 may be implemented as one or more CPU chips, cores (for example, as a multi-core processor), FPGAs, ASICs, and DSPs. The processor 430 communicates with input port 410, receiver unit 420, transmitter unit 440, output port 450, and memory 460. The processor 430 includes a coding module 470. The coding module 470 implements the embodiments disclosed above. For example, the coding module 470 implements, processes, prepares, or provides various coding operations. Thus, by including the coding module 470, the functionality of the video coding device 400 is greatly improved and the video coding device 400 is transformed into different states. Alternatively, the coding module 470 is implemented as instructions stored in memory 460 and executed by the processor 430.

[0142] Memory 460 may include one or more disks, tape drives, and solid-state drives, and may be used as an overflow data storage device to store the program when such a program is selected for execution, and to store instructions and data read during the execution of the program. Memory 460 may be, for example, volatile and / or non-volatile, and may be read-only memory (ROM), random-access memory (RAM), ternary associative memory (TCAM), and / or static random memory access memory (SRAM).

[0143] Figure 5 is a simplified block diagram of a device 500 that can be used as either or both of the source device 12 and destination device 14 in Figure 1, according to an exemplary embodiment.

[0144] The processor 502 in the device 500 may be a central processing unit. Alternatively, the processor 502 may be one or more other types of devices capable of manipulating or processing information that currently exists or will be developed in the future. The disclosed implementation can be carried out using a single processor, e.g., processor 502, as illustrated, but speed and efficiency advantages can be achieved using multiple processors.

[0145] The memory 504 within the device 500 may, in its implementation, be a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device can be used as memory 504. Memory 504 may contain code and data 506 accessed by the processor 502 using the bus 512. Memory 504 may further include an operating system 508 and an application program 510, the application program 510 including at least one program that allows the processor 502 to perform the method described herein. For example, the application program 510 may include applications 1 to N, the applications 1 to N further including video coding applications that perform the method described herein.

[0146] The device 500 may also include one or more output devices, such as a display 518. The display 518 may, in one example, be a touch-sensitive display combining a display with a touch-sensitive element capable of sensing touch input. The display 518 may be coupled to the processor 502 via a bus 512.

[0147] Although shown as a single bus in this specification, the bus 512 of the device 500 can consist of multiple buses. Furthermore, the secondary storage 514 can be directly coupled to other components of the device 500 or accessed via a network, and may include a single integrated unit such as a memory card or multiple units such as multiple memory cards. Thus, the device 500 can be implemented in a wide variety of configurations.

[0148] background Parameter set The parameter sets are fundamentally similar and share the same basic design goals: bitrate efficiency, error resilience, and the provision of a system layer interface. HEVC (H.265) has a hierarchy of parameter sets, including the Video Parameter Set (VPS), Sequence Parameter Set (SPS), and Picture Parameter Set (PPS), similar to their counterparts in AVC and VVC. Each slice references a single active PPS, SPS, and VPS to access the information used to decode the slice. Since the PPS contains information that applies to all slices in a picture, all slices in a picture must reference the same PPS. Slices in different pictures can also reference the same PPS. Similarly, the SPS contains information that applies to all pictures in the same coded video sequence.

[0149] While PPS may differ for individual pictures, it is common for many or all pictures in an encoded video sequence to refer to the same PPS. Reusing parameter sets makes the bitrate more efficient because it eliminates the need to send shared information multiple times. It is also more loss-resistant because the contents of the parameter set can be transmitted over a more reliable external communication link or repeated frequently within the bitstream to prevent loss.

[0150] Sequence Parameter Set (SPS): A syntactic structure containing zero or more syntactic elements that apply to the entire CLVS, determined by the content of syntactic elements in the PPS referenced by syntactic elements in each picture header.

[0151] Scalable video coding, tiers, and video parameter sets (VPS) Scalable video coding provides a mechanism for coding video across multiple layers, each representing a different quality representation of the same video scene. The base layer is the lowest quality representation. By referencing lower layers, one or more enhancement layers (ELs) may be coded to provide improved video quality. Decoding a subset of the layers of a scalable coded video bitstream results in lower quality than if the entire bitstream were decoded, but still acceptable video. This allows for a more gradual degradation compared to non-scalable video bitstreams, where lowering the bitrate typically results in a significant drop in video quality, often quickly becoming unwatchable.

[0152] Scalable video sequences exhibit several types of scalability, including temporal scalability, spatial scalability, and quality scalability. Figure 8 provides an example illustrating both spatial and temporal scalability. Figure 8 shows two layers coded at different resolutions: BL has a lower resolution, and EL has a higher resolution. Spatial scalability is achieved by providing decoders that decode BL, EL, or both.

[0153] In addition to spatial scalability, temporal scalability is achieved within the coding layer. In this example, each coding layer is divided into two temporal sublayers, labeled with temporal IDs 0 and 1, respectively. Temporal scalability is achieved by providing a decoder to decode temporal sublayer 0 (where temporal ID is equal to 0) or both sublayers 0 and 1.

[0154] Pictures in different layers are assigned different layer IDs, i.e., syntactic element nuh_layer_id. A coded layer video sequence (CLVS) is a sequence of pictures with the same nuh_layer_id value, consisting of a special coding layer video sequence that starts with a coding picture (CLVSS, e.g., an intra picture) in decoding order, followed by zero or more pictures that are not CLVSS pictures, including all subsequent pictures up to the CLVSS picture, but not subsequent pictures that are CLVSS pictures.

[0155] A coded video sequence (CVS) to which VPS can be applied consists of one or more coded layer video sequences (CLVS). In the example in Figure 8, assuming that the first picture of BL and EL is a CLVSS picture and all other pictures are not CLVSS pictures, this is a CVS consisting of two CLVSs.

[0156] Pictures at the same location but on different layers (such as A and B in Figure 8) form an access unit (AU). An access unit consists of a set of pictures that belong to different layers and include coded pictures that are associated with the same time and output from the decoded picture buffer.

[0157] A layer may include independent or dependent layers, where a dependent layer predicts that it will need to use interlayer prediction by referencing one or more reference layers, where the layer ID of the reference layer may be smaller than the layer ID of the dependent layer. Independent layers predict without using interlayer prediction.

[0158] The following table shows examples of syntactic elements that are signaled in a VPS.

[0159] [Table 1A]

[0160] [Table 1B]

[0161] [Table 1C]

[0162] For example, vps_video_parameter_set_id represents an ID that can be referenced by SPS, which will be introduced in the next section.

[0163] For example, vps_max_layers_minus1+1 represents the maximum number of layers allowed in CVS.

[0164] For example, vps_max_sublayers_minus1+1 represents the maximum number of time sublayers allowed in a cross-validation (CV) test.

[0165] For example, the vps_all_layers_same_num_sublayers_flag features a CVS in which all coding layers have the same number of time sublayers. It is signaled only if the maximum number of allowed layers and the maximum number of allowed time sublayers are greater than 1.

[0166] For example, vps_all_independent_layers_flag features a CVS where all coding layers are independent, meaning there are no inter-layer reference pictures in each layer. Inter-layer reference pictures are sometimes used when inter-predictions use pictures from other layers rather than the same layer.

[0167] For example, the vps_layer_id[i] for i in the range from 0 to vps_max_layers_minus1, including both endpoints, represents the layer ID for each layer from layer index 0 to vps_max_layers_minus1. Note the difference between layer index and layer ID. ValueYes, it exists. vps_layer_id[0] represents the ID of the 0th layer and may not be equal to 0.

[0168] For simplicity, this specification will skip the introduction of the remaining syntactic elements. For their explanations, please refer to VVC Draft 8.

[0169] Sequence Parameter Set (SPS) SPS is applied to one or more layers of an encoded video sequence and contains parameters that do not change for each picture within the encoded video sequence. In particular, SPS includes information that signals subpictures.

[0170] The following table shows some snapshots of subpicture signaling in SPS for ITU JVET-Q2001-v13, and the download links are as follows: http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 17_Brussels / wg11 / JVET-Q2001-v13.zip. For the remainder of this application, this prior art document will be referred to as VVC Draft 8 for simplicity.

[0171] [Table 2A]

[0172] [Table 2B]

[0173] If sps_video_parameter_set_id is greater than 0, it specifies the value of vps_video_parameter_set_id of the VPS referenced by SPS.

[0174] If sps_video_parameter_set_id is equal to 0, the following applies: -SPS does not refer to VPS, and when decoding each CLVS by referring to SPS, VPS is not referenced. The value of -vps_max_layers_minus1 is inferred to be equal to 0. - The CVS shall contain only one layer (i.e., all VCL NAL units in the CVS shall have the same nuh_layer_id value).

[0175] Several syntactic elements within the SPS signal location information and control flags for each subpicture. The location information for the i-th subpicture includes the following: • subpic_ctu_top_left_x[i] indicates the horizontal component of the top-left coordinate of subpicture i within the picture, or • subpic_ctu_top_left_y[i] indicates the vertical component of the top-left coordinate of subpicture i within the picture, or • subpic_width_minus1[i] indicates the width of subpicture i within the picture, or • subpic_height_minus1[i] indicates the height of subpicture i within the picture.

[0176] Some syntactic elements, such as sps_num_subpics_minus1, indicate the number of subpictures within a picture.

[0177] CTU, slicing, tiling, and picture partitioning into subpictures

[0178] Picture partitioning for CTU A picture is divided into a sequence of coding tree units (CTUs). The term CTB (coding tree block) is sometimes used interchangeably. The concept of CTU is the same as the concept of HEVC. For a picture with three sample sequences, a CTU consists of an N×N block of luminance samples and two corresponding blocks of saturation samples. Figure 6 shows an example of a picture divided into CTUs. The size of the CTUs within a frame should be the same, except for those at the boundaries of the picture (where incomplete CTUs may exist).

[0179] Partitioning pictures on tiles When tiling is enabled, the picture is divided into rectangular groups of CTUs separated by vertical and / or horizontal boundaries. The vertical and horizontal tile boundaries each separate the picture. From top to bottom It also intersects from the boundary of the left picture to the boundary of the right picture. The bitstream contains instructions related to the positions of the aforementioned horizontal and vertical tile boundaries.

[0180] Partitioning a picture onto a slice The concept of slicing provides picture partitioning so that each slice can be decoded independently of other slices of the same picture, and decoding refers to entropy, residuals, and predictive decoding. The difference from tiling is that slices can have more arbitrary shapes (making partitioning possibilities more flexible), and the purpose of slice partitioning is not parallel processing, but rather packet size matching and error resilience in the transmission environment.

[0181] A slice can consist of a complete picture or a portion thereof. In HEVC, a slice consists of multiple consecutive CTUs of pictures in processing order. A slice is identified by its starting CTU address, which is signaled in the slice header, picture parameter set, or other units.

[0182] In VVC Draft 8, a slice contains an integer number of complete tiles, or an integer number of consecutive CTU rows within a picture tile. Therefore, each vertical slice boundary is always also a vertical tile boundary. A horizontal boundary of a slice may contain a horizontal CTU boundary within a tile rather than a tile boundary, which occurs when a tile is split into multiple rectangular slices, each slice containing an integer number of consecutive complete CTU rows within the tile.

[0183] In some examples, there are two slicing modes: raster scan slice mode and rectangular slice mode. In raster scan slice mode, a slice contains a sequence of tiles in the tile raster scan of a picture. In rectangular slice mode, a slice contains a number of tiles that collectively form a rectangular area of ​​the picture, or a slice contains a number of consecutive CTU rows of a single tile that collectively form a rectangular area of ​​the picture. Tiles in a rectangular slice are scanned in tile raster scan order within the rectangular area corresponding to that slice.

[0184] All slices of a picture collectively form the entire picture; that is, all CTUs of a picture must be contained within one of the picture's slices. The same rule applies to tiles and subpictures.

[0185] Partitioning a picture into sub-pictures A rectangular partition of a picture's subpicture. A subpicture can be the entire picture or a part of a picture. A subpicture is a partition of a picture so that each subpicture can be decoded independently of other subpictures in the entire video sequence. In VVC Draft 8, this is true if it is indicated in the bitstream; that is, if the subpic_treated_as_pic_flag[i] directive is applied to subpicture i, then subpicture i can be decoded independently of other subpictures in the entire video sequence.

[0186] The difference between subpictures and tiles or slices is that subpictures create independently decodeable video sequences within a video sequence. With tiles and slices, independent decoding is only guaranteed within a single picture in the video sequence.

[0187] In VVC Draft 8, a subpicture contains one or more slices that collectively cover the rectangular area of ​​the picture. Therefore, each subpicture boundary is always a slice boundary, and each vertical subpicture boundary is always a vertical tile boundary.

[0188] Figure 7 provides examples of tiles, slices, and subpictures.

[0189] In the example shown in Figure 7, the picture is partitioned into 216 CTUs, 4 tiles, 4 slices, and 3 subpictures. The value of sps_num_subpics_minus1 is 2, and the position-related syntactic elements have the following values: In the case of sub-picture 0 • subpic_ctu_top_left_x[0] is not signaled, but is inferred to be 0. • subpic_ctu_top_left_y[0] is not signaled, but is inferred to be 0. • subpic_width_minus1[0], the value is 8. • The value of subpic_height_minus1[0] is 11. In the case of sub-picture 1 The subpic_ctu_top_left_x[1] has a value of 9. The subpic_ctu_top_left_y[1] has a value of 0. • subpic_width_minus1[1], the value is 8. • subpic_height_minus1[1], the value is 5. In the case of sub-picture 2 The subpic_ctu_top_left_x[2] has a value of 9. The subpic_ctu_top_left_y[2] has a value of 6. • subpic_width_minus1[2] is not signaled, but is inferred to be 8. • subpic_height_minus1[2] is not signaled, but is inferred to be 5.

[0190] The encoded bitstream consists of Network Abstraction Layer (NAL) units. A NAL unit is a syntactic structure containing an instruction for the type of data to follow and the bytes containing that data, in the form of a Raw Byte Sequence Payload (RBSP) with emulation-prevention bytes scattered as needed.

[0191] An RBSP is a syntactic structure containing an integer number of bytes encapsulated in a NAL unit, which is either empty or in the form of a string of data bits containing syntactic elements, followed by an RBSP stop bit and zero or more trailing bits equal to 0. The emulation prevention byte is a byte equal to 0x03.

[0192] Parameter sets such as SPS or VPS are encapsulated in NAL units. Each NAL unit has a NAL unit header, except for the syntactic element information specified in the SPS and VPS. The NAL unit header provides information such as the layer ID (i.e., syntactic nuh_layer_id) and the NAL unit type (i.e., syntactic nal_unit_type). For example, the NAL unit type for a NAL unit containing VPS is VPS_NUT, and the NAL unit type for a NAL unit containing SPS is SPS_NUT.

[0193] Generally, NAL units within a bitstream can be divided into two classes: Video Coding Layer (VCL) NAL units and non-VCL NAL units. NAL units containing parameter sets that do not provide the essential coded data for a picture (such as VPS or SPS) belong to the non-VCL NAL unit type. In contrast, NAL units containing the essential coded data for a picture (such as picture coded video data) are called VCL NAL units.

[0194] Table 5 of VVC Draft 8 defines the NAL unit types and their classifications for each NAL unit.

[0195] For further details, please refer to VVC Draft 8.

[0196] Decoding process for building a reference picture list This process is invoked at the start of the picture slice-by-slice decoding process.

[0197] Reference pictures are handled through a reference index, which is an index to the RPL. When decoding an I slice, the RPL is not used when decoding the slice data. When decoding a P slice, only RPL0 (i.e., RefPicList[0]) is used to decode the slice data. When decoding a B slice, both RPL0 and RPL1 (i.e., RefPicList[1]) are used to decode the slice data.

[0198] At the start of decoding each slice of the picture, the RPL RefPicList[0] and RefPicList[1] are derived. The RPL is used to mark the reference picture specified in Section 8.3.3, or to decode the slice data. Note 1 - In the case of an I-slice of a picture, RefPicList[0] and RefPicList[1] can be derived for the purpose of bitstream compatibility checking, but their derivation is not necessary for decoding the current picture or the picture that follows the current picture in the decoding order. In the case of a P-slice of a picture, RefPicList[1] can be derived for the purpose of bitstream compatibility checking, but its derivation is not necessary for decoding the current picture or the picture that follows the current picture in the decoding order.

[0199] If sps_idr_rpl_present_flag is equal to 0, pps_rpl_info_in_ph_flag is equal to 0, and nal_unit_type is equal to IDR_W_RADL or IDR_N_LP, then both RPL RefPicList[0] and RefPicList[1] are derived to be empty, i.e., contain 0 entries, and the following applies to each i equal to 0 or 1: The value of RplsIdx[i] is inferred to be equal to sps_num_ref_pic_lists[i]. The value of num_ref_entries[i][RplsIdx[i]] is inferred to be equal to 0. The value of NumRefIdxActive[i] is inferred to be equal to 0.

[0200] Otherwise, the RPL RefPicList[0] and RefPicList[1], the reference picture scaling ratios RefPicScale[i][j][0] and RefPicScale[i][j][1], and the reference picture scaling flags RprConstraintsActiveFlag[0][j] and RprConstraintsActiveFlag[1][j] are derived as follows: for( i = 0; i < 2; i++ ) { for( j = 0, k = 0, pocBase = PicOrderCntVal; j < num_ref_entries[i][RplsIdx[i]]; j++) { if( !inter_layer_ref_pic_flag[i][RplsIdx[i]][j] ) { if( st_ref_pic_flag[i][RplsIdx[i]][j] ) { RefPicPocList[i][j] = pocBase+DeltaPocValSt[i][RplsIdx[i]][j] if( there is a reference picture picA in the DPB with the same nuh_layer_id as the current picture and PicOrderCntVal equal to RefPicPocList[i][j] ) RefPicList[i][j] = picA else RefPicList[i][j] = "no reference picture" (200) pocBase = RefPicPocList[i][j] }else{ if( !delta_poc_msb_cycle_present_flag[i][k] ) { if(there is a reference picA in the DPB with the same nuh_layer_id as the current picture and PicOrderCntVal&(MaxPicOrderCntLsb-1) equal to PocLsbLt[i][k] ) RefPicList[i][j] = picA else RefPicList[i][j] = "no reference picture" RefPicLtPocList[i][j] = PocLsbLt[i][k] } else { if( there is a reference picA in the DPB with the same nuh_layer_id as the current picture and PicOrderCntVal equal to FullPocLt[i][k] ) RefPicList[i][j] = picA else RefPicList[i][j] = "no reference picture" RefPicLtPocList[i][j] = FullPocLt[i][k] } k++ } } else { layerIdx = DirectRefLayerIdx[GeneralLayerIdx[nuh_layer_id]][ilrp_idx[i][RplsIdx[i]][j]] refPicLayerId = vps_layer_id[layerIdx] if( there is a reference picture picA in the DPB with nuh_layer_id equal to refPicLayerId and the same PicOrderCntVal as the current picture ) RefPicList[i][j] = picA else RefPicList[i][j] = "no reference picture" } fRefWidth is set equal to CurrPicScalWinWidthL of the reference picture RefPicList[i][j] fRefHeight is set equal to CurrPicScalWinHeightL of the reference picture RefPicList[i][j] refPicWidth, refPicHeight, refScalingWinLeftOffset, refScalingWinRightOffset, refScalingWinTopOffset, and refScalingWinBottomOffset, are set equal to the values of pps_pic_width_in_luma_samples, pps_pic_height_in_luma_samples, pps_scaling_win_left_offset, pps_scaling_win_right_offset, pps_scaling_win_top_offset, and pps_scaling_win_bottom_offset, respectively, of the reference picture RefPicList[i][j] fRefNumSubpics is set equal to sps_num_subpics_minus1 of the reference picture RefPicList[i][j] RefPicScale[i][j][0] = ((fRefWidth<<14)+(CurrPicScalWinWidthL>>1)) / CurrPicScalWinWidthL RefPicScale[i][j][1] = ((fRefHeight<<14)+(CurrPicScalWinHeightL>>1)) / CurrPicScalWinHeightL RprConstraintsActiveFlag[i][j] = ( pps_pic_width_in_luma_samples != refPicWidth || pps_pic_height_in_luma_samples != refPicHeight || pps_scaling_win_left_offset != refScalingWinLeftOffset || pps_scaling_win_right_offset != refScalingWinRightOffset || pps_scaling_win_top_offset != refScalingWinTopOffset || pps_scaling_win_bottom_offset != refScalingWinBottomOffset || sps_num_subpics_minus1 != fRefNumSubpics ) } }

[0201] For each i equal to 0 or 1, the first NumRefIdxActive[i] entry in RefPicList[i] is called the active entry in RefPicList[i], and the other entries in RefPicList[i] are called the inactive entries in RefPicList[i].

[0202] Description of the problem and solution Problem 1 VPS is an arbitrary set of parameters. For example, in the case of a single-layer bitstream, there may be no VPS within the bitstream. The semantics of some syntactic elements within VPS have been found to be incomplete. If VPS is not present in the bitstream, there is no definition for them.

[0203] For example, in VVC Draft 8, the semantics of the syntactic element vps_max_layers_minus1 are specified as follows: vps_max_layers_minus1+1 specifies the maximum number of layers allowed in each CVS that references the VPS.

[0204] Embodiment 1 According to the first embodiment, to account for the case where the VPS does not exist in the bitstream, the semantics of vps_max_layers_minus1 are modified as follows: vps_max_layers_minus1+1 specifies the maximum number of layers allowed in each CV that references the VPS. If it does not exist, the value of vps_max_layers_minus1 is inferred to be equal to 0.

[0205] In other words, if a VPS does not exist, CVS only has one layer.

[0206] Problem 2 The same problem exists with other syntactic elements in VPS, from range 0 to vps_max_layers_minus, namely vps_layer_id[i], which does not specify an inferred value if the VPS does not exist. The semantics of vps_layer_id[i] are specified in VVC Draft 8 as follows: vps_layer_id[i] specifies the nuh_layer_id value of the i-th layer. For any two non-negative integer values ​​of m and n, if m is less than n, then the value of vps_layer_id[m] is less than vps_layer_id[n]. nuh_layer_id represents the layer ID. vps_layer_id[i] specifies the layer ID of the i-th layer. This definition also constrains that if there are more layers, the layer ID of a higher layer (with layer index n) must be greater than the layer ID of a lower layer (with layer index m), meaning that the layer IDs specified in the vps_layer_id array are in ascending order.

[0207] Note that the value of vps_layer_id[i] is used to derive several intermediate variables in VVC Draft 8. For example, the derivation of the variable OutputLayerIdInOls[0][0] depends on the value of vps_layer_id[i] for i in the range of 0 to vps_max_layers_minus1, including both ends, and OutputLayerIdInOls[0][0] may be used further to determine the decoding process, as specified in clause 8.1.2 of VVC Draft 8. If the VPS is not present in the bitstream, the decoding process may be undefined because the value of OutputLayerIdInOls[0][0] is undefined due to the lack of an inferred value of vps_layer_id[i] for i in the range of 0 to vps_max_layers_minus1, including both ends.

[0208] Embodiment 2 According to the second embodiment, to account for the case where the VPS does not exist in the bitstream, the semantics of vps_layer_id[i] are modified as follows: vps_layer_id[i] specifies the nuh_layer_id value of the i-th layer. For any two non-negative integer values ​​of m and n, if m is less than n, then the value of vps_layer_id[m] is less than vps_layer_id[n]. If it does not exist, vps_layer_id[0] is inferred to be equal to the value of the nuh_layer_id in the Sequence Parameter Set (SPS).

[0209] In other words, if no VPS exists, there is only one video coding layer. The syntax element vps_layer_id array can have only one entry (i.e., the 0th layer), and its value vps_layer_id[0] is inferred to be equal to the layer ID of the SPS.

[0210] Embodiment 3 According to the third embodiment, to account for the case where the VPS does not exist in the bitstream, the semantics of vps_layer_id[i] are modified as follows: vps_layer_id[i] specifies the nuh_layer_id value of the i-th layer. For any two non-negative integer values ​​of m and n, if m is less than n, then the value of vps_layer_id[m] is less than vps_layer_id[n]. If it does not exist, vps_layer_id[0] is inferred to be equal to the value of the nuh_layer_id of the first non-VCL NAL unit.

[0211] In other words, if no VPS exists, there is only one video coding layer. The syntactic element vps_layer_id array can have only one entry (i.e., the 0th layer), and its value vps_layer_id[0] is inferred to be equal to the layer ID value of the non-VCL NAL unit.

[0212] Embodiment 4 According to the fourth embodiment, to account for the case where the VPS does not exist in the bitstream, the semantics of vps_layer_id[i] are modified as follows: vps_layer_id[i] specifies the nuh_layer_id value of the i-th layer. For any two non-negative integer values ​​of m and n, if m is less than n, then the value of vps_layer_id[m] is less than vps_layer_id[n]. If it does not exist, vps_layer_id[0] is inferred to be equal to the value of the nuh_layer_id of the first VCL NAL unit.

[0213] In other words, if no VPS exists, there is only one video coding layer. The syntax element vps_layer_id array can have only one entry (i.e., the 0th layer), and its value vps_layer_id[0] is inferred to be equal to the layer ID value of the VCL NAL unit.

[0214] Embodiment 5 According to the fifth embodiment, to account for the case where the VPS does not exist in the bitstream, the semantics of vps_layer_id[i] are modified as follows: vps_layer_id[i] specifies the nuh_layer_id value of the i-th layer. For any two non-negative integer values ​​of m and n, if m is less than n, then the value of vps_layer_id[m] is less than vps_layer_id[n]. If it does not exist, vps_layer_id[0] is inferred to be equal to the nuh_layer_id value of any NAL unit.

[0215] In other words, if no VPS exists, there is only one video coding layer, meaning all NAL units have the same nuh_layer_id value in their NAL unit headers. The syntactic element vps_layer_id array can have only one entry (i.e., the 0th layer), and its value vps_layer_id[0] is inferred to be equal to the layer ID value of any NAL unit.

[0216] Methods for decoding a video bitstream and methods for encoding a video bitstream are provided herein in accordance with the above description. Correspondingly, apparatus for decoding a (encoded) video bitstream and apparatus for decoding a video bitstream are also provided herein.

[0217] FIG. 9 shows a method of decoding a video bitstream implemented by a decoding device, the bitstream including data representing a current video coding layer, the method comprising: a step 910 of determining whether the current video coding layer is the only layer within the bitstream; and a step 920 of using an identifier (e.g., nuh_layer_id) of a NAL unit of the current video coding layer as an identifier (e.g., vps_layer_id) of the current video coding layer if it is determined that the current video coding layer is the only layer within the bitstream (e.g., determined only if determined) (e.g., sps_video_parameter_set_id is equal to 0).

[0218] Note that the bitstream can be obtained by a wireless network or a wired network. The bitstream can be transmitted from a website, a server, or other remote sources using a coaxial cable, an optical fiber cable, a twisted pair, a digital subscriber line (DSL), or wireless technologies such as infrared, radio, microwave, WIFI, Bluetooth, LTE, or 5G.

[0219] The bitstream can be a bit sequence in the form of a network abstraction layer (NAL) unit stream or a byte stream, forming a representation of a sequence of access units (AUs) that form one or more coded video sequences (CVSs).

[0220] In a specific example, the bitstream format specifies the relationship between a network abstraction layer (NAL) unit stream and a byte stream, both of which are called bitstreams.

[0221] ]A bitstream can take one of two forms: a NAL unit stream or a byte stream. The NAL unit stream is conceptually the more "basic" type. A NAL unit stream contains a sequence of syntactic structures called NAL units, arranged in decode order. The decode order (and content) of NAL units within a NAL unit stream is constrained.

[0222] The byte stream format can be constructed from the NAL unit stream format by arranging the NAL units in decoded order and prefixing each NAL unit with a start code prefix and zero or more zero-value bytes to form a byte stream. The NAL unit stream format can be extracted from the byte stream format by searching for the position of a unique start code prefix pattern within this byte stream.

[0223] Figure 10 shows a method for encoding a video bitstream implemented by an encoding device, where the bitstream contains data representing the current video coding layer, and the method includes a step 1010 of determining whether the current video coding layer is the only layer in the bitstream, and if it is determined that the current video coding layer is the only layer in the bitstream, a step 1020 of using the identifier of the NAL unit of the current video coding layer as the identifier of the current video coding layer.

[0224] The methods described above can be incorporated into a video decoding device or a video encoding device (which generates a bitstream), respectively, as described below.

[0225] As shown in Figure 11, a video decoding apparatus 1100 provided herein according to one embodiment includes a decision unit 1110 and a prediction unit 1120. As shown in Figure 12, a video encoding apparatus 1200 provided herein according to one embodiment includes a decision unit 1210 and a prediction unit 1220.

[0226] The decision unit 1110 included in the video decoding device 1100 shown in Figure 11 is configured to determine whether the current video coding layer is the only layer in the bitstream. The prediction unit 1120 included in the video decoding device 1100 shown in Figure 11 determines that the current video coding layer is the only layer in the bitstream (for example, determined only if sps_video_parameter_set_id is equal to 0), and uses the identifier of the NAL unit of the current video coding layer (for example, nuh_layer_id) as the identifier of the current video coding layer (for example, vps_layer_id). ni kamo It has been done.

[0227] The decision unit 1210 included in the video encoding device 1200 shown in Figure 12 is configured to determine whether the current video coding layer is the only layer in the bitstream. The prediction unit 1220 included in the video encoding device 1200 shown in Figure 12 is configured to use the identifier of the NAL unit of the current video coding layer as the identifier of the current video coding layer if it is determined that the current video coding layer is the only layer in the bitstream.

[0228] The video decoding device 1100 shown in Figure 11 may be the decoder 30 shown in Figures 1A, 1B, and 3, and the video decoder 3206 shown in Figure 14, or may be configured therein. Device1100 may consist of a video coding device 400 shown in Figure 4, a device 500 shown in Figure 5, and a terminal device 3106 shown in Figure 13. Encoding shown in Figure 12 Device 1200 may be the encoder 20 shown in Figures 1A, 1B, and 3, or may be configured therein. Furthermore, encoding Device 1200 may consist of a video coding device 400 shown in Figure 4, a device 500 shown in Figure 5, and a capture device 3102 shown in Figure 13.

[0229] The following describes application examples of the encoding and decoding methods shown in the above embodiment, and systems using them.

[0230] Figure 13 is a block diagram showing a content supply system 3100 for realizing a content distribution service. This content supply system 3100 includes a capture device 3102, a terminal device 3106, and optionally a display 3126. The capture device 3102 communicates with the terminal device 3106 via a communication link 3104. The communication link may include the communication channel 13 described above. The communication link 3104 includes, but is not limited to, Wi-Fi, Ethernet, cable, wireless (3G / 4G / 5G), USB, or any combination thereof.

[0231] The capture device 3102 may generate data and encode it using an encoding method as shown in the embodiments described above. Alternatively, the capture device 3102 may deliver the data to a streaming server (not shown), which encodes the data and transmits the encoded data to the terminal device 3106. The capture device 3102 includes, but is not limited to, a camera, a smartphone or tablet, a computer or laptop, a video conferencing system, a PDA, an in-vehicle device, or any combination thereof. For example, the capture device 3102 may include the source device 12 described above. If the data includes video, the video encoder 20 included in the capture device 3102 may actually perform the video encoding process. If the data includes audio (i.e., voice), the audio encoder included in the capture device 3102 may actually perform the audio encoding process. In some practical scenarios, the capture device 3102 delivers the encoded video data and audio data by multiplexing them together. In other practical scenarios, for example in a video conferencing system, the encoded audio data and encoded video data are not multiplexed. The capture device 3102 distributes the encoded audio data and encoded video data separately to the terminal device 3106.

[0232] In the content supply system 3100, terminal devices 3106The device receives and plays back the encoded data. The terminal device 3106 may be a device with data receiving and retrieval capabilities, such as a smartphone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a television 3114, a set-top box (STB) 3116, a video conferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, an in-vehicle device 3124, or any combination thereof, or a device capable of decoding the above-mentioned encoded data. For example, the terminal device 3106 may include the destination device 14 described above. If the encoded data includes video, the video decoder 30 included in the terminal device is preferred to perform video decoding. If the encoded data includes audio, the audio decoder included in the terminal device is preferred to perform audio decoding.

[0233] For example, in the case of terminal devices equipped with a display, such as a smartphone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a television 3114, a personal digital assistant (PDA) 3122, or an in-vehicle device 3124, the terminal device can supply the decoded data to its display. In the case of terminal devices without a display, such as an STB 3116, a video conferencing system 3118, or a video surveillance system 3120, the decoded data is brought into contact with an external display 3126 to be received and displayed.

[0234] When each device in this system performs encoding or decoding, a picture encoding device or a picture decoding device can be used, as shown in the above embodiment.

[0235] Figure 14 shows the structure of an example terminal device 3106. After the terminal device 3106 receives a stream from the capture device 3102, the protocol progression unit 3202 analyzes the transmission protocol of the stream. The protocol includes, but is not limited to, Real-Time Streaming Protocol (RTSP), Hypertext Transfer Protocol (HTTP), HTTP Live Streaming Protocol (HLS), MPEG-DASH, Real-Time Transport Protocol (RTP), Real-Time Messaging Protocol (RTMP), or any combination thereof.

[0236] After the protocol processing unit 3202 processes the stream, a stream file is generated. The file is output to the demultiplexing unit 3204. The demultiplexing unit 3204 can separate the multiplexed data into encoded audio data and encoded video data. As described above, in some practical scenarios, such as in a video conferencing system, the encoded audio data and encoded video data are not multiplexed. In this situation, the encoded data is transmitted to the video decoder 3206 and audio decoder 3208 without going through the demultiplexing unit 3204.

[0237] Through inverse multiplexing processing, a video elementary stream (ES), an audio ES, and optionally subtitles are generated. A video decoder 3206 including the video decoder 30 described in the above embodiment decodes the video ES by a decoding method in order to generate a video frame, and supplies this data to a synchronization unit 3212. An audio decoder 3208 decodes the audio ES in order to generate an audio frame, and supplies this data to the synchronization unit 3212. Alternatively, the video frame may be stored in a buffer (not shown in FIG. 14) before being supplied to the synchronization unit 3212. Similarly, the audio frame may be stored in a buffer (not shown in FIG. 14) before being supplied to the synchronization unit 3212.

[0238] The synchronization unit 3212 synchronizes the video frame and the audio frame, and supplies the video / audio to a video / audio display 3216 For example, the synchronization unit 3212 synchronizes the presentation of video and audio information. The information may be encoded in syntax using time stamps related to the presentation of the encoded audio and visual data and time stamps related to the delivery of the data stream itself.

[0239] If the stream includes subtitles, a subtitle decoder 3210 decodes the subtitles, synchronizes them with the video frame and the audio frame, and supplies the video / audio / subtitle to a video / audio / subtitle display 3216.

[0240] Note that the present invention is not limited to the above system, and the picture encoding device or picture decoding device in the above embodiment can be incorporated into other systems, for example, an automotive system.

[0241] Mathematical operator The arithmetic operators used in this application are similar to those used in the C programming language. However, the results of integer division and arithmetic shift operations are more precisely defined, and additional operations such as exponentiation and real-valued division are defined. The numbering and counting rules generally start from 0, for example, "1st" corresponds to the 0th, "2nd" corresponds to the 1st, and so on.

[0242] Arithmetic operators The following arithmetic operators are defined as follows:

[0243] [Table 3]

[0244] Logical operators The following logical operators are defined as follows: x && y: Boolean logic "and" for x and y x|| y Boolean logic "or" for x and y ! Boolean logic "not" x ? y : If zx is TRUE or not equal to 0, it evaluates to the value of y; otherwise, it evaluates to the value of z.

[0245] Relational operators The following relational operators are defined as follows: > larger >= Above < Less than <= Below = = equals != is not equal to

[0246] When a relational operator is applied to a syntactic element or variable assigned the value "na" (not applicable), the value "na" is treated as a separate value for the syntactic element or variable. The value "na" is considered not to be equal to any other value.

[0247] Bitwise operators The following bitwise operators are defined as follows: & represents a bitwise "and". When performing operations on integer arguments, the two's complement representation of the integer value is used. When performing operations on a binary argument with fewer bits than other arguments, the shorter argument is extended by adding a number of valid bits equal to zero. | Bitwise "or". When performing operations on integer arguments, the two's complement representation of the integer value is used. When performing operations on a binary argument with fewer bits than other arguments, the shorter argument is extended by adding a number of valid bits equal to zero. ^ Bitwise "exclusive OR". When performing operations on integer arguments, the two's complement representation of the integer value is calculated. When performing operations on binary arguments with fewer bits than other arguments, the shorter argument is extended by adding valid bits equal to 0. x >> yy is an arithmetic right shift of the two's complement integer representation of x in binary. This function is defined only for non-negative integer values ​​of y. The bit shifted to the most significant bit (MSB) as a result of the right shift has a value equal to the MSB of x before the shift. The arithmetic left shift of the two's complement integer representation of x in binary, where x << yy. This function is defined only for non-negative integer values ​​of y. The bit shifted to the least significant bit (LSB) as a result of the left shift has a value equal to 0.

[0248] Assignment operator Next Assignment The operators are defined as follows: = Assignment operator The increment operator, x++, is equivalent to x = x + 1, and when used in array indices, it evaluates the value of the variable before the increment operation. The decrement operation, i.e., x--, is equivalent to x = x-1, and when used in an array index, it evaluates the value of the variable before the decrement operation. The += operator increments by the specified amount; that is, x+=3 is equal to x=x+3, and x+=(-3) is equal to x=x+(-3). -= decrements by the specified amount; that is, x-=3 is equal to x=x-3, and x-=(-3) is equal to x=x-(-3).

[0249] Range notation The following notation is used to specify a range of values. x = y..zx takes integer values ​​from y to z, including the values ​​at both ends, where x, y, and z are integers, and z is greater than y.

[0250] Mathematical functions The following mathematical function is defined.

[0251]

number

[0252] Asin(x) is the inverse sine function of the trigonometric function that operates on an argument x in the range of -1.0 to 1.0, including both endpoints, and has an output value in radians in the range of -π÷2 to π÷2, including both endpoints. Atan(x) is the inverse tangent function of the trigonometric function that operates on the argument x, and its output value is in radians and ranges from -π÷2 to π÷2, including both endpoints.

[0253]

number

[0254] Ceil(x) The smallest integer greater than or equal to x. Clip1 Y (x) = Clip3(0,(1 < <BitDepth Y )-1,x) Clip1 C (x) = Clip3(0,(1 < <BitDepth C )-1,x)

[0255]

number

[0256] Cos(x) is the cosine function of the trigonometric function that performs operations on the argument x in radians. Floor(x): The largest integer less than or equal to x.

[0257]

number

[0258] Ln(x) is the natural logarithm of x (a logarithm with base e, where e is the base constant of the natural logarithm, 2.718 281 828...). Log2(x) is the logarithm of x with base 2. Log10(x) is the logarithm of x with base 10.

[0259]

number

[0260] Round(x)=Sign(x)*Floor(Abs(x)+0.5)

[0261]

number

[0262] Sin(x) is a trigonometric sine function that performs calculations on the argument x in radians.

[0263]

number

[0264] Swap(x, y)=(y, x) Tan(x) is a trigonometric tangent function that calculates the value of argument x in radians.

[0265] Prioritization of actions If the precedence of an expression is not explicitly indicated using parentheses, the following rules apply: - Higher-priority operations are evaluated before lower-priority operations. - Operations with the same priority are evaluated from left to right.

[0266] The following table shows the order of operations from highest to lowest, with higher positions in the table indicating higher priority.

[0267] For operators also used in the C programming language, the precedence used herein is the same as that used in the C programming language.

[0268] [Table 4]

[0269] Text description of logical operations In text, logical operation statements are mathematically described in the following format: if (condition 0) statement 0 else if (condition 1) statement 1 ... else / * informative comment on remaining condition * / statement n This can be described as follows: ... as follows / ... the following applies: - If condition 0, statement 0 - Otherwise, if condition 1, statement 1 - ... - Otherwise (informative remark on remaining condition), statement n Each "If ... Otherwise, if ... Otherwise, ..." statement in the text begins with "... as follows" or "... the following applies," immediately followed by "If ...." The final condition in "If ... Otherwise, if ... Otherwise, ..." is always "Otherwise, ...." Interleaved "If ... Otherwise, if ... Otherwise, ..." statements can be identified by matching "... as follows" or "... the following applies" with the final "Otherwise, ...."

[0270] In text, logical operation statements are mathematically described in the following format: if( condition 0a && condition 0b ) statement 0 else if( condition 1a || condition 1b ) statement 1 ... else statement n This can be described as follows: ... as follows / ... the following applies: - If all of the following conditions are true, statement 0: - condition 0a - condition 0b - Otherwise, if one or more of the following conditions are true, statement 1: - condition 1a - condition 1b - ... - Otherwise, statement n

[0271] In text, logical operation statements are mathematically described in the following format: if (condition 0) statement 0 if (condition 1) statement 1 This can be described as follows: when condition 0, statement 0 When condition 1, statement 1.

[0272] While embodiments of the present invention have been described primarily in relation to video coding, it should be noted that embodiments of the coding system 10, encoder 20, and decoder 30 (and corresponding systems 10), as well as other embodiments described herein, may also be configured for still image processing or coding, i.e., processing or coding of individual pictures, independently of preceding or consecutive pictures, as in video coding. Generally, only the interprediction units 244 (encoder) and 344 (decoder) may not be available when picture processing coding is limited to a single picture 17. All other functions (also called tools or techniques) of the video encoder 20 and video decoder 30 may be equally used for still image processing, e.g., residual calculation 204 / 304, transformation 206, quantization 208, inverse quantization 210 / 310, (inverse) transformation 212 / 312, partitioning 262 / 362, intra-prediction 254 / 354, and / or loop filtering 220, 320, as well as entropy coding 270 and entropy decoding 304.

[0273] Embodiments, for example, embodiments of encoder 20 and decoder 30, and functions described herein with respect to encoder 20 and decoder 30, for example, can be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes over a communication medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium including any medium that facilitates the transfer of computer programs from one location to another, for example, according to a communication protocol. Thus, a computer-readable medium can generally correspond to (1) a non-transient, tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium accessible by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described herein. A computer program product may include a computer-readable medium.

[0274] As an example and not an limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other media that can be used to store desired program code in the form of instructions or data structures and can be accessed by a computer. Furthermore, all connections are appropriately called computer-readable media. For example, if instructions are transmitted from a website, server or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, or microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, or microwave are included in the definition of media. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals or other temporary media, but instead refer to non-temporary tangible storage media. As used herein, the terms "disk" and "disc" include Compact Disc (CD), LaserDisc, Optical Disc, Digital Versatile Disc (DVD), Floppy Disk, and Blu-ray® Disc. A disk typically reproduces data magnetically, while a disc reproduces data optically using a laser. Any combination of these should also be included within the scope of computer-readable media.

[0275] Instructions are for one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), and field-programmable microprocessors. gateThis can be performed by one or more processors, such as an array (FPGA) or other equivalent integrated or discrete logic circuits. Thus, the term “processor,” as used herein, may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. Furthermore, in some embodiments, the functions described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Moreover, these techniques can be fully implemented in one or more circuits or logic elements.

[0276] The techniques of this disclosure can be implemented in a wide variety of devices or apparatus, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chipsets). While various components, modules, or units are described in this disclosure to highlight the functional aspects of devices configured to perform the disclosed techniques, implementation by different hardware units is not necessarily required. Rather, as described above, various units may be combined into a codec hardware unit, or they may be provided by a set of interoperable hardware units including one or more processors, along with appropriate software and / or firmware.

[0277] In particular, ten further embodiments are also provided herein, listed as embodiments 1 to 10.

[0278] 1. A mode of a method for decoding a bitstream of a video or picture implemented by a decoding device, wherein the bitstream includes data representing the current video coding layer, and the method includes the steps of: using an identifier of a sequence parameter set in the bitstream as the identifier of the current video coding layer if there is no syntactic element in the bitstream that specifies an identifier of the current video coding layer (e.g., vps_layer_id[0]) (e.g., VPS does not exist); and using the identifier of the current video coding layer to predict the current slice of the current video coding layer.

[0279] 2. An embodiment of the method of Embodiment 1, wherein the step of using an identifier of a sequence parameter set in a bitstream as an identifier of the current video coding layer is to construct one or more reference picture lists of the current slice of the current video coding layer, and the step of predicting the current slice of the current video coding layer using the identifier of the current video coding layer is to include the step of predicting the current slice using one or more reference picture lists.

[0280] 3. A method for decoding a bitstream of a video or picture implemented by a decoding device, wherein the bitstream includes data representing the current video coding layer, and the method includes the steps of: using the identifier of the NAL unit of the current video coding layer as the identifier of the current video coding layer if there is no syntactic element in the bitstream that specifies the identifier of the current video coding layer (e.g., vps_layer_id[0]) (e.g., no VPS); and using the identifier of the current video coding layer to predict the current slice of the current video coding layer.

[0281] 4. An embodiment of the method of Embodiment 3, wherein the step of using an identifier of the NAL unit of the current video coding layer as an identifier of the current video coding layer is to construct one or more reference picture lists of the current slice of the current video coding layer, and the step of predicting the current slice of the current video coding layer using the identifier of the current video coding layer is to include the step of predicting the current slice using one or more reference picture lists.

[0282] 5. An embodiment of the method of embodiment 3 or 4, wherein the NAL unit of the current video coding layer is a first non-VCL NAL unit or a first VCL NAL unit of the current video coding layer present in the bitstream.

[0283] 6. An embodiment of the method of embodiment 3 or 4, wherein the NAL unit of the current video coding layer is any one of the NAL units of the current video coding layer present in the bitstream.

[0284] 7. An embodiment of a decoder (30) including a processing circuit for performing a method according to any one of embodiments 1 to 6.

[0285] 8. An embodiment of a computer program product that, when executed on a computer or processor, includes program code for performing a method according to any one of the embodiments 1 to 6 described above.

[0286] 9. An embodiment of a decoder comprising one or more processors and a non-temporary computer-readable storage medium coupled to the processors and storing a program for execution by the processors, wherein the program, when executed by the processors, configures the decoder to perform a method according to any one of the embodiments 1 to 6 described above.

[0287] 10. A form of non-temporary computer-readable medium that, when executed by a computer device, carries program code that causes the computer device to perform one of the methods described in aspects 1 to 6 above. [Explanation of Symbols]

[0288] 10 Coding Systems 10 Video Coding Systems 10 Short Coding Systems 12 Source Devices 13 Communication Channels 14 Destination device 16 Picture Sources 17. Picture or picture data 17. Raw picture or raw picture data 18. Preprocessor (or preprocessing unit) 18 Picture Preprocessor 19 Pre-processed picture 19 Preprocessed picture data 20 Video Encoders 20 Short Encoders 21 Encoded picture data 21 Encoded bitstream 22 Communication interface or communication unit 28 Communication interface or communication unit 30 Decoders 30 video decoders 30 Short Decoders 31 Decoded picture data 31 Decoded picture 32 Post-Processors 32 Post-Processing Units 33 Post-processed picture data 33 Post-processed pictures 34 Display Devices 201 Input 201 Input Interface 203 Picture Block 204 Residual Calculation Unit 205 Residual Block 205 Residual 206 Conversion Processing Unit 207 Conversion coefficient 208 Quantization Units 209 Quantized coefficients 209 Quantized transformation coefficients 209 Quantized residual coefficients 210 Inverse Quantization Unit 211 Dequantized coefficients 212 Inverse Transform Processing Unit 213 Reconstructed residual block 213 Conversion Blocks 214 Reconstruction Unit 214 Adder or analog adder 215 Reconstructed Blocks 216 buffers 220 Loop Filter Unit 220 Loop Filter 221 filtered blocks 221 Filtered Reconstruction Blocks 230 Decoded Picture Buffer (DPB) 231 Decoded picture 244 Interpretation Units 254 Intra Prediction Units 260 Mode Selection Unit 262 Partitioning Units 265 Prediction Blocks 265 Intra Prediction Block 265 predictors 266 Syntax Elements 270 Entropy Encoding Units 272 Output 272 Output Interfaces 304 Entropy Decoding Unit 309 Quantized coefficients 310 Inverse Quantization Unit 311 Conversion coefficient 311 Dequantization coefficients 312 Inverse Transform Processing Unit 313 Conversion Block 313 Reconstructed residual block 314 Reconstruction Unit 314 Analog Adder 315 Reconstructed Blocks 320 Loop Filter 320 Loop Filter Unit 321 filtered blocks 321 Decoded video blocks 330 Decoded Picture Buffer (DBP) 331 Decoded picture 344 Interpretation Units 354 Intra Prediction Units 360 Mode Applicable Unit 365 Prediction Block 400 video coding devices 410 Input Ports 420 Receiver Unit (Rx) 430 Central Processing Unit (CPU) 430 processors 440 Transmitter Unit (Tx) 450 output ports 460 memory 470 coding modules 500 devices 502 Processors 504 memory 506 Codes and Data 508 Operating Systems 510 Application Programs 512 Bus 514 Secondary Storage 518 displays 1100 Video Decoding Device 1110 Decision Unit 1120 prediction units 1200 video encoding devices 1210 Decision Unit 1220 prediction units 3100 Content Supply System 3102 Capture Device 3104 Communication Link 3106 Terminal device 3108 Smartphone or tablet 3110 Computer or laptop 3112 Network Video Recorder (NVR) / Digital Video Recorder (DVR) 3114 TV 3116 Set-top box (STB) 3118 Video conferencing system 3120 Video Surveillance System 3122 Personal Digital Assistant (PDA) 3124 In-vehicle devices 3126 Display 3126 External Display 3202 Protocol Progress Unit 3204 Reverse Multiplexing Unit 3206 Video Decoder 3208 Audio Decoder 3210 Subtitle Decoder 3212 Synchronization Unit 3214 Video / Audio Display 3216 Video / Audio / Subtitle Display

Claims

1. A method for decoding a video or picture bitstream implemented by a decoding device, wherein the bitstream includes data representing the current video coding layer, and the method The steps include receiving the bitstream, A step of determining whether the value of sps_video_parameter_set_id is equal to 0, wherein a value of sps_video_parameter_set_id equal to 0 indicates that the current video coding layer is the only layer in the bitstream. If it is determined that the value of sps_video_parameter_set_id is equal to 0, the step is to use the layer identifier of the Network Abstraction Layer (NAL) unit of the current video coding layer as the identifier of the current video coding layer, wherein the NAL unit is a syntactic structure that includes an indication of the type of data to follow and a byte containing that data, in the form of a Raw Byte Sequence Payload (RBSP) with emulation prevention bytes interspersed as needed. Methods that include...

2. The aforementioned method, The method according to claim 1, further comprising the step of predicting the current slice of the current video coding layer using the identifier of the current video coding layer.

3. The aforementioned method, The method according to claim 1 or 2, further comprising the step of obtaining the identifier of the current video coding layer by parsing the bitstream if it is determined that the value of sps_video_parameter_set_id is greater than 0.

4. The identifier of the current video coding layer is used to construct one or more reference picture lists of the current slice of the current video coding layer. The method according to any one of claims 1 to 3, wherein the step of predicting the current slice of the current video coding layer using the identifier of the current video coding layer includes the step of predicting the current slice using one or more reference picture lists.

5. The method according to any one of claims 1 to 4, wherein the NAL unit of the current video coding layer is a VCL NAL unit of the current video coding layer located in the bitstream.

6. The method according to claim 5, wherein the VCL NAL unit of the current video coding layer is any one of the VCL NAL units of the current video coding layer present in the bitstream.

7. A method for encoding a bitstream of a video or picture implemented by an encoding device, wherein the bitstream includes data representing the current video coding layer, and the method The steps include determining whether the current video coding layer is the only layer in the bitstream, If it is determined that the current video coding layer is the only layer in the bitstream, the step is to use the layer identifier of the network abstraction layer (NAL) unit of the current video coding layer as the identifier of the current video coding layer, wherein the NAL unit is a syntactic structure including an indication of the type of data to be followed and a byte containing that data, in the form of a Raw Byte Sequence Payload (RBSP) with emulation prevention bytes interspersed as needed. A step of encoding the value of sps_video_parameter_set_id into a bitstream, wherein a value of sps_video_parameter_set_id equal to 0 indicates that the current video coding layer is the only layer in the bitstream, and a value of sps_video_parameter_set_id greater than 0 indicates that the bitstream contains multiple layers. Methods that include...

8. The aforementioned method, The method of claim 7, further comprising the step of predicting the current slice of the current video coding layer using the identifier of the current video coding layer.

9. The aforementioned method, The method according to claim 7 or 8, further comprising the step of encoding the identifier of the current video coding layer into the bitstream if it is determined that the bitstream includes multiple layers.

10. Using the layer identifier of the NAL unit of the current video coding layer as the identifier of the current video coding layer means constructing one or more reference picture lists of the current slice of the current video coding layer, The method according to any one of claims 7 to 9, wherein the step of predicting the current slice of the current video coding layer using the identifier of the current video coding layer includes the step of predicting the current slice using one or more reference picture lists.

11. The method according to any one of claims 7 to 10, wherein the NAL unit of the current video coding layer is a VCL NAL unit of the current video coding layer located in the bitstream.

12. The method according to claim 11, wherein the VCL NAL unit of the current video coding layer is any one of the VCL NAL units of the current video coding layer present in the bitstream.

13. A coding device comprising a processing circuit for performing the method according to any one of claims 1 to 12.

14. A computer program, when executed on a computer or processor, comprising program code for performing the method described in any one of claims 1 to 12.

15. A non-temporary computer-readable medium that, when executed by a computer device, carries program code that causes the computer device to perform the method according to any one of claims 1 to 12.

16. A device for storing a bitstream of video or picture, comprising a communication interface and a storage medium, wherein the communication interface is configured to receive a bitstream of video or picture, and the storage medium is configured to store the bitstream, the bitstream comprising an sps_video_parameter_set_id syntax element, a nuh_layer_id syntax element and data representing the current video coding layer, wherein a sps_video_parameter_set_id value equal to 0 indicates that the current video coding layer is the only layer in the bitstream, and a sps_video_parameter_set_id value greater than 0 indicates that the bitstream comprises multiple layers, the nuh_layer_id syntax element represents a layer identifier of a Network Abstraction Layer (NAL) unit of the current video coding layer, and the NAL unit is a Raw Byte Sequence (NAL) with emulation prevention bytes interspersed as needed. A Payload)RBSP in the form of a syntactic structure including an instruction for the type of data to follow and the bytes containing that data, wherein the current video coding layer is the only layer in the bitstream when sps_video_parameter_set_id is equal to 0, and the nuh_layer_id is used as the identifier for the current video coding layer, for a device.

17. The device according to claim 16, wherein the communication interface is further configured to transmit the bitstream to another device.

Images