Encoder, decoder and corresponding method of indication and semantics in parameter set

Abstract

The invention relates to indicating syntax elements in a sequence parameter set of a video bitstream. Specifically, the present invention provides a method of decoding a video bitstream in which a sequence parameter set (SPS) is decoded in the video bitstream and includes syntax elements applied to a video sequence, the method comprising: obtaining a value of a first syntax element from the SPS, the value of the first syntax element is used for representing whether a parameter syntax structure of a decoded image buffer (DPB) exists in the SPS or not, and the value of the first syntax element is used for representing whether the parameter syntax structure of the DPB exists in the SPS or not; at least when it is determined that the value of the first syntax element indicates that the DPB parameter syntax structure exists in the SPS, the value of a second syntax element is obtained from the SPS, the value of the second syntax element is used for indicating whether the DPB parameter syntax structure exists or not, and the value of the second syntax element is used for indicating whether the DPB parameter syntax structure exists in the SPS. The DPB syntax element is applied to temporal sub-layers other than the highest temporal sub-layer in the video sequence.

Description

[0001] This application is a divisional application. The original application has the application number 202180017572.9 and the original application date is February 27, 2021. The entire contents of the original application are incorporated herein by reference.

[0002] Cross-referencing related applications This patent application claims priority to PCT / EP2020 / 055269, filed February 28, 2020; PCT / EP2020 / 065989, filed June 9, 2020; and PCT / EP2020 / 065999, filed June 9, 2020. The entire disclosure of the above patent applications is incorporated herein by reference. Technical Field

[0003] The embodiments of this application generally relate to the field of image processing, and more specifically, to the indication of syntax elements in a sequence parameter set. Background Technology

[0004] Video decoding (video encoding and decoding) is widely used in digital video applications such as broadcast digital television, video transmission based on the Internet and mobile networks, real-time conversational applications such as video chat and video conferencing, DVD and Blu-ray discs, video content capture and editing systems, and portable cameras for security applications.

[0005] Even relatively short videos require a significant amount of video data to describe, which can be challenging when streaming or otherwise transmitting data over communication networks with limited bandwidth. Therefore, video data is typically compressed before transmission over modern telecommunications networks. The size of the video can also be an issue when storing it on storage devices due to potentially limited memory resources. Video compression devices typically encode video data using software and / or hardware at the source side before transmitting or storing it, reducing the amount of data required to represent a digital video image. Video decompression devices then decode the video data and receive the compressed data at the destination side. Given limited network resources and the growing demand for higher video quality, there is a need to improve compression and decompression techniques to increase compression ratios with minimal impact on image quality.

[0006] Specifically, in the prior art, the syntax elements (used to provide decoding image buffer information) in the sequence parameter set of the code stream that indicate decoding are inefficient and even inconsistent. Therefore, the purpose of this application is to provide a technique for indicating these syntax elements to improve encoding efficiency. Summary of the Invention

[0007] This application provides embodiments of the apparatus and methods for encoding and decoding as described in the independent claims.

[0008] The above and other objectives are achieved through the subject matter claimed in the independent claims. Other implementations are apparent from the dependent claims, the description, and the drawings.

[0009] According to a first aspect, the present invention relates to a method for decoding a video stream implemented by a decoding device. A sequence parameter set (SPS) is decoded in the video stream and includes syntax elements applied to the video sequence. The method includes: obtaining a value of a first syntax element (e.g., a flag) from the SPS, wherein the value of the first syntax element is used to indicate whether a decoded picture buffer (DPB) parameter syntax structure exists in the SPS. The method further includes: obtaining a value of a second syntax element (e.g., a flag) from the SPS at least when it is determined that the value of the first syntax element indicates the presence of the DPB parameter syntax structure in the SPS, wherein the value of the second syntax element is used to indicate whether a DPB syntax element exists in the DPB parameter syntax structure, the DPB syntax element being applied to temporal sub-layers in the video sequence other than the highest temporal sub-layer.

[0010] For example, the value of the second syntax element can only be obtained when it is determined that the value of the first syntax element indicates the existence of the DPB parameter syntax structure in the SPS. Here and below, the first syntax element can be sps_ptl_dpb_hrd_params_present_flag as described in detail below, and the second syntax element can be sps_sublayer_dpb_params_flag. Here and below, the DPB syntax element can be one of max_dec_pic_buffering_minus1[i], max_num_reorder_pics[i], and max_latency_increase_plus1[i] as described in detail below.

[0011] The method for decoding video streams provided thus ensures efficient indication of DPB syntax elements. Specifically, if present, the second syntax element can reliably control the presence of DPB syntax elements in the DPB parameter syntax structure.

[0012] According to one implementation, the method further includes: (for example, when the second syntax element indicates that the DPB syntax element exists in the DPB parameter syntax structure; specifically, only when the second syntax element indicates that the DPB syntax element exists in the DPB parameter syntax structure) obtaining the value of the DPB syntax element based on the value of the second syntax element; and reconstructing the video sequence based on the value of the DPB syntax element. Therefore, the video sequence can be reconstructed reliably and efficiently by indicating DPB information.

[0013] The step of obtaining the value of the DPB syntax element based on the value of the second syntax element may include: When it is determined that the value of the second syntax element indicates the presence of the DPB syntax element in the DPB parameter syntax structure, the value of the DPB syntax element is obtained from the DPB parameter syntax structure; or When it is determined that the value of the second syntax element indicates that the DPB syntax element does not exist in the DPB parameter syntax structure, the value of the DPB syntax element is set to the value of another DPB syntax element in the DPB parameter syntax structure that is applied to the highest time sub-layer.

[0014] Therefore, it can be guaranteed that the definite values ​​of the DPB syntax elements are available under all circumstances and can be used to reconstruct the video sequence. There is no concern about any uncertain behavior in this regard.

[0015] The reliably obtained values ​​of the DPB syntax elements can thus be used to configure the DPB to store reference images for inter-frame prediction processing, and so on. Accordingly, the step of reconstructing the video sequence based on the values ​​of the DPB syntax elements may include: configuring the DPB based on the values ​​of the DPB syntax elements; and reconstructing the video sequence using the DPB.

[0016] Optionally, the step of reconstructing the video sequence based on the value of the DPB syntax element may include: reconstructing the video sequence by determining that the DPB used meets the requirements specified by the value of the DPB syntax element. Therefore, it is possible to check whether the provided DPB is suitable for reconstructing the video sequence.

[0017] According to one implementation, the method for decoding the video stream further includes: The value of the third syntax element is obtained from the SPS, wherein the value of the third syntax element is used to determine the maximum number of temporal sublayers present in the video sequence. If there is only one temporal sublayer, the value of the third syntax element can be 0. By simply indicating the third syntax element, it is helpful to determine the maximum number of temporal sublayers present in the video sequence, which is beneficial to improving coding efficiency.

[0018] Here and below, the third syntax element may be sps_max_sublayers_minus1, which is described in detail below.

[0019] The step of obtaining the value of the second syntax element from the SPS includes: When it is determined that the value of the first syntax element indicates the presence of the DPB parameter syntax structure in the SPS, the maximum number of time sub-layers in the video bitstream is determined to be greater than 1 based on the value of the third syntax element. When it is determined that the maximum number of time sub-layers is greater than 1, the value of the second syntax element is obtained from the SPS. For example, the value of the second syntax element can only be obtained from the SPS when it is determined that the maximum number of time sub-layers is greater than 1. Therefore, since the value of the second syntax element may not be read at all when the maximum number of time sub-layers is not greater than 1 (i.e., there is only one time sub-layer), (for example, if the value of the second syntax element becomes meaningless in this case), and the DPB syntax element may always be indicated in the SPS in a single time sub-layer, coding efficiency can be further improved.

[0020] According to the second aspect, a method for encoding a video bitstream, implemented by an encoding device, is provided. A sequence parameter set (SPS) is decoded into the video bitstream and includes syntax elements applied to the video sequence. The advantages of this method are the same as those discussed above. The method includes the following steps: Determine whether the SPS contains a decoded picture buffer (DPB) parameter syntax structure; By determining whether the DPB parameter syntax structure exists in the SPS, the value of a first syntax element (e.g., a flag) is encoded into the SPS, wherein the value of the first syntax element is used to indicate whether the DPB parameter syntax structure exists in the SPS; When it is determined (e.g., only if) that the DPB parameter syntax structure exists in the SPS, it is determined whether there is a DPB syntax element in the DPB parameter syntax structure, wherein the DPB syntax element is applied to the temporal sub-layers in the video sequence other than the highest temporal sub-layer. By determining whether the DPB syntax element exists in the DPB parameter syntax structure, the value of the second syntax element (e.g., a flag) is encoded into the SPS, wherein the value of the second syntax element is used to indicate that the DPB syntax element exists in the DPB parameter syntax structure.

[0021] According to one implementation, the encoding method further includes: when it is determined that the DPB syntax element exists in the DPB parameter syntax structure, determining the value of the DPB syntax element; and reconstructing the video sequence based on the value of the DPB syntax element.

[0022] According to one implementation, the encoding method further includes: setting the value of the DPB syntax element to the value of another DPB syntax element in the DPB parameter syntax structure applied to the highest temporal sublayer; and reconstructing the video sequence based on the value of the DPB syntax element.

[0023] The step of reconstructing the video sequence based on the value of the DPB syntax element may include: configuring the DPB to satisfy the value of the DPB syntax element; and using the DPB to reconstruct the video sequence.

[0024] According to one implementation, when it is determined that the DPB parameter syntax structure exists in the SPS and the maximum number of time sub-layers in the video bitstream is greater than 1, it is determined that the DPB parameter syntax structure contains DPB syntax elements.

[0025] In addition, an apparatus for decoding a video stream and an apparatus for encoding a video stream are provided, which have the same advantages as the methods described above.

[0026] According to a third aspect, an apparatus is provided for decoding a (decoded) video stream. The apparatus includes: The acquisition unit is used to acquire the value of a first syntax element (e.g., a flag) from the SPS, wherein the value of the first syntax element is used to indicate whether a decoded picture buffer (DPB) parameter syntax structure exists in the SPS, and the SPS is decoded in the video bitstream; A determining unit is configured to determine whether the value of the first syntax element indicates that the DPB parameter syntax structure exists in the SPS; The acquisition unit is further configured to: at least when it is determined (e.g., only when) that the value of the first syntax element indicates that the DPB parameter syntax structure exists in the SPS, acquire the value of a second syntax element (e.g., a flag) from the SPS, wherein the value of the second syntax element is used to indicate whether a DPB syntax element exists in the DPB parameter syntax structure, and the DPB syntax element is applied to time sub-layers in the video sequence other than the highest time sub-layer.

[0027] For example, the acquisition unit can only acquire the value of the second syntax structure when it is determined that the value of the first syntax element indicates that the DPB parameter syntax structure exists in the SPS.

[0028] According to one implementation, the acquisition unit is further configured to: acquire the value of the DPB syntax element based on the value of the second syntax element; and reconstruct the video sequence based on the value of the DPB syntax element.

[0029] The step of obtaining the value of the DPB syntax element based on the value of the second syntax element may include: When it is determined that the value of the second syntax element indicates the presence of the DPB syntax element in the DPB parameter syntax structure, the value of the DPB syntax element is obtained from the DPB parameter syntax structure; or When it is determined that the value of the second syntax element indicates that the DPB syntax element does not exist in the DPB parameter syntax structure, the value of the DPB syntax element is set to the value of another DPB syntax element in the DPB parameter syntax structure that is applied to the highest time sub-layer.

[0030] The process of reconstructing the video sequence based on the value of the DPB syntax element may include: configuring the DPB according to the value of the DPB syntax element; and using the DPB to reconstruct the video sequence.

[0031] DPB is used to store images, which are then used to build a list of reference images.

[0032] Optionally, reconstructing the video sequence based on the value of the DPB syntax element may include: reconstructing the video sequence by determining that the DPB used meets the requirements specified by the value of the DPB syntax element.

[0033] According to one implementation, the acquisition unit is further configured to: acquire the value of a third syntax element from the SPS, wherein the value of the third syntax element is used to determine the maximum number of temporal sub-layers present in the video sequence.

[0034] Obtaining the value of the second syntax element from the SPS may include: When it is determined that the value of the first syntax element indicates that the DPB parameter syntax structure exists in the SPS, the maximum number of time sub-layers in the video bitstream is determined to be greater than 1 based on the value of the third syntax element; when it is determined that the maximum number of time sub-layers is greater than 1, the value of the second syntax element is obtained from the SPS.

[0035] According to a fourth aspect, an apparatus for encoding a video stream is provided. The apparatus includes: The determination unit is used to determine whether the decoded picture buffer (DPB) parameter syntax structure exists in the SPS; An encoding unit is configured to encode the value of a first syntax element (e.g., a flag) into the SPS by determining whether the DPB parameter syntax structure exists in the SPS, wherein the value of the first syntax element is used to indicate whether the DPB parameter syntax structure exists in the SPS; The determining unit is further configured to: when it is determined (e.g., only when) that the DPB parameter syntax structure exists in the SPS, determine whether there is a DPB syntax element in the DPB parameter syntax structure, wherein the DPB syntax element is applied to time sub-layers in the video sequence other than the highest time sub-layer; The encoding unit is further configured to: encode the value of a second syntax element (e.g., a flag) into the SPS by determining whether the DPB syntax element exists in the DPB parameter syntax structure, wherein the value of the second syntax element is used to indicate whether the DPB syntax element exists in the DPB parameter syntax structure.

[0036] According to one implementation, the determining unit is further configured to: when it is determined that the DPB syntax element exists in the DPB parameter syntax structure, determine the value of the DPB syntax element; and reconstruct the video sequence based on the value of the DPB syntax element.

[0037] According to one implementation, the encoding unit is further configured to: set the value of the DPB syntax element to the value of another DPB syntax element in the DPB parameter syntax structure applied to the highest temporal sublayer; and reconstruct the video sequence based on the value of the DPB syntax element.

[0038] The process of reconstructing the video sequence based on the value of the DPB syntax element may include: configuring the DPB to satisfy the value of the DPB syntax element; and using the DPB to reconstruct the video sequence.

[0039] According to one implementation, the determining unit is configured to: determine that a DPB syntax element exists in the DPB parameter syntax structure when the determining unit determines that the DPB parameter syntax structure exists in the SPS and the maximum number of time sub-layers in the video bitstream is greater than 1.

[0040] The above methods can be implemented in either a decoding device or an encoding device. Therefore, an encoder is provided. The encoder includes processing circuitry for performing a method of encoding a video stream according to any of the above examples. Furthermore, an encoder is provided. The encoder includes: one or more processors; a non-transitory computer-readable storage medium coupled to the one or more processors and storing a program executable by the one or more processors, wherein, when the program is executed by the one or more processors, the encoder is used to perform a method of encoding a video stream according to any of the above examples. Similarly, a decoder is provided. The decoder includes processing circuitry for performing a method of decoding a video stream according to any of the above examples. A decoder is provided. The decoder includes: one or more processors; a non-transitory computer-readable storage medium coupled to the one or more processors and storing a program executable by the one or more processors, wherein, when the program is executed by the one or more processors, the decoder is used to perform a method of decoding a video stream according to any of the above examples.

[0041] Furthermore, a computer program product including program code is provided. When the program code is executed on a computer or processor, the program code is used to perform the method according to any of the above examples. Similarly, a non-transitory computer-readable medium including program code is provided. When the program code is executed by a computer device, it causes the computer device to perform the method according to any of the above examples.

[0042] Furthermore, a non-transient storage medium is provided. The non-transient storage medium includes an encoded bitstream generated by dividing a current image of a video signal or image signal into multiple blocks. The bitstream includes multiple syntax elements; the multiple syntax elements include a first syntax element in the SPS, the value of which indicates whether a decoded picture buffer (DPB) parameter syntax structure exists in the SPS; when the value of the first syntax element indicates the presence of the DPB parameter syntax structure in the SPS, the bitstream also includes a second syntax element in the SPS, the value of which indicates whether a DPB syntax element exists in the DPB parameter syntax structure, and the DPB syntax element is applied to temporal sub-layers in the video sequence other than the highest temporal sub-layer.

[0043] The following drawings and description illustrate one or more embodiments in detail. Other features, objects, and advantages will be apparent from the description, drawings, and claims. Attached Figure Description

[0044] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings and schematic diagrams.

[0045] Figure 1A This is a block diagram of an example of a video decoding system for implementing an embodiment of the present invention.

[0046] Figure 1B This is a block diagram of another example of a video decoding system for implementing embodiments of the present invention.

[0047] Figure 2 This is a block diagram of an example video encoder used to implement an embodiment of the present invention.

[0048] Figure 3 This is a block diagram of an exemplary structure for implementing a video decoder according to an embodiment of the present invention.

[0049] Figure 4 A block diagram of an example encoding or decoding device.

[0050] Figure 5 A block diagram for another example of an encoding or decoding device.

[0051] Figure 6 An example of a raster scan sequence is shown.

[0052] Figure 7 An example of blocks, stripes, and sub-images is shown.

[0053] Figure 8 A framework of an exemplary structure for a content delivery system 3100 that implements content distribution services.

[0054] Figure 9 A block diagram of an exemplary structure for a terminal device.

[0055] Figure 10 An example of image segmentation is shown.

[0056] Figure 11 Examples of the layers and sublayers of scalable video coding are shown.

[0057] Figure 12 Another example of image segmentation is shown.

[0058] Figure 13 Another example of image segmentation is shown.

[0059] Figure 14 An embodiment of a method for decoding a video stream is shown.

[0060] Figure 15An embodiment of a method for encoding a video stream is shown.

[0061] Figure 16 An embodiment of a method for decoding a video stream is shown.

[0062] Figure 17 An embodiment of a method for encoding a video stream is shown.

[0063] Unless otherwise explicitly stated below, the same reference numerals refer to the same features or at least functionally equivalent features. Detailed Implementation

[0064] In the following description, reference is made to the accompanying drawings, which form part of this invention, and which illustrate by way of description specific aspects of embodiments of the invention or aspects in which embodiments of the invention may be used. It should be understood that embodiments of the invention may be used in other aspects and may include structural or logical variations not depicted in the drawings. Therefore, the following detailed description is not to be construed as limiting, and the scope of the invention is defined by the appended claims.

[0065] For example, it should be understood that the disclosure relating to the described method can also apply to the corresponding device or system for performing the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more units, such as functional units, to perform the described one or more method steps (e.g., one unit performs one or more steps, or multiple units perform one or more of multiple steps respectively), even if such one or more units are not explicitly described or shown in the drawings. On the other hand, for example, if a specific apparatus is described according to one or more units (e.g., functional units), the corresponding method may include a step to perform the function of one or more units (e.g., one step performs the function of one or more units, or multiple steps perform the function of one or more of multiple units respectively), even if such one or more steps are not explicitly described or shown in the drawings. Furthermore, it should be understood that, unless otherwise expressly stated, features of the various exemplary embodiments and / or aspects described herein can be combined with each other.

[0066] Video decoding generally refers to the processing of image sequences that form a video or video sequence. In the field of video coding, the terms "frame" and "picture / image" can be used synonymously. Video decoding (or generally referred to as decoding) consists of two parts: video encoding and video decoding. Video encoding is performed on the source side and typically involves processing (e.g., by compression) the raw video image to reduce the amount of data representing the video image (thus storing and / or transmitting it more efficiently). Video decoding is performed on the destination side and typically involves inverse processing relative to the encoder to reconstruct the video image. The "decoding" of the video image (or generally referred to as an image) involved in the embodiments should be understood as the "encoding" or "decoding" of the video image or the corresponding video sequence. The encoding and decoding parts are also collectively referred to as encoding and decoding (encoding and decoding).

[0067] In lossless video decoding, the original video image can be reconstructed, meaning the reconstructed video image has the same quality as the original (assuming no transmission loss or other data loss during storage or transmission). In lossy video decoding, further compression is performed through quantization to reduce the amount of data representing the video image, and the decoder cannot completely reconstruct the video image, meaning the quality of the reconstructed video image is lower or worse than the quality of the original video image.

[0068] Several video coding standards belong to the "lossy hybrid video codec" group (i.e., combining spatial and temporal prediction in the sample domain with 2D transform decoding in the transform domain for applying quantization). Each image in a video sequence is typically segmented into a set of non-overlapping blocks, usually decoded at the block level. In other words, the encoder side typically processes, i.e., encodes the video at the block (video block) level, for example, generating prediction blocks through spatial (intra-frame) prediction and / or temporal (inter-frame) prediction, subtracting the prediction blocks from the current block (the block currently being processed / to be processed) to obtain residual blocks, transforming and quantizing the residual blocks in the transform domain to reduce the amount of data to be transmitted (compressed), while the decoder side applies the inverse processing relative to the encoder to the encoded or compressed blocks to reconstruct the current block for representation. Furthermore, the encoder replicates the decoder processing loop, such that the encoder and decoder generate the same predictions (e.g., intra-frame and inter-frame predictions) and / or reconstructions for processing subsequent blocks, i.e., decoding.

[0069] In the following embodiment of the video decoding system 10, the video encoder 20 and the video decoder 30 are based on Figures 1A to 3 Describe it.

[0070] Figure 1AThis is a schematic block diagram of an example decoding system 10, such as a video decoding system 10 (or simply decoding system 10) that can utilize the technology of this application. The video encoder 20 (or simply encoder 20) and video decoder 30 (or simply decoder 30) in the video decoding system 10 are two examples, i.e., devices that can be used to perform various technologies according to the various examples described in this application.

[0071] like Figure 1A As shown, the decoding system 10 includes a source device 12, which provides encoded image data 21 to a destination device 14 or the like for decoding the encoded image data 21.

[0072] The source device 12 includes an encoder 20 and may additionally (optionally) include an image source 16, a preprocessor (or preprocessing unit) 18 (e.g., an image preprocessor 18), and a communication interface or communication unit 22.

[0073] Image source 16 may include or may be any type of image capture device such as a camera for capturing real-world images, and / or any type of image generation device such as a computer graphics processor for generating computer-generated animated images, or any other device for acquiring and / or providing real-world images, computer-generated images (e.g., screen content, virtual reality (VR) images), and / or any combination thereof (e.g., augmented reality (AR) images). Image source may be any type of memory / storage for storing any of the aforementioned images.

[0074] To distinguish between the processing performed by the preprocessor 18 and the preprocessing unit 18, the image or image data 17 may also be referred to as the raw image or raw image data 17.

[0075] The preprocessor 18 receives (raw) image data 17 and performs preprocessing on the image data 17 to obtain a preprocessed image 19 or preprocessed image data 19. The preprocessing performed by the preprocessor 18 may include trimming, color format conversion (e.g., from RGB to YCbCr), color correction, or noise reduction. It is understood that the preprocessing unit 18 may be an optional component.

[0076] Video encoder 20 is used to receive preprocessed image data 19 and provide encoded image data 21 (in combination with...) Figure 2 (More details will be provided later).

[0077] The communication interface 22 in the source device 12 can be used to receive the encoded image data 21 and send the encoded image data 21 (or data obtained after further processing of the encoded image data 21) to another device (e.g., the destination device 14) or any other device via the communication channel 13 for storage or direct reconstruction.

[0078] Destination device 14 includes decoder 30 (e.g., video decoder 30) and may additionally (optionally) include communication interface or communication unit 28, post-processor 32 (or post-processing unit 32) and display device 34.

[0079] The communication interface 28 in the destination device 14 is used to receive encoded image data 21 (or data obtained after further processing of encoded image data 21) directly from the source device 12 or from any other source such as a storage device (e.g., an encoded image data storage device) and to provide the encoded image data 21 to the decoder 30.

[0080] Communication interfaces 22 and 28 can be used to send or receive encoded image data 21 or encoded data 13 via a direct communication link (e.g., a direct wired or wireless connection) between source device 12 and destination device 14 or via any type of network (e.g., a wired network, a wireless network or any combination thereof, or any type of private and public network or any type of combination thereof).

[0081] For example, communication interface 22 can be used to encapsulate encoded image data 21 into a suitable format (e.g., data packets) and / or process the encoded image data through any type of transmission encoding or processing method for transmission over a communication link or communication network.

[0082] For example, the communication interface 28 corresponding to the communication interface 22 can be used to receive transmitted data and process the transmitted data through any type of corresponding transmission decoding or processing and / or decapsulation method to obtain encoded image data 21.

[0083] Both communication interface 22 and communication interface 28 can be configured as Figure 1A The communication channel 13, from source device 12 to destination device 14, is a one-way communication interface indicated by the arrow, or configured as a two-way communication interface, and can be used to send and receive messages, etc., to establish connections, acknowledge and exchange any other information related to the communication link and / or data transmission (e.g., encoded image data transmission), etc.

[0084] Decoder 30 is used to receive encoded image data 21 and provide decoded image data 31 or decoded image 31 (hereinafter combined) Figure 3 or Figure 5 (More details will be provided later).

[0085] The post-processor 32 in the destination device 14 is used to post-process the decoded image data 31 (also referred to as reconstructed image data) (e.g., decoded image 31) to obtain post-processed image data 33 (e.g., post-processed image 33). The post-processing performed by the post-processing unit 32 may include color format conversion (e.g., from YCbCr to RGB), color correction, trimming, or resampling, or any other processing, to provide the decoded image data 31 for display by the display device 34, etc.

[0086] The display device 34 in the destination device 14 is used to receive post-processed image data 33 in order to display the image to a user or viewer. The display device 34 can be or can include any type of display for representing the reconstructed image, such as an integrated or external display or screen. For example, the display can include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS) display, a digital light processor (DLP), or any other type of display.

[0087] Although Figure 1A In this embodiment, source device 12 and destination device 14 are shown as separate devices; however, device embodiments may also include both devices or the functions of both devices simultaneously, i.e., source device 12 or its corresponding functions and destination device 14 or its corresponding functions. In these embodiments, the same hardware and / or software, or separate hardware and / or software, or any combination thereof, may be used to implement the functions of source device 12 or its corresponding functions and destination device 14 or its corresponding functions.

[0088] It is obvious to those skilled in the art from the description that Figure 1A The presence and (precise) functional division of different units or functions within the source device 12 and / or destination device 14 shown may vary depending on the actual device and application.

[0089] Encoder 20 (e.g., video encoder 20) or decoder 30 (e.g., video decoder 30), or encoder 20 and decoder 30, can be... Figure 1BThe processing circuitry shown is used to implement this, for example, one or more microprocessors, one or more digital signal processors (DSPs), one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), one or more discrete logic devices, one or more hardware devices, one or more dedicated video decoding processors, or any combination thereof. Encoder 20 can be implemented via processing circuitry 46 to include reference... Figure 2 The encoder 20 describes various modules and / or any other encoder system or subsystem described herein. The decoder 30 may be implemented via processing circuitry 46 to include references. Figure 3 The decoder 30 describes various modules and / or any other decoder system or subsystem described herein. Processing circuitry can be used to perform the various operations described below. Figure 5 As shown, if the aforementioned technical components are implemented in software, a device can store the instructions for that software in a suitable non-transitory computer-readable storage medium, and these instructions can be executed in hardware using one or more processors to perform the techniques of this invention. The video encoder 20 or video decoder 30 can be integrated into a single device as part of a combined encoder / decoder (codec), such as... Figure 1B As shown.

[0090] Source device 12 and destination device 14 can include any of a variety of devices, including any type of handheld or fixed device, such as a laptop or notebook computer, mobile phone, smartphone, tablet or tablet computer, camera, desktop computer, set-top box, television, display device, digital media player, video game console, video streaming device (e.g., content service server or content distribution server), broadcast receiver device, broadcast transmitter device, etc., and may or may not use any type of operating system. In some cases, source device 12 and destination device 14 can be used for wireless communication. Therefore, source device 12 and destination device 14 can be wireless communication devices.

[0091] In some cases, Figure 1AThe video encoding and decoding system 10 shown is merely an example, and the techniques in this application can be applied to video encoding setups (e.g., video encoding or video decoding) that do not necessarily involve any data communication between the encoding and decoding devices. In other examples, data is retrieved from local storage, streamed over a network, etc. A video encoding device can encode data and store it in memory, and / or a video decoding device can retrieve data from memory and decode it. In some examples, encoding and decoding are performed by devices that do not communicate with each other but only encode data to memory and / or retrieve data from memory and decode the data.

[0092] For ease of description, this document (for example) refers to reference software developed by the Joint Collaboration Team on Video Coding (JCT-VC) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG), namely High-Efficiency Video Coding (HEVC) or Versatile Video Coding (VVC), a next-generation video coding standard. Those skilled in the art will understand that embodiments of the invention are not limited to HEVC or VVC.

[0093] Encoders and Encoding Methods Figure 2 This is a schematic block diagram of an exemplary video encoder 20 used to implement the technology in this application. Figure 2 In the example, the video encoder 20 includes an input terminal 201 (or input interface 201), a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a loop filter unit 220, a decoded picture buffer (DPB) 230, a mode selection unit 260, an entropy coding unit 270, and an output terminal 272 (or output interface 272). The mode selection unit 260 may include an inter-frame prediction unit 244, an intra-frame prediction unit 254, and a segmentation unit 262. The inter-frame prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). Figure 2 The video encoder 20 shown can also be called a hybrid video encoder or a video encoder based on a hybrid video codec.

[0094] The residual calculation unit 204, transform processing unit 206, quantization unit 208, and mode selection unit 260 can form the forward signal path of the encoder 20, while the inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, buffer 216, loop filter 220, decoded picture buffer (DPB) 230, inter-frame prediction unit 244, and intra-frame prediction unit 254 can form the backward signal path of the video encoder 20. The backward signal path of the video encoder 20 corresponds to the decoder (see [link to decoder]). Figure 3 The signal path of the video decoder 30 in the video encoder 20. The inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, loop filter 220, decoded picture buffer (DPB) 230, inter-frame prediction unit 244 and intra-frame prediction unit 254 also constitute the "built-in decoder" of the video encoder 20.

[0095] Image and image segmentation (images and blocks) Encoder 20 can be used to receive image 17 (or image data 17) via input terminal 201, etc. Image 17 can be an image in a series of images that make up a video or video sequence. The received image or image data can also be a pre-processed image 19 (or pre-processed image data 19). For simplicity, the following description uses image 17. Image 17 can also be referred to as the current image or the image to be decoded (especially in video decoding to distinguish the current image from other images in the same video sequence (i.e., video sequences that also include the current image) such as previously encoded and / or decoded images).

[0096] A (digital) image is, or can be viewed as, a two-dimensional array or matrix composed of samples with intensity values. Samples in an array can also be called pixels (short for image elements). The number of samples in the array or image along the horizontal and vertical directions (or axes) defines the image size and / or resolution. To represent color, three color components are typically used; that is, an image can be represented as, or can include, an array of three samples. In RGB format or color space, an image includes corresponding red, green, and blue sample arrays. However, in video decoding, each pixel is typically represented in a luminance and chrominance format or color space, such as YCbCr, including the luminance component represented by Y (sometimes also L) and two chrominance components represented by Cb and Cr. The luminance component Y represents brightness or grayscale intensity (e.g., both are the same in grayscale images), while the two chrominance components Cb and Cr represent chrominance or color information components. Therefore, an image in YCbCr format consists of a luminance sample array composed of luminance sample values ​​(Y) and two chrominance sample arrays composed of chrominance values ​​(Cb and Cr). An RGB format image can be converted or transformed to YCbCr format, and vice versa. This process is also called color transformation or conversion. If the image is black and white, it may only include the luminance sample array. Accordingly, for example, an image may be a black and white luminance sample array or a luminance sample array in 4:2:0, 4:2:2, and 4:4:4 color formats, along with two corresponding chrominance sample arrays.

[0097] Embodiments of the video encoder 20 may include an image segmentation unit ( Figure 2 (Not shown in the image) is used to segment image 17 into multiple (typically non-overlapping) image blocks 203. These blocks may also be referred to as root blocks, macroblocks (in H.264 / AVC), or coding tree blocks (CTBs) or coding tree units (CTUs) (in H.265 / HEVC and VVC). Image segmentation units can be used to apply the same block size and a corresponding grid with a defined block size to all images in a video sequence, or to vary the block size between images, subsets of images, or groups of images, and to segment each image into multiple corresponding blocks.

[0098] In other embodiments, the video encoder may be used to directly receive blocks 203 in image 17, such as one, several, or all of the blocks that make up image 17. Image block 203 may also be referred to as the current image block or the image block to be decoded.

[0099] Similar to image 17, image block 203 is also, or can be considered as, a two-dimensional array or matrix composed of samples with intensity values ​​(sample values), but the size of image block 203 is smaller than that of image 17. In other words, depending on the applied color format, block 203 may include, for example, a single sample array (e.g., a luminance array in the case of black and white image 17, or a luminance or chrominance array in the case of a color image), three sample arrays (e.g., one luminance array and two chrominance arrays in the case of color image 17), or any other number and / or type of array. The number of samples in the horizontal and vertical directions (or axes) of block 203 defines the size of block 203. Accordingly, a block may be an M×N (M columns × N rows) sample array, or an M×N transform coefficient array, etc.

[0100] Figure 2 The embodiment of the video encoder 20 shown can be used to encode the image 17 block by block, for example, to perform encoding and prediction for each block 203.

[0101] Figure 2 The embodiment of the video encoder 20 shown can also be used to segment and / or encode images by slices (also known as video slices). An image can be segmented into one or more (typically non-overlapping) slices or encoded by one or more (typically non-overlapping) slices, each slice may include one or more blocks (e.g., CTUs).

[0102] Figure 2 The embodiment of the video encoder 20 shown can also be used to segment and / or encode an image by tile groups (also referred to as video tile groups) and / or blocks (also referred to as video blocks). An image can be segmented into one or more (typically non-overlapping) tile groups or encoded by one or more (typically non-overlapping) tile groups; each tile group may include one or more blocks (e.g., CTUs) or one or more blocks, etc.; each block may be rectangular, etc., and may include one or more complete or partial blocks (e.g., CTUs, etc.).

[0103] Residual calculation The residual calculation unit 204 can be used to calculate the residual block 205 (also referred to as residual 205) based on the image block 203 and the prediction block 265 (which is described in detail later) to obtain the residual block 205 in the sample domain: for example, subtracting the sample value of the prediction block 265 from the sample value of the image block 203 on a sample-by-sample (pixel-by-pixel) basis.

[0104] Transformation The transform processing unit 206 can be used to apply transforms such as discrete cosine transform (DCT) or discrete sine transform (DST) to the sample values ​​of the residual block 205 to obtain the transform coefficients 207 in the transform domain. The transform coefficients 207 can also be called transform residual coefficients, representing the residual block 205 in the transform domain.

[0105] Transform processing unit 206 can be used to apply an integer approximation of DCT / DST (e.g., a transform specified for H.265 / HEVC). This integer approximation is typically scaled by a certain factor compared to orthogonal DCT transforms. To maintain the norm of the residual blocks processed by the forward and inverse transforms, other scaling factors are applied as part of the transform process. The scaling factor is typically selected based on certain constraints, such as powers of 2 used for shift operations, the bit depth of the transform coefficients, and a trade-off between precision and implementation cost. For example, a specific scaling factor is specified for the inverse transform (and for the corresponding inverse transform by inverse transform processing unit 212, etc.) by the inverse transform processing unit 212, etc.; correspondingly, the encoder 20 can specify a corresponding scaling factor for the forward transform by transform processing unit 206, etc.

[0106] Embodiments of the video encoder 20 (correspondingly, the transform processing unit 206) can be used to directly output or encode or compress transform parameters (e.g., one or more types of transforms) through the entropy encoding unit 270, so that the video decoder 30 can receive and use the transform parameters for decoding, etc.

[0107] Quantification The quantization unit 208 can be used to quantize the transformation coefficients 207 by applying scalar quantization or vector quantization, etc., to obtain the quantized coefficients 209. The quantized coefficients 209 can also be called quantized transformation coefficients 209 or quantized residual coefficients 209.

[0108] The quantization process can reduce the bit depth associated with some or all of the transform coefficients 207. For example, n-bit transform coefficients can be rounded down to m-bit transform coefficients during quantization, where n is greater than m. The degree of quantization can be modified by adjusting the quantization parameter (QP). For example, for scalar quantization, different degrees of scaling can be used to achieve finer or coarser quantization. Smaller quantization step sizes correspond to finer quantization, while larger quantization step sizes correspond to coarser quantization. The appropriate quantization step size can be indicated by the quantization parameter (QP). For example, the quantization parameter can be a set of predefined indices for appropriate quantization step sizes. For example, smaller quantization parameters can correspond to fine quantization (smaller quantization step size), larger quantization parameters can correspond to coarse quantization (larger quantization step size), and vice versa. Quantization may include division by the quantization step size, while the corresponding dequantization and / or dequantization performed by the dequantization unit 210, etc., may include multiplication by the quantization step size. According to embodiments of some standards such as HEVC, the quantization parameter can be used to determine the quantization step size. In general, the quantization step size can be calculated based on the quantization parameter using a fixed-point approximation of an equation including division. Additional scaling factors can be introduced for quantization and dequantization to recover the norm of the residual block, which may have been modified by scaling used in the fixed-point approximation of the equations used for the quantization step size and quantization parameters. In one exemplary implementation, the scaling of the inverse transform and dequantization can be combined. Alternatively, a custom quantization table can be used and signaled from the encoder to the decoder in the bitstream, etc. Quantization is a lossy operation, where the loss increases with the quantization step size.

[0109] An embodiment of the video encoder 20 (correspondingly, the quantization unit 208) can be used to directly output or encode quantization parameters (QP) through the entropy coding unit 270, so that the video decoder 30 can receive and use the quantization parameters for decoding, etc.

[0110] Inverse Quantization The dequantization unit 210 is used to apply the dequantization of the quantization coefficients to the quantization coefficients by the quantization unit 208 to obtain the dequantized coefficients 211, for example, by applying a dequantization scheme opposite to the quantization scheme applied by the quantization unit 208, using the same quantization step size as the quantization unit 208. The dequantized coefficients 211 can also be called dequantized residual coefficients 211, corresponding to the transform coefficients 207, but due to the loss caused by quantization, they are usually different from the transform coefficients.

[0111] Inverse Transformation The inverse transform processing unit 212 is used to apply the inverse transform of the transform applied by the transform processing unit 206, such as the inverse discrete cosine transform (DCT) or the inverse discrete sine transform (DST) or other inverse transforms, to obtain the reconstructed residual block 213 (or the corresponding dequantized coefficients 213) in the sample domain. The reconstructed residual block 213 can also be referred to as the transform block 213.

[0112] reconstruction Reconstruction unit 214 (e.g., adder or summer 214) is used to add transform block 213 (i.e. reconstructed residual block 213) to prediction block 265 in such a way as to obtain reconstructed block 215 in the sample domain: for example, by adding the sample values ​​of reconstructed residual block 213 and prediction block 265 one sample at a time.

[0113] Filtering Loop filter unit 220 (or simply "loop filter" 220) is used to filter the reconstructed block 215 to obtain the filtered block 221, or is typically used to filter reconstructed samples to obtain filtered sample values. For example, the loop filter unit is used to smoothly perform pixel shifts or otherwise improve video quality. Loop filter unit 220 may include one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, a co-filter, or any combination thereof. Although loop filter unit 220 in... Figure 2 The loop filter unit 220 is shown as an in-loop filter, but in other configurations, it can be implemented as a post-loop filter. The filtered block 221 can also be referred to as the filtered reconstructed block 221.

[0114] An embodiment of the video encoder 20 (correspondingly, the loop filter unit 220) can be used to directly output or encode loop filter parameters (e.g., sample adaptive offset information) through the entropy coding unit 270, so that the decoder 30 can receive and use the same loop filter parameters or the corresponding loop filter for decoding, etc.

[0115] Decoding image buffer The decoded picture buffer (DPB) 230 can be a memory that stores reference images or, typically, reference image data, used by the video encoder 20 to encode video data. The DPB 230 can be formed from any of a variety of storage devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of storage devices. The decoded picture buffer (DPB) 230 can be used to store one or more filtered blocks 221. The decoded picture buffer 230 can also be used to store other previously filtered blocks (e.g., previously filtered reconstructed blocks 221) in the same current image or different images (e.g., previous reconstructed images), and can provide previously fully reconstructed (i.e., decoded) images (and corresponding reference blocks and samples) and / or partially reconstructed current images (and corresponding reference blocks and samples) for inter-frame prediction, etc. The decoded picture buffer (DPB) 230 can also be used to: store one or more unfiltered reconstructed blocks 215 or normally store unfiltered reconstructed samples if the reconstructed block 215 is not filtered by the loop filter unit 220, or to store any other blocks or samples obtained after further processing of the reconstructed blocks or samples.

[0116] Pattern selection (segmentation and prediction) The mode selection unit 260 includes a segmentation unit 262, an inter-frame prediction unit 244, and an intra-frame prediction unit 254, and is used to receive or acquire raw image data such as the original block 203 (the current block 203 of the current image 17) and reconstructed image data (e.g., filtered and / or unfiltered reconstructed samples or blocks of the same (current) image and / or one or more previous decoded images) from the decoded image buffer 230 or other buffers (e.g., a line buffer, not shown in the figure). The reconstructed image data is used as reference image data for predictions such as inter-frame prediction or intra-frame prediction to obtain prediction block 265 or prediction value 265.

[0117] The mode selection unit 260 can be used to determine or select a segmentation method for the current block prediction mode (including no segmentation) and to determine or select a prediction mode (e.g., intra-frame or inter-frame prediction mode), and generate a corresponding prediction block 265. The prediction block 265 is used to calculate the residual block 205 and reconstruct the reconstructed block 215.

[0118] Embodiments of the mode selection unit 260 can be used to select a segmentation method and a prediction mode (e.g., from those modes supported or available by the mode selection unit 260). The prediction mode provides the best match or minimum residual (minimum residual implies better compression in transmission or storage), or provides minimum signaling overhead (minimum signaling overhead implies better compression in transmission or storage), or considers or balances both. The mode selection unit 260 can be used to determine the segmentation method and prediction mode based on rate distortion optimization (RDO), i.e., selecting the prediction mode that provides minimum rate distortion. The terms "best," "minimum," and "optimal" in this document do not necessarily refer to "best," "minimum," or "optimal" overall, but can also refer to situations that meet termination or selection criteria. For example, values ​​exceeding or falling below a threshold or other constraints may lead to a "suboptimal choice," but reduce complexity and processing time.

[0119] In other words, the segmentation unit 262 can be used to divide the block 203 into smaller block partitions or sub-blocks (reforming blocks) in the following ways: for example, iteratively using quad-tree (QT) segmentation, binary-tree (BT) segmentation, or triple-tree (TT) segmentation or any combination thereof; and to perform predictions on each block partition or sub-block, etc., wherein mode selection includes selecting the tree structure of the segmented block 203, and the prediction mode is applied to each block partition or sub-block.

[0120] The segmentation (e.g., by segmentation unit 260) and prediction processing (performed by inter-frame prediction unit 244 and intra-frame prediction unit 254) performed by the exemplary video encoder 20 are described in detail below.

[0121] segmentation Segmentation unit 262 can divide (or partition) the current block 203 into smaller partitions, such as smaller blocks the size of a square or rectangle. These smaller blocks (also called sub-blocks) can be further divided into even smaller partitions. This is also called tree segmentation or hierarchical tree segmentation. Root blocks at root level 0 (level 0, depth 0), etc., can be recursively segmented into two or more blocks at the next lower tree level, such as nodes at tree level 1 (level 1, depth 1). These blocks can then be further segmented into two or more blocks at the next lower level, such as tree level 2 (level 2, depth 2), etc., until the segmentation ends (because the termination criterion is met, such as reaching the maximum tree depth or minimum block size). Blocks that are not further segmented are also called leaf blocks or leaf nodes of the tree. A tree segmented into two partitions is called a binary-tree (BT), a tree segmented into three partitions is called a ternary-tree (TT), and a tree segmented into four partitions is called a quad-tree (QT).

[0122] As mentioned above, the term "block" as used in this paper can be a portion of an image, particularly a square or rectangular portion. Referring to HEVC and VVC, a block can be or may correspond to a coding tree unit (CTU), coding unit (CU), prediction unit (PU), and transform unit (TU), and / or correspond to multiple corresponding blocks, such as a coding tree block (CTB), coding block (CB), transform block (TB), or prediction block (PB).

[0123] For example, a coding tree unit (CTU) can be or may include one CTB consisting of luminance samples from an image with three sample arrays and two corresponding CTBs consisting of chrominance samples from the same image. Alternatively, it can be or may include one CTB consisting of samples from a black and white image or an image decoded using three separate color planes and syntax structures. These syntax structures are used to decode the aforementioned samples. Correspondingly, a coding tree block (CTB) can be an N×N sample block, where N can be set to a value such that a component is divided into multiple CTBs; this is a segmentation method. A coding unit (CU) can be or may include one coding block consisting of luminance samples from an image with three sample arrays and two corresponding coding blocks consisting of chrominance samples from the same image. Alternatively, it can be or may include one coding block consisting of samples from a black and white image or an image decoded using three separate color planes and syntax structures. These syntax structures are used to decode the aforementioned samples. Correspondingly, a coding block (CB) can be an M×N sample block, where M and N can be set to a certain value so that a CTB is divided into multiple coding blocks, which is one kind of segmentation method.

[0124] In an embodiment, for example according to HEVC, a coding tree unit (CTU) can be divided into multiple CUs by a quadtree structure represented as a coding tree. At the CU level, it is determined whether to use inter-frame (temporal) prediction or intra-frame (spatial) prediction to decode the image region. Each CU can be further divided into one, two, or four PUs based on the PU partitioning type. The same prediction process is performed within a PU, and relevant information is sent to the decoder on a PU-by-PU basis. After obtaining residual blocks through the prediction process according to the PU partitioning type, the CU can be further divided into transform units (TUs) according to other quadtree structures similar to the coding tree of the CU.

[0125] In embodiments, for example, according to the latest video coding standard currently being developed, known as Versatile Video Coding (VVC), coded blocks are segmented using a combination of quad-tree and binary-tree (QTBT) segmentation. In a QTBT block structure, a CU can be a square or a rectangle. For example, a coding tree unit (CTU) is first segmented using a quad-tree structure. The leaf nodes of the quad-tree are further segmented using binary or ternary / triple tree structures. The segmented leaf nodes are called coding units (CUs), and this segmentation is used for prediction and transform processing without any further segmentation. This means that CUs, PUs, and TUs have the same block size in the QTBT coded block structure. Meanwhile, multiple segmentations, such as ternary tree segmentation, can be used with the QTBT block structure.

[0126] In one example, the mode selection unit 260 in the video encoder 20 can be used to perform any combination of the segmentation techniques described herein.

[0127] As described above, the video encoder 20 is used to determine or select the best or optimal prediction mode from (e.g., a predetermined) set of prediction modes. The set of prediction modes may include intra-frame prediction modes and / or inter-frame prediction modes, etc.

[0128] Intra-frame prediction The intra-prediction mode set can include 35 different intra-prediction modes, such as non-directional or directional modes like DC (or mean) mode and planar mode defined in HEVC, or it can include 67 different intra-prediction modes, such as non-directional or directional modes like DC (or mean) mode and planar mode defined in VVC.

[0129] Intra-prediction unit 254 is used to generate intra-prediction block 265 using reconstructed samples of adjacent blocks of the same current image based on intra-prediction modes in the intra-prediction mode set.

[0130] Intra-prediction unit 254 (or commonly referred to as mode selection unit 260) is also used to output intra-prediction parameters (or commonly referred to as information representing the selected intra-prediction mode of the block) to entropy coding unit 270 in the form of syntax element 266 to be included in the encoded image data 21 so that video decoder 30 can receive and use the prediction parameters for decoding, etc.

[0131] Inter-frame prediction The set of (possible) inter-frame prediction modes depends on the available reference image (i.e., at least a portion of the decoded image stored in the DPB230 as described above) and other inter-frame prediction parameters, such as whether the entire reference image or only a portion of the reference image (e.g., the search window region around the current block) is used to search for the best matching reference block, and / or, for example, whether pixel interpolation is performed, such as half-pixel interpolation and / or quarter-pixel interpolation.

[0132] In addition to the prediction modes mentioned above, skip mode and / or direct mode can also be used.

[0133] Inter-frame prediction unit 244 may include a motion estimation (ME) unit and a motion compensation (MC) unit (both in... Figure 2 (Not shown in the image). The motion estimation unit can be used to receive or acquire image block 203 (current image block 203 of current image 17) and decoded image 231, or at least one or more previous reconstructed blocks (e.g., reconstructed blocks of one or more other / different previous decoded images 231) for motion estimation. For example, a video sequence may include the current image and the previous decoded image 231, or in other words, the current image and the previous decoded image 231 may be part of or constitute a series of images that form a video sequence.

[0134] For example, encoder 20 can be used to select a reference block from multiple reference blocks of the same or different images in multiple other images, and provide the offset (spatial offset) between the position (x-coordinate, y-coordinate) of the reference image (or reference image index) and / or the position of the reference block and the position of the current block as an inter-frame prediction parameter to the motion estimation unit. This offset is also called a motion vector (MV).

[0135] The motion compensation unit is used to acquire (e.g., receive) inter-frame prediction parameters and perform inter-frame prediction based on or using the inter-frame prediction parameters to obtain inter-frame prediction blocks 265. Motion compensation performed by the motion compensation unit may include extracting or generating prediction blocks based on motion / block vectors determined by motion estimation, and may also include performing interpolation to obtain sub-pixel precision. Interpolation filtering can generate other pixel samples based on known pixel samples, potentially increasing the number of candidate prediction blocks that can be used to decode image blocks. Once the motion vector corresponding to the PU of the current image block is received, the motion compensation unit can locate the prediction block pointed to by the motion vector in one of the reference image lists.

[0136] The motion compensation unit can also generate syntax elements associated with blocks and video stripes for use by the video decoder 30 when decoding image blocks of the video stripes. In addition to or as a substitute for stripes and corresponding syntax elements, chunk groups and / or chunks and their corresponding syntax elements can be generated or used.

[0137] Entropy coding Entropy coding unit 270 is used to apply or not apply entropy coding algorithms or schemes (such as variable length coding (VLC), context adaptive VLC (CAVLC), arithmetic coding schemes, binarization, context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding methods or techniques) to (uncompressed) quantized coefficients 209, inter-frame prediction parameters, intra-frame prediction parameters, loop filter parameters, and / or other syntax elements to obtain encoded image data 21 that can be output through output terminal 272 in the form of encoded bitstream 21, etc., so that video decoder 30 can receive and use these parameters for decoding, etc. The encoded bitstream 21 can be sent to video decoder 30, or stored in memory for later transmission or retrieval by video decoder 30.

[0138] Other structural variations of the video encoder 20 can be used to encode video streams. For example, a non-transform-based encoder 20 can directly quantize residual signals for certain blocks or frames without a transform processing unit 206. In another implementation, the encoder 20 may include a quantization unit 208 and an inverse quantization unit 210 combined into a single unit.

[0139] Decoder and Decoding Method Figure 3 This is an example of a video decoder 30 used to implement the technology in this application. The video decoder 30 is used to receive, for example, encoded image data 21 (e.g., encoded bitstream 21) encoded by encoder 20 to obtain a decoded image 331. The encoded image data or bitstream includes information for decoding the encoded image data, such as data and associated syntax elements representing image blocks of encoded video stripes (and / or chunks or blocks).

[0140] exist Figure 3 In the example, decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., a summer 314), a loop filter 320, a decoded picture buffer (DBP) 330, a mode application unit 360, an inter-frame prediction unit 344, and an intra-frame prediction unit 354. The inter-frame prediction unit 344 may be or may include a motion compensation unit. In some examples, video decoder 30 may perform substantially the same functions as the reference unit. Figure 2 The video encoder 100 described in the text is the inverse of the encoding process and the decoding process.

[0141] As described with reference to encoder 20, the inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, loop filter 220, decoded picture buffer (DPB) 230, inter-frame prediction unit 344, and intra-frame prediction unit 354 also constitute the "built-in decoder" of video encoder 20. Accordingly, inverse quantization unit 310 can be functionally identical to inverse quantization unit 110, inverse transform processing unit 312 can be functionally identical to inverse transform processing unit 212, reconstruction unit 314 can be functionally identical to reconstruction unit 214, loop filter 320 can be functionally identical to loop filter 220, and decoded picture buffer 330 can be functionally identical to decoded picture buffer 230. Therefore, the explanation of the corresponding units and functions of video encoder 20 is correspondingly applicable to the corresponding units and functions of video decoder 30.

[0142] Entropy Decoding Entropy decoding unit 304 is used to parse bitstream 21 (or commonly referred to as encoded image data 21) and perform entropy decoding on encoded image data 21 to obtain quantization coefficients 309 and / or decoded encoding parameters. Figure 3(Not shown in the image) , such as inter-frame prediction parameters (e.g., reference image index and motion vector), intra-frame prediction parameters (e.g., intra-frame prediction mode or index), transform parameters, quantization parameters, loop filter parameters, and / or other syntax elements, etc. The entropy decoding unit 304 can be used to apply a decoding algorithm or scheme corresponding to the encoding scheme described by the entropy coding unit 270 in the reference encoder 20. The entropy decoding unit 304 can also be used to provide inter-frame prediction parameters, intra-frame prediction parameters, and / or other syntax elements to the mode application unit 360, and to provide other parameters to other units in the decoder 30. The video decoder 30 can receive video strip-level and / or video block-level syntax elements. In addition to stripes and corresponding syntax elements, or as a substitute for stripes and corresponding syntax elements, it can also receive and / or use chunk groups and / or chunks and corresponding syntax elements.

[0143] Inverse Quantization The dequantization unit 310 can be used to receive quantization parameters (QP) (or commonly referred to as dequantization-related information) and quantization coefficients from encoded image data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304, etc.), and dequantize the decoded quantization coefficients 309 according to these quantization parameters to obtain dequantized coefficients 311. Dequantized coefficients 311 can also be referred to as transform coefficients 311. The dequantization process may include determining the degree of quantization using the quantization parameters determined by the video encoder 20 for each video block in a video strip (or chunk or group of chunks), and also determining the degree of dequantization to be applied.

[0144] Inverse Transformation The inverse transform processing unit 312 can be used to receive the dequantized coefficients 311 (also referred to as transform coefficients 311) and transform the dequantized coefficients 311 to obtain the reconstructed residual block 213 in the sample domain. The reconstructed residual block 213 can also be referred to as transform block 313. The transform can be an inverse transform, such as inverse DCT, inverse DST, inverse integer transform, or a conceptually similar inverse transform process. The inverse transform processing unit 312 can also be used (e.g., parsed and / or decoded by the entropy decoding unit 304, etc.) to receive transform parameters or corresponding information from the encoded image data 21 to determine the transform to be applied to the dequantized coefficients 311.

[0145] reconstruction Reconstruction unit 314 (e.g., adder or summer 314) can be used to add reconstructed residual block 313 to prediction block 365 to obtain reconstructed block 315 in the sample domain by, for example, adding the sample values ​​of reconstructed residual block 313 and the sample values ​​of prediction block 365.

[0146] Filtering Loop filter unit 320 (in or after the decoding loop) is used to filter the reconstructed block 315 to obtain the filtered block 321, thereby facilitating pixel transformation or otherwise improving video quality. Loop filter unit 320 may include one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, a co-filter, or any combination thereof. Although loop filter unit 320 is in... Figure 3 The loop filter unit 320 is shown as an in-loop filter, but in other configurations, it can be implemented as a post-loop filter.

[0147] Decoding image buffer The decoded video block 321 from one image is then stored in the decoded image buffer 330. The decoded image buffer 330 stores the decoded image 331 as a reference image so that motion compensation and / or output or display can be performed on other images later.

[0148] The decoder 30 is used to output the decoded image 331 through the output terminal 332, etc., to present to the user or for the user to view.

[0149] predict Inter-frame prediction unit 344 may be functionally identical to inter-frame prediction unit 244 (particularly to motion compensation unit), and intra-frame prediction unit 354 may be functionally identical to intra-frame prediction unit 254. Both units perform segmentation or partitioning decisions and execute predictions based on (e.g., parsed and / or decoded by entropy decoding unit 304, etc.) the segmentation method and / or prediction parameters or corresponding information received from the encoded image data 21. Pattern application unit 360 may be used to perform block-by-block predictions (intra-frame or inter-frame predictions) based on the reconstructed image, blocks, or corresponding samples (filtered or unfiltered) to obtain prediction blocks 365.

[0150] When a video strip is decoded into an intra-decoded (I) strip, the intra-prediction unit 354 in the mode application unit 360 generates a prediction block 365 for the image blocks of the current video strip based on the signaled intra-prediction mode and data from previously decoded blocks of the current image. When a video image is decoded into an inter-decoded (i.e., B or P) strip, the inter-prediction unit 344 (e.g., a motion compensation unit) in the mode application unit 360 generates a prediction block 365 for video blocks in the current video strip based on motion vectors and other syntax elements received from the entropy decoding unit 304. For inter-prediction, these prediction blocks can be generated based on one of the reference images in one of the reference image lists. The video decoder 30 can construct reference frame lists 0 and 1 using a default construction technique based on the reference images stored in the DPB 330. In addition to or as an alternative to stripes (e.g., video stripes), the same or similar processes can be applied to or by embodiments that use chunk groups (e.g., video chunk groups) and / or chunks (e.g., video chunks), for example, video can be decoded using I, P, or B chunk groups and / or chunks.

[0151] The pattern application unit 360 is used to determine prediction information for video blocks in the current video strip by parsing motion vectors or related information and other syntax elements, and uses the prediction information to generate prediction blocks for the current video block being decoded. For example, the pattern application unit 360 uses some received syntax elements to determine the prediction mode (e.g., intra-frame or inter-frame prediction) for decoding video blocks in the video strip, the inter-frame prediction stripe type (e.g., B stripe, P stripe, or GPB stripe), the construction information for one or more reference image lists for the stripe, the motion vector for each inter-frame coded video block of the stripe, the inter-frame prediction state for each inter-frame decoded video block of the stripe, and other information to decode video blocks in the current video stripe. In addition to or as an alternative to stripes (e.g., video stripes), the same or similar process can be applied to or applied by embodiments using chunk groups (e.g., video chunk groups) and / or chunks (e.g., video chunks), for example, video can be decoded using I, P, or B chunk groups and / or chunks.

[0152] Figure 3 The embodiment of the video decoder 30 shown can be used to segment and / or decode an image by stripes (also referred to as video stripes). An image can be segmented into one or more (typically non-overlapping) stripes or decoded by one or more (typically non-overlapping) stripes, each stripe may include one or more blocks (e.g., CTUs).

[0153] Figure 3The embodiment of the video decoder 30 shown can be used to segment and / or decode an image by chunk groups (also referred to as video chunk groups) and / or chunks (also referred to as video chunks). An image can be segmented into one or more (typically non-overlapping) chunk groups or decoded by one or more (typically non-overlapping) chunk groups; each chunk group may include one or more blocks (e.g., CTUs) or one or more chunks, etc.; each chunk may be a rectangle, etc., and may include one or more complete or partial blocks (e.g., CTUs, etc.).

[0154] Other variations of the video decoder 30 can be used to decode the encoded image data 21. For example, the decoder 30 can generate an output video stream without the loop filter unit 320. For example, the non-transform-based decoder 30 can directly dequantize the residual signal for certain blocks or frames without the inverse transform processing unit 312. In another implementation, the video decoder 30 may include a dequantization unit 310 and an inverse transform processing unit 312 combined into a single unit.

[0155] It should be understood that in encoder 20 and decoder 30, the processing result of the current step can be further processed and then output to the next step. For example, after interpolation filtering, motion vector derivation, or loop filtering, the processing result of interpolation filtering, motion vector derivation, or loop filtering can be further calculated, such as clipping or shifting.

[0156] It should be noted that further calculations can be performed on the derived motion vector of the current block (including but not limited to the control point motion vector in affine mode, the sub-block motion vectors in affine mode, planar mode, and ATMVP mode, and the time motion vector). For example, the value of the motion vector can be restricted to a predefined range based on the representation bits of the motion vector. If the representation bits of the motion vector are bitDepth, then the range is... ,in," The exponentiation is ".". For example, if bitDepth is set to 16, the range is -32768 to 32767; if bitDepth is set to 18, the range is -131072 to 131071. For instance, the derived motion vector (e.g., the MV of four 4×4 sub-blocks in an 8×8 block) is constrained such that the maximum difference between the integer parts of the MVs of these four 4×4 sub-blocks does not exceed N pixels, for example, not more than 1 pixel. Two methods for constraining motion vectors based on bitDepth are provided here.

[0157] Method 1: Remove the most significant bit (MSB) of the overflow through smoothing operations. (1).

[0158] (2).

[0159] (3).

[0160] (4).

[0161] in, The horizontal component of the motion vector of an image block or sub-block; The vertical component of the motion vector of an image block or sub-block; and This represents the intermediate value.

[0162] For example, if The value is -32769, so the value obtained after using formulas (1) and (2) is 32767. In computer systems, decimal numbers are stored in two's complement form. The two's complement of -32769 is 1,0111,1111,1111,1111 (17 bits). If we discard the MSB, the resulting two's complement is 0111,1111,1111,1111 (decimal 32767), which is the same as the output obtained after using formulas (1) and (2).

[0163] (5).

[0164] (6).

[0165] (7).

[0166] (8).

[0167] These operations can be performed during the summation of MVP and MVD, as shown in Equations (5) to (8).

[0168] Method 2: Remove overflowing MSBs by limiting the value.

[0169]

[0170] Where vx is the horizontal component of the motion vector of an image block or sub-block, vy is the vertical component of the motion vector of an image block or sub-block, and x, y, and z correspond to the three input values ​​of the MV clipping process, respectively. The function Clip3 is defined as follows:

[0171] Figure 4This is a schematic diagram of a video decoding device 400 provided according to one embodiment of the present invention. The video decoding device 400 is suitable for implementing the disclosed embodiments described herein. In one embodiment, the video decoding device 400 may be a decoder (e.g., Figure 1A The video decoder 30 or encoder (e.g.) in the video decoder 30 Figure 1A (Video encoder 20 in the middle).

[0172] The video decoding device 400 includes an input port 410 and a receiving unit (Rx) 420 for receiving data, a processor, logic unit, or central processing unit (CPU) 430 for processing the data, a transmitting unit (Tx) 440 and an output port 450 for transmitting the data, and a memory 460 for storing the data. The video decoding device 400 may also include optical-to-electrical (OE) components and electro-optical (EO) components coupled to the input port 410, the receiving unit 420, the transmitting unit 440, and the output port 450, serving as outputs or inlets for optical or electrical signals.

[0173] Processor 430 is implemented through hardware and software. Processor 430 can be implemented as one or more CPU chips, one or more cores (e.g., a multi-core processor), one or more FPGAs, one or more ASICs, and one or more DSPs. Processor 430 communicates with ingress port 410, receiver unit 420, transmitter unit 440, egress port 450, and memory 460. Processor 430 includes decoding module 470. Decoding module 470 implements the disclosed embodiments described above. For example, decoding module 470 performs, processes, prepares, or provides various decoding operations. Therefore, including decoding module 470 provides a substantial improvement to the functionality of video decoding device 400 and affects the transitions of video decoding device 400 to different states. Optionally, decoding module 470 is implemented with instructions stored in memory 460 and executed by processor 430.

[0174] Memory 460 may include one or more disks, one or more tape drives, and one or more solid-state drives, and may be used as an overflow data storage device to store programs as selected for execution, as well as instructions and data read during program execution. For example, memory 460 may be volatile and / or non-volatile, and may be read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and / or static random-access memory (SRAM).

[0175] Figure 5 A simplified block diagram of an apparatus 500 provided for an exemplary embodiment. The apparatus 500 can be used as the source device 12 and / or the destination device 14 in FIG1.

[0176] The processor 502 in device 500 may be a central processing unit. Alternatively, the processor 502 may be any other type of device or multiple devices, existing or to be developed in the future, capable of operating or processing information. While the disclosed implementation may be carried out using a single processor such as the processor 502 shown in the figure, using multiple processors can improve speed and efficiency.

[0177] In one implementation, the memory 504 in device 500 may be a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may be used as memory 504. Memory 504 may include code and data 506 accessed by processor 502 via bus 512. Memory 504 may also include an operating system 508 and an application program 510, which includes at least one program that causes processor 502 to perform the methods described herein. For example, application program 510 may include application 1 to application N, and may also include a video decoding application that performs the methods described herein.

[0178] The device 500 may also include one or more output devices, such as a display 518. In one example, the display 518 may be a touch-sensitive display that combines a display with a touch-sensitive element capable of sensing touch input. The display 518 may be coupled to the processor 502 via a bus 512.

[0179] Although bus 512 in device 500 is described herein as a single bus, bus 512 may include multiple buses. Furthermore, auxiliary memory 514 may be directly coupled to other components in device 500 or accessible via a network, and may include a single integrated unit (e.g., a memory card) or multiple units (e.g., multiple memory cards). Therefore, device 500 can be implemented in a variety of configurations.

[0180] Parameter Set The parameter sets are largely similar and share the same fundamental design goals: increasing bit rate, enhancing error resilience, and providing a system-level interface. The parameter sets in HEVC (H.265) are hierarchical, including the Video Parameter Set (VPS), Sequence Parameter Set (SPS), and Picture Parameter Set (PPS), similar to their counterparts in AVC and VVC. Each slice references an active PPS, SPS, and VPS to access information used for decoding that slice. The PPS contains information applicable to all slices in a picture; therefore, all slices in a picture must reference the same PPS. Slices in different pictures can also reference the same PPS. Similarly, the SPS contains information applicable to all pictures in the same coded video sequence.

[0181] Although different images may correspond to different PPSs, many or all images in an encoded video sequence typically reference the same PPS. Reusing parameter sets increases the bitrate because it avoids sending shared information multiple times. Furthermore, reusing parameter sets provides robustness because it allows parameter set content to be carried over more reliable external communication links or to be frequently repeated within the bitstream to ensure it is not lost.

[0182] Scalable video coding, layers, and Video Parameter Sets (VPS) Adaptive video coding provides a mechanism for decoding video across multiple layers, each representing a quality representation of the same video scene. The base layer (BL) is the lowest quality representation. One or more enhancement layers (EL) can be decoded by referencing the lower layers, providing high video quality. Decoding a subset of each layer of an adaptively decoded video stream results in lower but still acceptable quality video. This allows for a more graceful degradation compared to inadaptive video streams, where a lower bitrate typically leads to a further decrease in video quality.

[0183] There are several types of adaptability in adaptable video sequences, including temporal adaptability, spatial adaptability, and quality adaptability. Figure 11 An example of spatial adaptability and temporal adaptability is shown. Figure 11 In this approach, the two layers are decoded at different resolutions. BL has a lower resolution, while EL has a higher resolution. Spatial adaptability is achieved by providing a decoder to decode both BL and / or EL.

[0184] In addition to spatial adaptability, temporal adaptability is also implemented within the coding layer. In this example, each coding layer is divided into two temporal sublayers, labeled with time ID 0 and ID 1, respectively. Temporal adaptability is achieved by providing a decoder to decode temporal sublayer 0 (time ID equals 0) or sublayers 0 and 1.

[0185] Each layer of images is assigned a layer ID, such as the syntax element nuh_layer_id. A coded layer video sequence (CLVS) is a series of images with the same nuh_layer_id value. These images, in the decoding order, include a special coding layer video sequence starting (CLVSS) coded picture (e.g., an intra-frame image), followed by zero or more images that are not CLVSS images. These images include all subsequent images, up to but not including any subsequent images that are CLVSS images.

[0186] A coded video sequence (CVS) is one of multiple coded layer video sequences (CLVS). Figure 11 In the example, assuming the first image in BL and EL is a CLVSS image, while all other images are not CLVSS images, then the CVS includes two CLVS.

[0187] Sequence Parameter Set (SPS) SPS comprises parameters applied to a layer of the encoded video sequence and does not change as the images change within the encoded video sequence.

[0188] In some extreme cases, any image in CLVS may not use SPS.

[0189] It is also possible for different CLVS to share an SPS.

[0190] In the latest VVC specification draft (i.e. http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 19_Teleconference / wg11 / JVET-S0152-v5.zip For simplicity, hereinafter referred to as the VVC draft, the definition of SPS is as follows: Sequence parameter set (SPS): A syntax structure that includes syntax elements applied to zero or more complete CLVSs, which are determined by the contents of syntax elements in the PPS, which is referenced by syntax elements in each image header.

[0191] Please refer to the VVC draft for the definitions of PPS and image header.

[0192] Specifically, SPS includes information about the decoded image buffer (i.e., DPB).

[0193] Some portions of the table below show snapshots of a portion of the DPB indicated in the SPS within the VVC.

[0194] Indicates (signal) information in the decoded picture buffer (DPB). Decoding image buffer A Decoded Picture Buffer (DPB) is a buffer used to store decoded images for reference (e.g., as reference images for inter-frame prediction). An example published in the VVC draft highlights the relevant syntax elements for the DBP parameters in the sequence parameter set (SPS).

[0195] Table 1: Syntax Elements Related to Decoding Images in SPS

[0196] A value of 1 for `sps_ptl_dpb_hrd_params_present_flag` indicates that the syntax structures `profile_tier_level()` and `dpb_parameters()` exist in SPS. Furthermore, SPS may also contain the syntax structures `general_hrd_parameters()` and `ols_hrd_parameters()`. A value of 0 for `sps_ptl_dpb_hrd_params_present_flag` indicates that these four syntax structures do not exist in SPS.

[0197] The value of sps_ptl_dpb_hrd_params_present_flag must be equal to 1 when sps_video_parameter_set_id is greater than 0 and there exists an OLS that includes only one layer whose nuh_layer_id is equal to the nuh_layer_id of the SPS, or when sps_video_parameter_set_id is equal to 0.

[0198] An output layer set (OLS) is a group of layers, one or more of which are designated as output layers. An output layer is one layer in the output layer set that is being output.

[0199] The syntax tables for the syntax structures profile_tier_level(), dpb_parameters(), general_hrd_parameters(), and ols_hrd_parameters() can be found in the VVC draft.

[0200] The syntax element `sps_max_sublayers_minus1` represents the number of available time sublayers. When the number of available time sublayers is greater than 1 (e.g., the value of the syntax element `sps_max_sublayers_minus1` is greater than 0), the value of the syntax element `sps_sublayer_dpb_params_flag` is indicated in the bitstream. The syntax element `sps_sublayer_dpb_params_flag` indicates whether decoded image information is indicated for each available sublayer (when the value of this syntax element equals 1) or only for the highest time sublayer (when the value of this syntax element equals 0). When the value of `sps_sublayer_dpb_params_flag` is not present, for example, when there is only one time sublayer (the value of `sps_max_sublayers_minus1` == 0), the value of `sps_max_sublayers_minus1` is inferred to be 0.

[0201] In some examples, the value of the syntax element `sps_ptl_dpb_hrd_params_present_flag` indicates whether the indicator structure `dpb_parameter()` is indicated in SPS. When the value of the syntax element `sps_ptl_dpb_hrd_params_present_flag` is equal to 1, the indicated data structure `dpb_parameter()` is invoked, where the number of available time sublayers minus 1 (`sps_max_sublayers_minus1`) and the flag `sps_sublayer_dpb_params_flag` are used as the first and second parameters, respectively.

[0202] In one example, the indicator structure dpb_parameter() in the VVC draft is defined as follows: Table 2: Definition of syntax structure dpb_parameters

[0203] The syntax structure dpb_parameters() provides information on the DPB size, maximum image reordering number, and maximum delay for one or more OLS.

[0204] When the VPS includes the syntax structure dpb_parameters(), the OLS to which the syntax structure dpb_parameters() applies is specified by the VPS. When the SPS includes the syntax structure dpb_parameters(), the syntax structure applies only to the OLS that includes the lowest layer of all layers referencing the SPS, and that lowest layer is an independent layer.

[0205] `max_dec_pic_buffering_minus1[i] + 1` represents the maximum required size of the DPB, in units of the image storage buffer, when `Htid` equals `i`. The value of `max_dec_pic_buffering_minus1[i]` should be up to `MaxDpbSize - 1` (inclusive), where `MaxDpbSize` is as described in Section A.4.2. When `i` is greater than 0, `max_dec_pic_buffering_minus1[i]` must be greater than or equal to `max_dec_pic_buffering_minus1[i - 1]`. When max_dec_pic_buffering_minus1[i] (where i ranges from 0 to maxSubLayersMinus1 – 1, inclusive) does not exist, max_dec_pic_buffering_minus1[i] is inferred to be max_dec_pic_buffering_minus1[maxSubLayersMinus1] since subLayerInfoFlag is equal to 0.

[0206] `max_num_reorder_pics[i]` represents the maximum number of images that can be decoded before any image in the OLS and whose output order follows that image when `Htid` equals `i`. The value of `max_num_reorder_pics[i]` should range to `max_dec_pic_buffering_minus1[i]` (inclusive). When `i` is greater than 0, `max_num_reorder_pics[i]` must be greater than or equal to `max_num_reorder_pics[i – 1]`. When `max_num_reorder_pics[i]` (where `i` ranges from 0 to `maxSubLayersMinus1 – 1`, inclusive) does not exist, since `subLayerInfoFlag` equals 0, `max_num_reorder_pics[i]` is inferred to be `max_num_reorder_pics[maxSubLayersMinus1]`.

[0207] The value of mmax_latency_increase_plus1[i] is not equal to 0 and is used to calculate the value of MaxLatencyPictures[i]. The value of MaxLatencyPictures[i] represents the maximum number of images in the OLS whose output order can precede any image in the OLS and whose decoding order can follow any image when Htid equals i.

[0208] When max_latency_increase_plus1[i] is not equal to 0, the value of MaxLatencyPictures[i] is represented as follows: MaxLatencyPictures[ i ] = max_num_reorder_pics[ i ]+ max_latency_increase_plus1[ i ] – 1 (112) When max_latency_increase_plus1[i] equals 0, it does not represent the corresponding limit.

[0209] The range of max_latency_increase_plus1[i] should be up to (Including end values). When max_latency_increase_plus1[i] (where i ranges from 0 to maxSubLayersMinus1 – 1, including end values) does not exist, max_latency_increase_plus1[i] is inferred to be max_latency_increase_plus1[maxSubLayersMinus1] since subLayerInfoFlag is equal to 0.

[0210] For more details on the syntax elements and variables (max_dec_pic_buffering_minus1[i], max_num_reorder_pics[i], and max_latency_increase_plus1[i]) used to interpret the DPB parameters, please refer to the VVC draft.

[0211] The `dpb_parameters` structure indicates the decoded image buffer information for a single sublayer or the decoded image information for each sublayer, controlled by the value of `subLayerInfoFlag`. When `subLayerInfoFlag` is 0, it indicates the decoded image buffer information for the highest-level sublayer in the available time sublayers (i = maxSubLayersMinus1). When `subLayerInfoFlag` is 1, it indicates the decoded image buffer information for each sublayer in the available time sublayers (i ranges from 0 to maxSubLayersMinus1, inclusive).

[0212] Semantic issues with the syntax element sps_sublayer_dpb_params_flag In the VVC draft, the semantic definition of sps_sublayer_dpb_params_flag is as follows: `sps_sublayer_dpb_params_flag` controls whether the syntax elements `max_dec_pic_buffering_minus1[i]`, `max_num_reorder_pics[i]`, and `max_latency_increase_plus1[i]` exist in the syntax structure `dpb_parameters()` in SPS. When `sps_sub_dpb_params_info_present_flag` does not exist, its value is inferred to be 0.

[0213] There are two problems with the above semantics. First, there is a typo in the last sentence.

[0214] When sps_sub_dpb_params_info_present_flag does not exist, the value of sps_sub_dpb_params_info_present_flag is inferred to be 0.

[0215] The `sps_sub_dpb_params_info_present_flag` is not defined elsewhere, so the last line must be modified as follows: When sps_sublayer_dpb_params_flag does not exist, the value of sps_sublayer_dpb_params_flag is inferred to be 0.

[0216] Second, because the range of i in max_dec_pic_buffering_minus1[i], max_num_reorder_pics[i], and max_latency_increase_plus1[i] is undefined, the semantics of sps_sublayer_dpb_params_flag are not accurate enough. As can be seen from Table 2, when i equals maxSubLayersMinus1 (corresponding to sps_max_sublayers_minus1 in Table 1), the syntax elements max_dec_pic_buffering_minus1[i], max_num_reorder_pics[i], and max_latency_increase_plus1[i] are always indicated, regardless of whether sps_sublayer_dpb_params_flag is equal to 0 or 1. In this case, sps_sublayer_dpb_params_flag does not control whether max_dec_pic_buffering_minus1[i], max_num_reorder_pics[i], and max_latency_increase_plus1[i] exist, which contradicts the current definition.

[0217] In some examples, the syntax element sps_sublayer_dpb_params_flag is used only as the second parameter of dpb_parameters.

[0218] For the following embodiments, it is assumed that the typo (the first problem) has been corrected, and the proposed embodiments focus on solving the second problem, namely, the semantic inaccuracy of sps_sublayer_dpb_params_flag.

[0219] Example 1 According to the first embodiment, the semantic modification of sps_sublayer_dpb_params_flag is as follows: sps_sublayer_dpb_params_flag is used to control whether the syntax elements max_dec_pic_buffering_minus1[i], max_num_reorder_pics[i], and max_latency_increase_plus1[i] exist in the syntax structure dpb_parameters() in SPS. When sps_max_sublayers_minus1 is greater than 0, i ranges from 0 to sps_max_sublayers_minus1 – 1 (inclusive). When sps_max_sublayers_minus1 is equal to 0, the value of sps_sublayer_dpb_params_flag is inferred to be 0, while max_dec_pic_buffering_minus1[0], max_num_reorder_pics[0], and max_latency_increase_plus1[0] are always unique sublayer indicators of the reference SPS.

[0220] Thus, the semantics of sps_sublayer_dpb_params_flag become clearer. Only when multiple sublayers exist (sps_max_sublayers_minus1 is greater than 0) can sps_sublayer_dpb_params_flag control whether minus1[i], max_num_reorder_pics[i], and max_latency_increase_plus1[i] exist in the syntax structure dpb_parameters(), where i ranges from 0 to sps_max_sublayers_minus1 – 1 (inclusive). Otherwise (sps_max_sublayers_minus1 equals 0), max_dec_pic_buffering_minus1[0], max_num_reorder_pics[0], and max_latency_increase_plus1[0] are always indicated, regardless of the value of sps_sublayer_dpb_params_flag. Therefore, the semantics may also be changed in Embodiment 2 as follows.

[0221] Example 2 sps_sublayer_dpb_params_flag is used to control whether the syntax elements max_dec_pic_buffering_minus1[i], max_num_reorder_pics[i], and max_latency_increase_plus1[i] exist in the syntax structure dpb_parameters() in SPS. When sps_max_sublayers_minus1 is greater than 0, the range of i is from 0 to sps_max_sublayers_minus1 – 1 (inclusive). When sps_max_sublayers_minus1 is equal to 0, the value of sps_sublayer_dpb_params_flag is inferred to be 1, and max_dec_pic_buffering_minus1[0], max_num_reorder_pics[0], and max_latency_increase_plus1[0] are always unique sublayer indicators of the reference SPS.

[0222] Optionally, the inference rule for the value of sps_sublayer_dpb_params_flag can even be removed, as shown in Example 3.

[0223] Example 3 The `sps_sublayer_dpb_params_flag` is used to control whether the syntax elements `max_dec_pic_buffering_minus1[i]`, `max_num_reorder_pics[i]`, and `max_latency_increase_plus1[i]` exist in the syntax structure `dpb_parameters()` in the SPS. When `sps_max_sublayers_minus1` is greater than 0, the range of `i` is from 0 to `sps_max_sublayers_minus1 - 1` (inclusive). When `sps_max_sublayers_minus1` is equal to 0, `max_dec_pic_buffering_minus1[0]`, `max_num_reorder_pics[0]`, and `max_latency_increase_plus1[0]` are always unique sublayer indicators of the reference SPS.

[0224] Alternatively, there are other interpretations when sps_max_sublayers_minus1 equals 0, as shown in Example 4.

[0225] Example 4 The `sps_sublayer_dpb_params_flag` is used to control whether the syntax elements `max_dec_pic_buffering_minus1[i]`, `max_num_reorder_pics[i]`, and `max_latency_increase_plus1[i]` exist in the syntax structure `dpb_parameters()` in the SPS. When `sps_max_sublayers_minus1` is greater than 0, the range of `i` is from 0 to `sps_max_sublayers_minus1 - 1` (inclusive). When `sps_max_sublayers_minus1` is equal to 0, the value of `sps_sublayer_dpb_params_flag` is meaningless, while `max_dec_pic_buffering_minus1[0]`, `max_num_reorder_pics[0]`, and `max_latency_increase_plus1[0]` are always unique sublayer indicators of the reference SPS.

[0226] The data structure dpb_parameter() is called not only in SPS but also in VPS. Below is a snapshot of a VPS that calls dpb_parameter().

[0227] Table 3: Syntax Structures in VPS (dpb_parameters)

[0228] This indicates the number of dpb_parameters() syntax structures in the VPS.

[0229] A value of 1 for `vps_default_ptl_dpb_hrd_max_tid_flag` indicates that the syntax elements `vps_ptl_max_tid[i]`, `vps_dpb_max_tid[i]`, and `vps_hrd_max_tid[i]` do not exist and are inferred to be the default value `vps_max_sublayers_minus1`. A value of 0 for `vps_default_ptl_dpb_hrd_max_tid_flag` indicates that the syntax elements `vps_ptl_max_tid[i]`, `vps_dpb_max_tid[i]`, and `vps_ptl_max_tid[i]` exist. When `vps_default_ptl_dpb_hrd_max_tid_flag` does not exist, its value is inferred to be 1.

[0230] For a detailed explanation of vps_ptl_max_tid[i] and vps_hrd_max_tid[i], please refer to the VVC draft.

[0231] `vps_dpb_max_tid[i]` represents the TemporalId of the highest sublayer representation of DPB parameters that may exist in the `dpb_parameters()` syntax structure of the VPS. The value of `vps_dpb_max_tid[i]` should be within the range of `vps_max_sublayers_minus1` (inclusive). When `vps_dpb_max_tid[i]` does not exist, the value of `vps_dpb_max_tid[i]` is inferred to be `vps_max_sublayers_minus1`.

[0232] `vps_sublayer_dpb_params_present_flag` controls whether the syntax elements `max_dec_pic_buffering_minus1[]`, `max_num_reorder_pics[]`, and `max_latency_increase_plus1[]` exist in the syntax structure `dpb_parameters()` in the VPS. When `vps_sub_dpb_params_info_present_flag` does not exist, `vps_sub_dpb_params_info_present_flag` is inferred to be 0.

[0233] The semantics of vps_sublayer_dpb_params_present_flag have similar issues to those of sps_sublayer_dpb_params_flag, as described above.

[0234] The sizes of the syntax element arrays max_dec_pic_buffering_minus1[ ], max_num_reorder_pics[ ], and max_latency_increase_plus1[ ] are undefined in the semantics. As can be seen from Table 2, when there is only one sublayer (corresponding to vps_dpb_max_tid[ i ] equal to 0 in Table 4), the syntax elements max_dec_pic_buffering_minus1[ ], max_num_reorder_pics[ ], and max_latency_increase_plus1[ ] are always indicated regardless of whether vps_sublayer_dpb_params_present_flag is equal to 0 or 1. vps_sublayer_dpb_params_present_flag does not control the existence of max_dec_pic_buffering_minus1[ ], max_num_reorder_pics[ ], and max_latency_increase_plus1[ ] in this case, which contradicts the current definition.

[0235] Example 9 In one example, the semantics of vps_sublayer_dpb_params_present_flag are modified as follows: `vps_sublayer_dpb_params_present_flag` controls whether the syntax elements `max_dec_pic_buffering_minus1[j]`, `max_num_reorder_pics[j]`, and `max_latency_increase_plus1[j]` exist in the syntax structure `dpb_parameters()` in the VPS. When `vps_dpb_max_tid[i]` in the VPS is greater than 0, the range of `i` is from 0 to `vps_dpb_max_tid[i] - 1` (inclusive). When `vps_sub_dpb_params_info_present_flag` does not exist, `vps_sub_dpb_params_info_present_flag` is inferred to be 0.

[0236] It should be noted that j, instead of i, is used to represent the range of the syntax element arrays max_dec_pic_buffering_minus1[ ], max_num_reorder_pics[ ], and max_latency_increase_plus1[ ], in order to avoid confusion with i in vps_dpb_max_tid[ i ].

[0237] When performing inter-frame prediction, the reference image can be a decoded image in the same layer or a reference image in a lower layer. In the latter case, the reference image in the lower layer is called an inter-layer reference picture (ILRP), and the lower layer that includes one or more ILRPs is called the reference layer (or dependent layer).

[0238] To support inter-layer prediction, several intermediate variables are used based on the syntax elements indicated in the VPS. For example, in Equation 37 of the VVC draft, the derivation process for the variables dependencyFlag[i][j], NumDirectRefLayers[i], DirectRefLayerIdx[i][d], NumRefLayers[i], RefLayerIdx[i][r], and LayerUsedAsRefLayerFlag[j] is defined as follows: The derivation of variables dependencyFlag[i][j], NumDirectRefLayers[i], DirectRefLayerIdx[i][d], NumRefLayers[i], RefLayerIdx[i][r], and LayerUsedAsRefLayerFlag[j] is as follows: for( i = 0; i<= vps_max_layers_minus1; i++ ) { for( j = 0; j<= vps_max_layers_minus1; j++ ) { dependencyFlag[ i ][ j ]= vps_direct_ref_layer_flag[ i ][ j ] for( k = 0; k <i; k++ ) if( vps_direct_ref_layer_flag[ i ][ k ]&&dependencyFlag[ k ][ j ]) dependencyFlag[i][j] = 1 } LayerUsedAsRefLayerFlag[ i ] = 0 } for( i = 0; i<= vps_max_layers_minus1; i++ ) { for( j = 0, d = 0, r = 0; j<= vps_max_layers_minus1; j++ ) {(37) if( vps_direct_ref_layer_flag[ i ][ j ]) { DirectRefLayerIdx[ i ][ d++ ]= j LayerUsedAsRefLayerFlag[j] = 1 } if(dependencyFlag[i][j]) RefLayerIdx[i][r++] = j } NumDirectRefLayers[i] = d NumRefLayers[i] = r } A dependencyFlag[i][j] equal to 1 indicates that the layer with index j is the reference layer of the layer with index i. A dependencyFlag[i][j] equal to 0 indicates that the layer with index j is not the reference layer of the layer with index i.

[0239] The dependency between layer i at index i and its reference layer j can be direct or indirect. If layer j is a direct dependency of layer i, the dependency is indicated using the syntax element vps_direct_ref_layer_flag[i][j] in the VPS (where i ranges from 0 to vps_max_layers_minus1, and j ranges from 0 to i–1). As shown in Table 4, syntax elements irrelevant to this invention have been removed:

[0240] The syntax above illustrates how a VPS indicates inter-layer dependencies when not all layers are independent.

[0241] `vps_max_layers_minus1` represents the maximum allowed number of layers in each CVS of the referenced VPS. Layer dependency information is indicated only starting from the second lowest layer (i.e., i=1); when not all layers are independent layers, layer dependency information is indicated by the syntax element `vps_all_independent_layers_flag`.

[0242] A `vps_all_independent_layers_flag` value of 1 indicates that all layers specified by the VPS are decoded independently, and inter-layer prediction is not required. A `vps_all_independent_layers_flag` value of 0 indicates that one or more layers specified by the VPS can use inter-layer prediction. When `vps_all_independent_layers_flag` does not exist, its value is inferred to be 1.

[0243] In VVC, the lowest layer (i.e., the one corresponding to vps_independent_layer_flag[0]) is always an independent layer. Therefore, the syntax element vps_independent_layer_flag[i] indicates the second lowest layer (i.e., vps_independent_layer_flag[1]), and the semantics of this syntax element are quite simple.

[0244] A value of 1 for `vps_independent_layer_flag[i]` indicates that the layer at index `i` does not use inter-layer prediction. A value of 0 for `vps_independent_layer_flag[i]` indicates that the layer at index `i` can use inter-layer prediction, and that the VPS contains the syntax element `vps_direct_ref_layer_flag[i][j]` (where `j` ranges from 0 to `i - 1`, inclusive). When `vps_independent_layer_flag[i]` is not present, its value is inferred to be 1.

[0245] Only when a layer is not an independent layer (i.e., vps_independent_layer_flag[ i ] is 0), the direct dependency layer of this layer is indicated by vps_direct_ref_layer_flag[ i ][ j ] (where the range of j is from 0 to i–1).

[0246] The semantic definition of vps_direct_ref_layer_flag[ i ][ j ] is as follows: vps_direct_ref_layer_flag[ i ][ j ] being equal to 0 means that the layer with index j is not a direct reference layer of the layer with index i. vps_direct_ref_layer_flag [ i ][ j ] being equal to 1 means that the layer with index j is a direct reference layer of the layer with index i. When vps_direct_ref_layer_flag[ i ][ j ] (where the ranges of i and j are from 0 to vps_max_layers_minus1, including the end values) does not exist, vps_direct_ref_layer_flag[ i ][ j ] is inferred to be 0. When vps_independent_layer_flag[ i ] is equal to 0, at least one value of j must be within the range from 0 to i–1 (including the end values) such that the value of vps_direct_ref_layer_flag[ i ][ j ] is equal to 1.

[0247] When a layer is a dependency layer (i.e., vps_independent_layer_flag[ i ] of this layer is equal to 0), the constraint condition in the semantics of vps_direct_ref_layer_flag[ i ][ j ] means that at least one layer in the lower layer j (range from 0 to i–1, including the end values) of this layer must be a reference layer of layer i.

[0248] In the case of indirect dependency, when layer i directly depends on layer k (k < i), and layer k depends on layer j (j < k), a situation may occur. In this case, layer i with index depends on layer j, and the corresponding dependencyFlag[ i ][ j ] is equal to 1.

[0249] The problem with the derivation process is that since only one layer can depend on lower layers, the loop for finding the dependent lower layer j in the derivation process can be simplified. There is no need to loop over all layers in the VPS, but only over the layers lower than the current layer i.

[0250] Example 5 In one example, the derivation process for the variables dependencyFlag[i][j], NumDirectRefLayers[i], DirectRefLayerIdx[i][d], NumRefLayers[i], RefLayerIdx[i][r], and LayerUsedAsRefLayerFlag[j] is modified as follows: The derivation of variables dependencyFlag[i][j], NumDirectRefLayers[i], DirectRefLayerIdx[i][d], NumRefLayers[i], RefLayerIdx[i][r], and LayerUsedAsRefLayerFlag[j] is as follows: for( i = 0; i<= vps_max_layers_minus1; i++ ) { for( j = 0; j <i; j++ ) { dependencyFlag[ i ][ j ]= vps_direct_ref_layer_flag[ i ][ j ] for( k = 0; k <i; k++ ) if( vps_direct_ref_layer_flag[ i ][ k ]&&dependencyFlag[ k ][ j ]) dependencyFlag[i][j] = 1 } LayerUsedAsRefLayerFlag[ i ] = 0 } for( i = 0; i<= vps_max_layers_minus1; i++ ) { for( j = 0, d = 0, r = 0; j <i; j++ ) {(37) if( vps_direct_ref_layer_flag[ i ][ j ]) { DirectRefLayerIdx[ i ][ d++ ]= j LayerUsedAsRefLayerFlag[j] = 1 } if(dependencyFlag[i][j]) RefLayerIdx[i][r++] = j } NumDirectRefLayers[i] = d NumRefLayers[i] = r } Example 6 In another example, the variables dependencyFlag[i][j], NumDirectRefLayers[i], DirectRefLayerIdx[i][d], NumRefLayers[i], RefLayerIdx[i][r], and LayerUsedAsRefLayerFlag[j] are modified as follows: The derivation of variables dependencyFlag[i][j], NumDirectRefLayers[i], DirectRefLayerIdx[i][d], NumRefLayers[i], RefLayerIdx[i][r], and LayerUsedAsRefLayerFlag[j] is as follows: for( i = 0; i<= vps_max_layers_minus1; i++ ) { for( j = 0; j <i; j++ ) { dependencyFlag[ i ][ j ]= vps_direct_ref_layer_flag[ i ][ j ] for( k = j+1; k <i; k++ ) if( vps_direct_ref_layer_flag[ i ][ k ]&&dependencyFlag[ k ][ j ]) dependencyFlag[i][j] = 1 } LayerUsedAsRefLayerFlag[ i ] = 0 } for( i = 0; i<= vps_max_layers_minus1; i++ ) { for( j = 0, d = 0, r = 0; j <i; j++ ) {(37) if( vps_direct_ref_layer_flag[ i ][ j ]) { DirectRefLayerIdx[ i ][ d++ ]= j LayerUsedAsRefLayerFlag[j] = 1 } if(dependencyFlag[i][j]) RefLayerIdx[i][r++] = j } NumDirectRefLayers[i] = d NumRefLayers[i] = r } Example 7 In another example, the variables dependencyFlag[i][j], NumDirectRefLayers[i], DirectRefLayerIdx[i][d], NumRefLayers[i], RefLayerIdx[i][r], and LayerUsedAsRefLayerFlag[j] are modified as follows: The derivation of variables dependencyFlag[i][j], NumDirectRefLayers[i], DirectRefLayerIdx[i][d], NumRefLayers[i], RefLayerIdx[i][r], and LayerUsedAsRefLayerFlag[j] is as follows: for( i = 0; i<= vps_max_layers_minus1; i++ ) { for( j = 0; j<= vps_max_layers_minus1; j++ ) { dependencyFlag[ i ][ j ]= vps_direct_ref_layer_flag[ i ][ j ] for( k = j+1; k <i; k++ ) if( vps_direct_ref_layer_flag[ i ][ k ]&&dependencyFlag[ k ][ j ]) dependencyFlag[ i ][ j ]= 1 } LayerUsedAsRefLayerFlag[ i ] = 0 } for( i = 0; i<= vps_max_layers_minus1; i++ ) { for( j = 0, d = 0, r = 0; j<= vps_max_layers_minus1; j++ ) {(37) if( vps_direct_ref_layer_flag[ i ][ j ]) { DirectRefLayerIdx[ i ][ d++ ]= j LayerUsedAsRefLayerFlag[ j ] = 1 } if( dependencyFlag[ i ][ j ]) RefLayerIdx[ i ][ r++ ]= j } NumDirectRefLayers[ i ] = d NumRefLayers[ i ] = r } Only the loop for( k = 0; k<i; k++ ) is modified to for( k = j+1; k<i; k++ ).

[0251] Example 8 Correspondingly, the semantics of vps_direct_ref_layer_flag[ i ][ j ] are modified as follows: `vps_direct_ref_layer_flag[i][j]` equal to 0 indicates that the layer at index `j` is not a direct reference layer to the layer at index `i`. `vps_direct_ref_layer_flag[i][j]` equal to 1 indicates that the layer at index `j` is a direct reference layer to the layer at index `i`. When `vps_direct_ref_layer_flag[i][j]` (where `i` ranges from 1 to `vps_max_layers_minus1`, and `j` ranges from 0 to `i–1`, inclusive) does not exist, `vps_direct_ref_layer_flag[i][j]` is inferred to be 0. When `vps_independent_layer_flag[i]` equals 0, at least one value of `j` must be in the range of 0 to `i–1` (inclusive) such that the value of `vps_direct_ref_layer_flag[i][j]` equals 1.

[0252] Several other variable syntax elements represent dependencies similar to vps_direct_ref_layer_flag[i][j], and their scopes must be modified in the same way.

[0253] Based on the above description, this paper provides a method for decoding a video stream and a method for encoding a video stream. Accordingly, this paper provides an apparatus for decoding (decoded) a video stream and an apparatus for encoding a video stream.

[0254] Figure 14A method for decoding a video stream, implemented by a decoding device, is illustrated. A sequence parameter set (SPS) is decoded into the video stream and includes syntax elements applied to the video sequence. The method includes: obtaining a value of a first syntax element (e.g., sps_ptl_dpb_hrd_params_present_flag described above) from the SPS (e.g., by parsing the stream), wherein the value of the first syntax element is used to indicate whether a decoded picture buffer (DPB) parameter syntax structure exists in the SPS. The method further includes: at least when it is determined that the value of the first syntax element indicates that the DPB parameter syntax structure exists in the SPS, obtaining the value of a second syntax element (e.g., sps_sublayer_dpb_params_flag described above) from the SPS (e.g., by parsing the bitstream), wherein the value of the second syntax element is used to indicate whether a DPB syntax element (e.g., one of max_dec_pic_buffering_minus1[i], max_num_reorder_pics[i], and max_latency_increase_plus1[i] described in detail above) exists in the DPB parameter syntax structure, and the DPB syntax element is applied to the temporal sublayers of the video sequence other than the highest temporal sublayer.

[0255] It should be noted that the bitstream can be obtained through a wireless network or a wired network. The bitstream can be transmitted from a website, server, or other remote source using wireless technologies such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or infrared, radio, microwave, Wi-Fi, Bluetooth, LTE, or 5G.

[0256] The bitstream is a series of bits, in the form of a network abstraction layer (NAL) unit stream or a byte stream, and constitutes a series of access units (AUs), which together form one or more coded video sequences (CVSs).

[0257] In a specific example, the bitstream format represents the relationship between network abstraction layer (NAL) cell streams and byte streams, which are called bitstreams.

[0258] Bitstreams can have two formats: NAL unit stream format and byte stream format. The NAL unit stream format is conceptually the more "basic" type. The NAL unit stream format consists of a series of syntactic structures called NAL units. This sequence is ordered according to the decoding order. Several constraints are applied to the decoding order (and content) of the NAL units in the NAL unit stream.

[0259] The NAL units are sorted according to the decoding order, and a start code prefix and zero or more zero-value bytes are added before each NAL unit to form a byte stream. This allows the byte stream format to be constructed from the NAL unit stream format. The unique start code prefix pattern is then searched for within the byte stream, allowing the NAL unit stream format to be extracted from the byte stream format.

[0260] Figure 15 A method for encoding a video stream, implemented by an encoding device, is shown. A sequence parameter set (SPS) is decoded in the video stream and includes syntax elements applied to the video sequence. The method includes: determining 1510 whether a decoded picture buffer (DPB) parameter syntax structure exists in the SPS; and encoding 1520 the value of a first syntax element (e.g., sps_ptl_dpb_hrd_params_present_flag described above) into the SPS by determining whether the DPB parameter syntax structure exists in the SPS, wherein the value of the first syntax element is used to indicate whether the DPB parameter syntax structure exists in the SPS. Figure 15 The method described includes: when it is determined in step 1510 (e.g., only if it is determined) that the DPB parameter syntax structure exists in the SPS, determining 1530 whether a DPB syntax element (e.g., one of max_dec_pic_buffering_minus1[i], max_num_reorder_pics[i], and max_latency_increase_plus1[i] as described in detail above) exists in the DPB parameter syntax structure, wherein the DPB syntax element is applied to a temporal sublayer other than the highest temporal sublayer in the video sequence; and encoding 1540 the value of a second syntax element (e.g., sps_sublayer_dpb_params_flag as described above) into the SPS by determining whether the DPB syntax element exists in the DPB parameter syntax structure, wherein the value of the second syntax element is used to indicate that the DPB syntax element exists in the DPB parameter syntax structure.

[0261] The methods described above can be applied to video decoding devices or video encoding devices (generating bitstreams), as follows.

[0262] like Figure 16 As shown, a video decoding apparatus 1600 provided herein according to one embodiment includes an acquisition unit 1610 (e.g., including a parser) and a determination unit 1620. Figure 17 As shown, a video encoding apparatus 1700 provided herein according to one embodiment includes a determining unit 1710 and an encoding unit 1720.

[0263] Figure 16 The video decoding apparatus 1600 shown includes an acquisition unit 1610 for acquiring the value of a first syntax element (e.g., sps_ptl_dpb_hrd_params_present_flag described above) from the SPS, wherein the SPS is decoded in the video bitstream, and the value of the first syntax element is used to indicate whether a decoded picture buffer (DPB) parameter syntax structure exists in the SPS. Figure 16 The video decoding apparatus 1600 shown includes a determining unit 1620 for determining whether the value of the first syntax element indicates the presence of the DPB parameter syntax structure in the SPS. Further, the obtaining unit 1610 is configured to: at least when it is determined that the value of the first syntax element indicates the presence of the DPB parameter syntax structure in the SPS, obtain the value of a second syntax element (e.g., the sps_sublayer_dpb_params_flag described above) from the SPS, wherein the value of the second syntax element is used to indicate whether a DPB syntax element (e.g., one of max_dec_pic_buffering_minus1[i], max_num_reorder_pics[i], and max_latency_increase_plus1[i] described in detail above) exists in the DPB parameter syntax structure, and the DPB syntax element is applied to temporal sublayers in the video sequence other than the highest temporal sublayer.

[0264] Figure 17 The determination unit 1710 included in the video encoding apparatus 1700 shown is used to determine whether the decoded picture buffer (DPB) parameter syntax structure exists in the SPS. Figure 17The encoding unit 1720 included in the video encoding apparatus 1700 shown is configured to: encode the value of a first syntax element (e.g., sps_ptl_dpb_hrd_params_present_flag described above) into the SPS by determining whether the DPB parameter syntax structure exists in the SPS, wherein the value of the first syntax element is used to indicate whether the DPB parameter syntax structure exists in the SPS. Further, the determining unit 1710 is configured to: when it is determined (e.g., only when it is determined) that the DPB parameter syntax structure exists in the SPS, determine whether a DPB syntax element (e.g., one of max_dec_pic_buffering_minus1[i], max_num_reorder_pics[i], and max_latency_increase_plus1[i] described in detail above) exists in the DPB parameter syntax structure, wherein the DPB syntax element is applied to temporal sub-layers in the video sequence other than the highest temporal sub-layer. Further, the encoding unit 1720 is configured to: encode the value of the second syntax element (e.g., sps_sublayer_dpb_params_flag described above) into the SPS by determining whether the DPB syntax element exists in the DPB parameter syntax structure, wherein the value of the second syntax element is used to indicate whether the DPB syntax element exists in the DPB parameter syntax structure.

[0265] Figure 16 The video decoding device 1600 shown can be Figure 1A , Figure 1B and Figure 3 The decoder 30 shown and Figure 9 The video decoder 3206 shown may also be included therein. Furthermore, the decoding device 1700 may be included in... Figure 4 The video decoding device 400 shown Figure 5 The device 500 shown and Figure 8 The terminal device 3106 shown. Figure 17 The encoding device 1700 shown can be Figure 1A , Figure 1B and Figure 3 The encoder 20 shown may also be included therein. Furthermore, the encoding device 1700 may be included in... Figure 4 The video decoding device 400 shown Figure 5 The device 500 shown and Figure 8 The capture device 3102 shown.

[0266] Specifically, the SPS includes information indicating the sub-image.

[0267] The table below shows some snapshots from ITU JVET-Q2001-v13 indicating a portion of a sub-image in SPS. Download links are provided below: http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 17_Brussels / wg11 / JVET-Q2001-v13.zip. For the remainder of this application, this document is referred to as VVC Draft 8 for simplicity.

[0268]

[0269] In SPS, certain syntax elements indicate the location information and control flags for each sub-image. The location information for the i-th sub-image includes: subpic_ctu_top_left_x[i] represents the horizontal component of the top-left coordinate of subpic i in the image; or `subpic_ctu_top_left_y[i]` represents the vertical component of the top-left coordinate of subpic i in the image; or subpic_width_minus1[i] represents the width of subpic i in the image; or subpic_height_minus1[i] represents the height of subpici i in the image.

[0270] Some syntax elements indicate the number of subpics within an image, such as sps_num_subpics_minus1.

[0271] The image is segmented into CTUs, strips, blocks, and sub-images. Segment the image into CTUs The image is segmented into a series of coding tree units (CTUs). The term "CTU" is sometimes used interchangeably with "coding tree block" (CTB). In one example, the term "CTU" has the same definition as CTU in ITU-T H.265. For an image with a 3-sample array, one CTU consists of one N×N luminance sample block and two corresponding chrominance sample blocks. Figure 6 An example of image segmentation into CTUs is shown. All CTUs must be the same size, except for those located at image boundaries (incomplete CTUs may exist).

[0272] Segment the image into blocks In some examples, when tiled rendering is used, the image is divided into rectangular CTU groups separated by vertical and / or horizontal boundaries. The vertical and horizontal tile boundaries intersect the image from top to bottom and from the left to the right edge, respectively. Indicative information related to the positions of the horizontal and vertical tile boundaries is decoded in the bitstream.

[0273] Figure 7 The example illustrates dividing an image into nine blocks. In this example, the block boundaries are marked with bold dashed lines.

[0274] When the image is vertically divided into more than one block, the scanning order of the CTUs changes as the raster scanning order of the CTUs in the image changes. The rules for scanning CTUs are as follows: 1. The blocks are scanned from left to right and from top to bottom according to the raster scan order, which is called the block scan order. This means that starting from the top left block, all blocks in the same block row are scanned from left to right. Then, starting from the first block in the second block row (the block row below), all blocks in the second block row are scanned from left to right. This process is repeated until all blocks have been scanned.

[0275] 2. For a block, scan the CTUs in that block according to the raster scan order. For each CTU row, scan the CTUs from left to right and the CTU rows from top to bottom. Figure 7 The scan order of CTUs in the block is shown, and the numbers corresponding to CTUs indicate the scan order.

[0276] Image segmentation enables image partitioning, allowing each block to be decoded independently of other blocks within the same image. Decoding includes entropy decoding, residual decoding, and predictive decoding. Furthermore, segmentation allows images to be divided into regions of similar size. Therefore, image blocks can be processed in parallel, making it suitable for multi-core processing environments where each processing core is identical to the others.

[0277] The terms "processing order" and "scanning order" are used in this application as follows: Processing refers to the encoding or decoding of CTUs in the encoder or decoder. The scan order is related to the indexing of a specific partition in the image. CTUs are indexed in ascending order in the image under a specified scan order. The CTU scan order in a block refers to how the CTUs in the block are indexed, which may differ from the order in which they are processed (i.e., the processing order).

[0278] Segment the image into strips Striping enables image segmentation, allowing each strip to be decoded independently along with other strips in the same image. Decoding includes entropy decoding, residual decoding, and predictive decoding. Unlike block segmentation, stripes can have arbitrary shapes, not necessarily rectangular (offering more segmentation possibilities). The purpose of striping is not parallel processing, but rather to enhance error resilience and improve packet size matching in transmission environments.

[0279] A stripe may include an entire image or a portion of an image. In ITU-T H.265, a stripe comprises consecutive CTUs within the image in the processing order. A stripe is identified by its starting CTU address, which is indicated in the stripe header, image parameter set, or other unit. In one example, when a stripe is required to include an integer number of blocks, it may be identified based on the starting block address.

[0280] In VVC Draft 8, a stripe comprises an integer number of blocks in an image or an integer number of consecutive CTU rows within a block. Therefore, the vertical stripe boundary is also the vertical block boundary. The horizontal boundary of a stripe may not be a block boundary; the horizontal CTU boundary can be included within a block. In one example, when a block is divided into multiple rectangular stripes, each stripe comprises an integer number of consecutive complete CTU rows within the block.

[0281] In some examples, two striping modes exist: raster scan striping mode and rectangular striping mode. In raster scan striping mode, a strip comprises a series of blocks under block raster scans in the image. In rectangular striping mode, a strip comprises multiple blocks that together form a rectangular region in the image; or, a strip comprises multiple consecutive CTU rows within a single block that together form a rectangular region in the image. The blocks within a rectangular strip are scanned within the rectangular region corresponding to that strip in the order of block raster scans.

[0282] All stripes in an image together constitute the entire image; that is, all CTUs in an image are included within one of the stripes in the image. Similar rules apply to blocks and sub-images.

[0283] Segment the image into sub-images A sub-image can be a rectangular partition of an image. A sub-image can be the entire image or a portion of an image. Sub-images enable image segmentation, allowing each sub-image to be decoded independently of other sub-images in the entire video sequence. In VVC Draft 8, a sub-image i can be decoded independently of other sub-images in the entire video sequence when the subpic_treated_as_pic_flag[i] is true (e.g., the value of subpic_treated_as_pic_flag[i] is 1).

[0284] The difference between a sub-image and chunks or stripes is that a sub-image creates an independently decodeable video sequence. For chunks and stripes, independent decoding is performed in a single image within the video sequence.

[0285] In VVC Draft 8, a subimage comprises one or more stripes that collectively cover a rectangular region in the image. Therefore, each subimage boundary is always a stripe boundary, and each vertical subimage boundary is always a vertical block boundary.

[0286] Figure 10 An example of blocks, stripes, and sub-images is shown.

[0287] exist Figure 10 In one example shown, the image is segmented into 216 CTUs, 4 blocks, 4 stripes, and 3 subimages. The value of sps_num_subpics_minus1 is 2, and the position-related syntax elements have the following values: For sub-image 0, subpic_ctu_top_left_x[0] was not indicated, but was inferred to be 0; subpic_ctu_top_left_y[0] was not indicated, but was inferred to be 0; The value of subpic_width_minus1[0] is 8; The value of subpic_height_minus1[0] is 11.

[0288] For sub-image 1, The value of subpic_ctu_top_left_x[1] is 9; The value of subpic_ctu_top_left_y[1] is 0; The value of subpic_width_minus1[1] is 8; The value of subpic_height_minus1[1] is 5.

[0289] For sub-image 2, The value of subpic_ctu_top_left_x[2] is 9; The value of subpic_ctu_top_left_y[2] is 6; subpic_width_minus1[2] was not indicated, but was inferred to be 8; subpic_height_minus1[2] was not indicated, but was inferred to be 5.

[0290] Indicates information about the Decoded Picture Buffer (DPB). Decoding image buffer A Decoded Picture Buffer (DPB) is a buffer used to store decoded pictures for reference (e.g., as reference pictures for inter-frame prediction). In an example published in a VVC draft (e.g., ITU JVET-Q2001-v13), the relevant syntax elements for DBP parameters in the sequence parameter set (SPS) are highlighted.

[0291] Table 6: Syntax Elements Related to Decoding Images in SPS

[0292] sps_max_sublayers_minus1 + 1 represents the maximum number of time-limited sublayers that can exist in each CLVS of the reference SPS. The value of sps_max_sublayers_minus1 should be up to vps_max_sublayers_minus1 (inclusive).

[0293] `sps_sublayer_dpb_params_flag` controls whether the syntax elements `max_dec_pic_buffering_minus1[i]`, `max_num_reorder_pics[i]`, and `max_latency_increase_plus1[i]` exist in the syntax structure `dpb_parameters()` in SPS. When `sps_sub_dpb_params_info_present_flag` does not exist, its value is inferred to be 0.

[0294] A value of 1 for `sps_ptl_dpb_hrd_params_present_flag` indicates that the syntax structures `profile_tier_level()` and `dpb_parameters()` exist in SPS. Furthermore, SPS may also contain the syntax structures `general_hrd_parameters()` and `ols_hrd_parameters()`. A value of 0 for `sps_ptl_dpb_hrd_params_present_flag` indicates that these four syntax structures do not exist in SPS. The value of `sps_ptl_dpb_hrd_params_present_flag` must be equal to `vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]`.

[0295] The syntax element `sps_max_sublayers_minus1` represents the number of available time sublayers. When the number of available time sublayers is greater than 1 (e.g., the value of the syntax element `sps_max_sublayers_minus1` is greater than 0), the value of the syntax element `sps_sublayer_dpb_params_flag` is indicated in the bitstream. The syntax element `sps_sublayer_dpb_params_flag` indicates whether decoded image information is indicated for each available sublayer (when the value of this syntax element equals 1) or only for the highest time sublayer (when the value of this syntax element equals 0). When the value of `sps_sublayer_dpb_params_flag` is not present, for example, when there is only one time sublayer (the value of `sps_max_sublayers_minus1` == 0), the value of `sps_max_sublayers_minus1` is inferred to be 0.

[0296] In some examples, the value of the syntax element `sps_ptl_dpb_hrd_params_present_flag` indicates whether the indicator structure `dpb_parameter()` is indicated in SPS. When the value of the syntax element `sps_ptl_dpb_hrd_params_present_flag` is equal to 1, the indicated data structure `dpb_parameter()` is invoked, where the number of available time sublayers minus 1 (`sps_max_sublayers_minus1`) and the flag `sps_sublayer_dpb_params_flag` are used as the first and second parameters, respectively.

[0297] In one example, the indicator structure dpb_parameter() in the VVC draft (e.g., ITU JVET-Q2001-v13) is defined as follows: Table 7: Definitions of dpb_parameters

[0298] The syntax structure dpb_parameters() provides information on the DPB size, maximum image reordering number, and maximum delay for one or more OLS.

[0299] (Output layer set (OLS): A set of layers that are specified as output layers.) When the VPS includes the syntax structure dpb_parameters(), the OLS to which the syntax structure dpb_parameters() applies is specified by the VPS. When the SPS includes the syntax structure dpb_parameters(), the syntax structure applies only to the OLS that includes the lowest layer of all layers referencing the SPS, and that lowest layer is an independent layer.

[0300] `max_dec_pic_buffering_minus1[i] + 1` represents the maximum required size of the DPB, in units of the image storage buffer, when `Htid` equals `i`. The value of `max_dec_pic_buffering_minus1[i]` should be up to `MaxDpbSize - 1` (inclusive), where `MaxDpbSize` is as described in Section A.4.2. When `i` is greater than 0, `max_dec_pic_buffering_minus1[i]` must be greater than or equal to `max_dec_pic_buffering_minus1[i - 1]`. When max_dec_pic_buffering_minus1[i] (where i ranges from 0 to maxSubLayersMinus1 – 1, inclusive) does not exist, max_dec_pic_buffering_minus1[i] is inferred to be max_dec_pic_buffering_minus1[maxSubLayersMinus1] since subLayerInfoFlag is equal to 0.

[0301] `max_num_reorder_pics[i]` represents the maximum number of images that can be decoded before any image in the OLS and whose output order follows that image when `Htid` equals `i`. The value of `max_num_reorder_pics[i]` should range to `max_dec_pic_buffering_minus1[i]` (inclusive). When `i` is greater than 0, `max_num_reorder_pics[i]` must be greater than or equal to `max_num_reorder_pics[i – 1]`. When `max_num_reorder_pics[i]` (where `i` ranges from 0 to `maxSubLayersMinus1 – 1`, inclusive) does not exist, since `subLayerInfoFlag` equals 0, `max_num_reorder_pics[i]` is inferred to be `max_num_reorder_pics[maxSubLayersMinus1]`.

[0302] The value of mmax_latency_increase_plus1[i] is not equal to 0 and is used to calculate the value of MaxLatencyPictures[i]. The value of MaxLatencyPictures[i] represents the maximum number of images in the OLS whose output order can precede any image in the OLS and whose decoding order can follow any image when Htid equals i.

[0303] When max_latency_increase_plus1[i] is not equal to 0, the value of MaxLatencyPictures[i] is represented as follows: MaxLatencyPictures[ i ] = max_num_reorder_pics[ i ]+ max_latency_increase_plus1[ i ] – 1 (7-111) When max_latency_increase_plus1[i] equals 0, it does not represent the corresponding limit.

[0304] The value range of max_latency_increase_plus1[i] should be up to 2. 32– 2 (inclusive of end values). When max_latency_increase_plus1[i] (where i ranges from 0 to maxSubLayersMinus1 – 1, inclusive of end values) does not exist, max_latency_increase_plus1[i] is inferred to be max_latency_increase_plus1[maxSubLayersMinus1] since subLayerInfoFlag is equal to 0.

[0305] The `dpb_parameters` structure indicates the decoded image buffer information for a single sublayer or the decoded image information for each sublayer, controlled by the value of `subLayerInfoFlag`. When `subLayerInfoFlag` is 0, it indicates the decoded image buffer information for the highest-level sublayer in the available time sublayers (i = maxSubLayersMinus1). When `subLayerInfoFlag` is 1, it indicates the decoded image buffer information for each sublayer in the available time sublayers (i ranges from 0 to maxSubLayersMinus1, inclusive).

[0306] In some examples, the syntax element sps_sublayer_dpb_params_flag is used only as the second parameter of dpb_parameters.

[0307] Example 10 According to the tenth embodiment, the value of the syntax element sps_sublayer_dpb_params_flag is decoded into the bitstream based on the value of the syntax element sps_ptl_dpb_hrd_params_present_flag, as shown below: Table 8: Syntax Elements in SPS

[0308] In this embodiment, when the value of sps_max_sublayers_minus1 is greater than 0 and the value of sps_ptl_dpb_hrd_params_present_flag is equal to 0, the value of the syntax element sps_sublayer_dpb_params_flag does not need to be decoded in the bitstream.

[0309] Example 11 According to the eleventh embodiment, the value of the syntax element sps_sublayer_dpb_params_flag is decoded into the bitstream based on the value of the syntax element sps_ptl_dpb_hrd_params_present_flag, as shown below: Table 9: Syntax Elements in SPS

[0310] In this embodiment, when the value of `sps_max_sublayers_minus1` is greater than 0 and the value of `sps_ptl_dpb_hrd_params_present_flag` is equal to 0, the value of the syntax element `sps_sublayer_dpb_params_flag` does not need to be decoded in the bitstream. Furthermore, the syntax element is reconstructed, resulting in a clearer design. The syntax element `sps_ptl_dpb_hrd_params_present_flag` is upgraded to represent the syntax element `sps_sublayer_dpb_params_flag` and the indicator structure `dpb_parameters()`.

[0311] Example 12 According to the twelfth embodiment, the value of the syntax element sps_sublayer_dpb_params_flag is decoded into the bitstream based on the value of the syntax element sps_ptl_dpb_hrd_params_present_flag, as shown below: Table 10: Syntax Elements in PPS

[0312] Accordingly, the indicator structure pb_parameters is modified as follows:

[0313] In this embodiment, when the value of `sps_max_sublayers_minus1` is greater than 0 and the value of `sps_ptl_dpb_hrd_params_present_flag` is equal to 0, the value of the syntax element `sps_sublayer_dpb_params_flag` does not need to be decoded in the bitstream. Compared to embodiment 11, the indication structure `dpb_parameter()` is indicated by the parameters `firstSubLayer` and `maxSubLayersMinus1` from the first sublayer to the highest sublayer, respectively. The selection of the first sublayer is moved up to the next higher level `dpb_parameter()`. In this way, the semantics of `sps_sublayer_dpb_params_flag` are clearer.

[0314] In one example disclosed in a VVC draft (e.g., ITU JVET-Q2001-v13), the stripe has two modes: raster scan stripe mode and rectangular stripe mode. However, according to the current definition, the stripe can be in either raster scan mode or rectangular mode.

[0315] The definition of a strip is as follows: A stripe comprises an integer number of complete blocks in an image, or an integer number of consecutive complete CTU rows within a block in an image. Therefore, each vertical stripe boundary is always also a vertical block boundary. The horizontal boundary of a stripe may not be a block boundary, but rather includes the horizontal CTU boundary within a block; this occurs when a block is divided into multiple rectangular stripes, each stripe comprising an integer number of consecutive complete CTU rows within the block.

[0316] Two stripe modes are supported: raster scan stripe mode and rectangular stripe mode. In raster scan stripe mode, a stripe comprises a series of complete blocks in the image under block raster scan. In rectangular stripe mode, a stripe comprises multiple complete blocks that together form a rectangular region in the image; or, a stripe comprises multiple consecutive complete CTU rows within a block that together form a rectangular region in the image. Blocks within a rectangular stripe are scanned within the corresponding rectangular region according to the block raster scan sequence.

[0317] The only constraint on raster scan strips is that the strip "comprises a series of complete blocks under a block raster scan of the image." For example... Figure 12 As shown, for example, the image contains only two blocks and one stripe. This stripe can be in raster scan stripe mode or rectangular stripe mode. This can cause confusion between the two stripe modes.

[0318] Example 13 It is recommended to further restrict the definition of raster scan stripes, as follows: In raster scan strip mode, a strip comprises a series of complete blocks in an image under block raster scan, the image including at least one strip that constitutes a non-rectangular shape.

[0319] For example, under this definition of raster scan stripes, Figure 13 The image shown is a raster scan strip, but Figure 12 The image shown can only be displayed in rectangular stripe mode.

[0320] Accordingly, the instructions for the block structure have been modified.

[0321] In one example, the chunking information is indicated in PPS as follows:

[0322] A value of 1 for `no_pic_partition_flag` indicates that image segmentation should not be applied to each image in the reference PPS. A value of 0 for `no_pic_partition_flag` indicates that each image in the reference PPS can be segmented into more than one block or strip.

[0323] The requirement for bitstream consistency is that the value of no_pic_partition_flag must be the same for all PPS referenced by the coded picture within CLVS.

[0324] The requirement for bitstream consistency is: when the value of sps_num_subpics_minus1 + 1 is greater than 1, the value of no_pic_partition_flag must not be equal to 1.

[0325] pps_log2_ctu_size_minus5 + 5 represents the size of the luminance coding tree block for each CTU. pps_log2_ctu_size_minus5 must be equal to sps_log2_ctu_size_minus5.

[0326] `num_exp_tile_columns_minus1 + 1` represents the number of explicitly provided tile column widths. The value of `num_exp_tile_columns_minus1` should range to `PicWidthInCtbsY - 1` (inclusive). When `no_pic_partition_flag` equals 1, the value of `num_exp_tile_columns_minus1` is inferred to be 0.

[0327] `num_exp_tile_rows_minus1 + 1` represents the number of explicitly provided tile row heights. The value of `num_exp_tile_rows_minus1` should range to `PicHeightInCtbsY - 1` (inclusive). When `no_pic_partition_flag` equals 1, the value of `num_tile_rows_minus1` is inferred to be 0.

[0328] `tile_column_width_minus1[i] + 1` represents the width of the i-th tile column in CTB units, where i ranges from 0 to `num_exp_tile_columns_minus1 – 1` (inclusive). `tile_column_width_minus1[num_exp_tile_columns_minus1]` is used to deduce the width of tile columns whose index is greater than or equal to `num_exp_tile_columns_minus1` (as described in 6.5.1). The value of `tile_column_width_minus1[i]` should range to `PicWidthInCtbsY – 1` (inclusive). When `tile_column_width_minus1[0]` does not exist, the value of `tile_column_width_minus1[0]` is deduced to be `PicWidthInCtbsY – 1`.

[0329] `tile_row_height_minus1[i] + 1` represents the height of the i-th tile row in CTB units, where i ranges from 0 to `num_exp_tile_rows_minus1 – 1` (inclusive). `tile_row_height_minus1[num_exp_tile_rows_minus1]` is used to deduce the height of tile rows whose index is greater than or equal to `num_exp_tile_rows_minus1` (as described in Section 6.5.1). The value of `tile_row_height_minus1[i]` should range to `PicHeightInCtbsY – 1` (inclusive). When `tile_row_height_minus1[0]` does not exist, the value of `tile_row_height_minus1[0]` is deduced to be `PicHeightInCtbsY – 1`.

[0330] A `rect_slice_flag` value of 0 indicates that the slices within each slice are in the raster scan order, and the slice information is not indicated in the PPS. A `rect_slice_flag` value of 1 indicates that the slices within each slice cover a rectangular region in the image, and the slice information is indicated in the PPS. When `rect_slice_flag` does not exist, it is inferred to be 1. When `subpic_info_present_flag` is equal to 1, the value of `rect_slice_flag` must be equal to 1.

[0331] `single_slice_per_subpic_flag` equal to 1 indicates that each subpicture includes only one rectangular stripe. `single_slice_per_subpic_flag` equal to 0 indicates that each subpicture may include one or more rectangular stripes. When `single_slice_per_subpic_flag` equals 1, `num_slices_in_pic_minus1` is inferred to be `sps_num_subpics_minus1`. When `single_slice_per_subpic_flag` does not exist, its value is inferred to be 0. [Editor's Note (GJS): Consider renaming this flag or otherwise making it explicit to avoid interpreting it as only relevant when multiple subpictures are included in each image.] When the number of blocks in the image is greater than 1, the flag rect_slice_flag is parsed from the bitstream. The value of rect_slice_flag indicates whether the image uses rectangular stripe mode (when the value is equal to 1) or raster scan stripe mode (when the value is equal to 0).

[0332] Accordingly, in the strip header,

[0333] A picture_header_in_slice_header_flag value of 1 indicates that a PH syntax structure exists in the slice header. A picture_header_in_slice_header_flag value of 0 indicates that a PH syntax structure does not exist in the slice header.

[0334] The requirement for bitstream consistency is that the value of picture_header_in_slice_header_flag must be the same for all encoded slices in CLVS.

[0335] When the picture_header_in_slice_header_flag of the encoded stripe is equal to 1, the requirement for bitstream consistency is: there must be no VCL NAL unit with nal_unit_type of PH_NUT in CLVS.

[0336] When picture_header_in_slice_header_flag equals 0, all coded stripes in the current image must have picture_header_in_slice_header_flag equal to 0, and the current PU must have a PHNAL unit.

[0337] `slice_subpic_id` represents the subpick ID of the subpick containing the slice. If `slice_subpic_id` exists, the value of the variable `CurrSubpicIdx` is deduced such that `SubpicIdVal[CurrSubpicIdx]` equals `slice_subpic_id`. Otherwise (if `slice_subpic_id` does not exist), `CurrSubpicIdx` is deduced to be 0. The length of `slice_subpic_id` is (sps_subpic_id_len_minus1 + 1) bits.

[0338] `slice_address` represents the slice address. When `slice_address` does not exist, its value is inferred to be 0. When `rect_slice_flag` equals 1 and `NumSlicesInSubpic[CurrSubpicIdx]` equals 1, the value of `slice_address` is inferred to be 0.

[0339] If rect_slice_flag equals 0, the following applies: The stripe address is the raster scan block index.

[0340] The length of slice_address is Ceil(Log2(NumTilesInPic)) bits.

[0341] The value range of slice_address should be up to NumTilesInPic – 1 (inclusive).

[0342] Otherwise (rect_slice_flag equals 1), the following applies: The strip address is the sub-image level strip index of the strip.

[0343] The length of slice_address is Ceil( Log2( NumSlicesInSubpic[ CurrSubpicIdx ] )) bits.

[0344] The value range of slice_address should be up to NumSlicesInSubpic[ CurrSubpicIdx ] – 1 (inclusive).

[0345] The requirement for stream consistency applies under the following constraints: If rect_slice_flag is equal to 0 or subpic_info_present_flag is equal to 0, then the value of slice_address must not be equal to the value of slice_address of any other encoded slice NAL unit of the same encoded image.

[0346] Otherwise, the pair of values ​​for slice_subpic_id and slice_address must not be equal to the pair of values ​​for slice_subpic_id and slice_address of any other encoded slice NAL unit of the same encoded image.

[0347] The shape of the stripes in the image should be such that each CTU, when decoded, should have its entire left and upper boundaries encompassing the image boundary or the boundaries of one or more previously decoded CTUs.

[0348] sh_extra_bit[i] can be equal to 1 or 0. Decoders conforming to this specification version can ignore the value of sh_extra_bit[i]. Its value does not affect the consistency of the decoder with the profile specified in this specification version.

[0349] `num_tiles_in_slice_minus1 + 1`, when present, indicates the number of segments in a slice. The value of `num_tiles_in_slice_minus1` should range to `NumTilesInPic - 1` (inclusive).

[0350] It is recommended to modify the above indication mechanism in PPS and strip header as follows:

[0351]

[0352] Specifically, the following embodiments are also provided (in the new examples) in this document.

[0353] 1. A decoding method implemented by a decoding device or an encoding device, wherein the method comprises: The bitstream is acquired, in which the sequence parameter set (SPS) is decoded into the bitstream; The value of the first syntax element sps_max_sublayers_minus1 is obtained according to the bitstream, wherein the value of the first syntax element sps_max_sublayers_minus1 is used to represent the maximum number of temporal sublayers existing in the coded layer video sequence (CLVS) with reference to the SPS; The value of the second syntax element sps_ptl_dpb_hrd_params_present_flag is obtained from the bitstream, wherein the value of the second syntax element sps_ptl_dpb_hrd_params_present_flag is used to indicate whether there is a decoded picture buffer (DPB) parameter syntax structure (e.g., dpb_parameters) in the SPS. When the value of the first syntax element sps_max_sublayers_minus1 is greater than the first default value (e.g., the first default value is equal to 0), and when the value of the second syntax element sps_ptl_dpb_hrd_params_present_flag is equal to the second default value (e.g., the second default value is equal to 1), the value of the third syntax element sps_sublayer_dpb_params_flag is parsed from the bitstream, wherein the value of the third syntax element sps_sublayer_dpb_params_flag is used to control whether there are syntax elements (e.g., max_dec_pic_buffering_minus1[i], max_num_reorder_pics[i] and / or max_latency_increase_plus1[i]) in the DPB parameter syntax structure (e.g., dpb_parameters) in the SPS, i ranges from 0 to sps_max_sublayers_minus1 – 1, inclusive)

[0354] 2. The method according to Embodiment 1, wherein the method further includes: When the value of the first syntax element sps_max_sublayers_minus1 is less than or equal to the first default value, or when the value of the second syntax element sps_ptl_dpb_hrd_params_present_flag is not equal to the second default value, the value of the third syntax element sps_sublayer_dpb_params_flag is set to the third default value (for example, the third default value is equal to 0 or 1).

[0355] 3. The method according to embodiment 1 or 2, wherein the value of the third syntax element sps_sublayer_dpb_params_flag is decoded in the SPS.

[0356] 4. A decoding method implemented by a decoding device or an encoding device, wherein the method comprises: The bitstream is acquired, in which the sequence parameter set (SPS) is decoded into the bitstream; The value of the first syntax element sps_max_sublayers_minus1 is obtained according to the bitstream, wherein the value of the first syntax element sps_max_sublayers_minus1 is used to represent the maximum number of temporal sublayers existing in the coded layer video sequence (CLVS) with reference to the SPS; The value of the second syntax element sps_ptl_dpb_hrd_params_present_flag is obtained from the bitstream, wherein the value of the second syntax element sps_ptl_dpb_hrd_params_present_flag is used to indicate whether there is a decoded picture buffer (DPB) parameter syntax structure (e.g., dpb_parameters) in the SPS. Determine whether the value of the second syntax element sps_ptl_dpb_hrd_params_present_flag is equal to the second default value (e.g., the second default value is equal to 1); When it is determined that the value of the second syntax element sps_ptl_dpb_hrd_params_present_flag is equal to the second default value, it is determined whether the value of the first syntax element sps_max_sublayers_minus1 is greater than the first default value (e.g., the first default value is equal to 0). When it is determined that the value of the first syntax element sps_max_sublayers_minus1 is greater than the first default value (e.g., the first default value is equal to 0), the value of the third syntax element sps_sublayer_dpb_params_flag is parsed from the bitstream, wherein the value of the third syntax element sps_sublayer_dpb_params_flag is used to control whether there are syntax elements (e.g., max_dec_pic_buffering_minus1[i], max_num_reorder_pics[i] and / or max_latency_increase_plus1[i]) in the DPB parameter syntax structure (e.g. dpb_parameters) in the SPS, where i ranges from 0 to sps_max_sublayers_minus1 – 1, inclusive) when sps_max_sublayers_minus1 is greater than 0.

[0357] 5. The method according to Embodiment 4, wherein the method further comprises: When it is determined that the value of the first syntax element sps_max_sublayers_minus1 is less than or equal to the first default value, or when it is determined that the value of the second syntax element sps_ptl_dpb_hrd_params_present_flag is not equal to the second default value, the value of the third syntax element sps_sublayer_dpb_params_flag is set to the third default value (for example, the third default value is equal to 0 or 1).

[0358] 6. The method according to embodiment 4 or 5, wherein the value of the third syntax element sps_sublayer_dpb_params_flag is decoded in the SPS (in other examples, the value of the third syntax element sps_sublayer_dpb_params_flag is decoded in the picture parameter set (PPS), or the value of the third syntax element sps_sublayer_dpb_params_flag is decoded in the video parameter set (VPS)).

[0359] 7. The method according to any one of embodiments 1 to 6, wherein the value of the first syntax element is decoded in a picture parameter set (PPS) or the value of the first syntax element is decoded in a video parameter set (VPS) (in another example, the value of the second syntax element is decoded in a picture parameter set (PPS) or the value of the second syntax element is decoded in a video parameter set (VPS)).

[0360] 8. The method according to any one of embodiments 1 to 7, wherein when sps_max_sublayers_minus1 equals 0, max_dec_pic_buffering_minus1[0], max_num_reorder_pics[0] and max_latency_increase_plus1[0] are always unique sublayer indicators referring to the SPS.

[0361] 9. A decoding method implemented by a decoding device or an encoding device, wherein the method comprises: The indirect reference layer with index j is determined only from at least one layer with index less than i; When inter-layer prediction is enabled for the current image, the reference image of the current image is obtained in the indirect reference layer with index j; The current image is predicted using the reference image in the indirect reference layer with index j.

[0362] 10. The method according to embodiment 9, wherein determining the indirect reference layer of the layer with index i includes: determining that among at least one layer with an index less than i and greater than j, only the layer with index k is the direct reference layer of the layer with index i, wherein the indirect reference layer with index j is the reference layer of the direct reference layer with index k.

[0363] 11. The method according to embodiment 9, wherein determining the indirect reference layer of the layer with index i includes: determining that among at least one layer with index less than i, only the layer with index k is the direct reference layer of the layer with index i, wherein the indirect reference layer with index j is the reference layer of the direct reference layer with index k.

[0364] 12. A decoding method implemented by a decoding device or an encoding device, wherein the method comprises: When it is determined that among at least one layer with an index less than i and greater than j, only the layer with index k is a direct reference layer of the layer with index i, an indirect reference layer with index j of the layer with index i is determined, wherein the layer with index j is a reference layer of the direct reference layer with index k.

[0365] When inter-layer prediction is enabled for the current image, the reference image of the current image is obtained in the indirect reference layer with index j; The current image is predicted using the reference image in the indirect reference layer with index j.

[0366] 13. A decoding method implemented by a decoding device or an encoding device, wherein the method comprises: The bitstream is acquired, in which the picture parameter set (PPS) is decoded into the bitstream; The number of block columns (NumTileColumns) in the current image (e.g., the current image includes at least one stripe forming a non-rectangular shape) is obtained based on the bitstream. The number of tile rows (NumTileRows) in the current image is obtained based on the bitstream. The value of the variable NumTilesInPic is obtained based on the number of blocks NumTileColumns and the number of blocks NumTileRows. When the value of the variable NumTilesInPic is greater than a preset value (for example, the preset value is 3), the value of the syntax element rect_slice_flag is parsed from the bitstream, wherein the value of the syntax element rect_slice_flag is used to indicate whether to indicate stripe information in the PPS.

[0367] 14. The method according to embodiment 13, wherein the method further comprises: When the value of the variable NumTilesInPic is less than or equal to the preset value, the value of the syntax element rect_slice_flag is set to the fourth default value (for example, the fourth default value is equal to 1).

[0368] 15. The method according to embodiment 13 or 14, wherein the value of the variable NumTilesInPic is equal to .

[0369] 16. The method according to any one of embodiments 13 to 15, wherein the method further comprises: When the value of the syntax element rect_slice_flag is equal to the fifth default value (e.g., the fifth default value is equal to 0), and when the value of the variable NumTilesInPic is greater than the preset value, the value of slice_address of the current slice is parsed from the bitstream, wherein the current slice is included in the current image.

[0370] 17. The method according to embodiment 16, wherein the value of the stripe address is decoded in the stripe header of the current stripe.

[0371] 18. The method according to any one of embodiments 13 to 17, wherein the method further comprises: When the value of the syntax element rect_slice_flag is equal to the fifth default value, and when the value of the variable NumTilesInPic is greater than the preset value, the number of tiles num_tiles_in_slice_minus1 included in the current stripe is obtained.

[0372] 19. The method according to embodiment 18, wherein the number of blocks num_tiles_in_slice_minus1 is decoded in the strip header of the current stripe.

[0373] 20. A decoder (30), wherein the decoder (30) includes processing circuitry for performing the method according to any one of embodiments 1 to 19.

[0374] 21. A computer program product including program code, wherein, when the program code is executed on a computer or processor, the program code is used to perform the method according to any one of embodiments 1 to 19.

[0375] 22. A decoder, wherein the decoder comprises: One or more processors; A non-transitory computer-readable storage medium coupled to the one or more processors and storing a program executed by the one or more processors, wherein, when the program is executed by the one or more processors, the decoder is configured to perform the method according to any one of embodiments 1 to 19.

[0376] 23. A non-transitory computer-readable storage medium comprising program code, wherein when the program code is executed by a computer device, the computer device performs the method according to any one of embodiments 1 to 19.

[0377] The following explains the application of the encoding and decoding methods shown in the above embodiments, as well as the systems that use these applications.

[0378] Figure 8 This is a block diagram of a content delivery system 3100 used to implement content distribution services. The content delivery system 3100 includes a capture device 3102, a terminal device 3106, and optionally a display 3126. The capture device 3102 communicates with the terminal device 3106 via a communication link 3104. This communication link may include the aforementioned communication channel 13. The communication link 3104 includes, but is not limited to, Wi-Fi, Ethernet, cable, wireless (3G / 4G / 5G), USB, or any combination thereof.

[0379] The capture device 3102 is used to generate data and can encode the data using the encoding method shown in the above embodiments. Optionally, the capture device 3102 can distribute the data to a streaming media server (not shown in the figure), which encodes the data and sends the encoded data to the terminal device 3106. The capture device 3102 includes, but is not limited to, cameras, smartphones or tablets, computers or laptops, video conferencing systems, PDAs, in-vehicle devices, or any combination thereof. For example, the capture device 3102 may include the source device 12 as described above. When the data includes video, the video encoder 20 included in the capture device 3102 can actually perform video encoding processing. When the data includes audio (i.e., sound), the audio encoder included in the capture device 3102 can actually perform audio encoding processing. In some practical scenarios, the capture device 3102 distributes the encoded video data and encoded audio data by multiplexing them together. In other practical scenarios, such as in a video conferencing system, the encoded audio data and encoded video data are not multiplexed. The capture device 3102 distributes the encoded audio data and the encoded video data to the terminal device 3106.

[0380] In the content delivery system 3100, the terminal device 310 receives and regenerates encoded data. The terminal device 3106 can be a device with data reception and recovery capabilities, such as a smartphone or tablet computer 3108, a computer or laptop computer 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a television 3114, a set-top box (STB) 3116, a video conferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, an in-vehicle device 3124, or any combination of the above devices capable of decoding the encoded data. For example, the terminal device 3106 may include the destination device 14 described above. When the encoded data includes video, the video decoder 30 included in the terminal device prioritizes video decoding. When the encoded data includes audio, the audio decoder included in the terminal device prioritizes audio decoding.

[0381] For terminal devices with displays, such as smartphones or tablets 3108, computers or laptops 3110, network video recorders (NVRs) / digital video recorders (DVRs) 3112, televisions 3114, personal digital assistants (PDAs) 3122, or in-vehicle devices 3124, the terminal device can feed the decoded data to its display. For terminal devices without displays, such as STBs 3116, video conferencing systems 3118, or video surveillance systems 3120, an external display 3126 is connected to receive and display the decoded data.

[0382] When each device in the system performs encoding or decoding, an image encoding device or an image decoding device as shown in the above embodiments can be used.

[0383] Figure 9This is a schematic diagram of an exemplary structure of terminal device 3106. After terminal device 3106 receives a stream from capture device 3102, protocol processing unit 3202 analyzes the transport protocol of the stream. This protocol includes, but is not limited to, Real-Time Streaming Protocol (RTSP), Hypertext Transfer Protocol (HTTP), HTTP Live Streaming Protocol (HLS), MPEG-DASH, Real-time Transport Protocol (RTP), Real-Time Messaging Protocol (RTMP), or any combination thereof.

[0384] After processing the stream, the protocol processing unit 3202 generates a stream file. The file is then output to the demultiplexing unit 3204. The demultiplexing unit 3204 can separate the multiplexed data into encoded audio data and encoded video data. As described above, for other practical scenarios, such as in a video conferencing system, the encoded audio data and encoded video data are not multiplexed. In this case, the encoded data is sent to the video decoder 3206 and audio decoder 3208 without going through the demultiplexing unit 3204.

[0385] Through demultiplexing, a video elementary stream (ES), an audio ES, and optional subtitles are generated. Video decoder 3206, including video decoder 30 described in the above embodiments, decodes the video ES using the decoding method shown in the above embodiments to generate video frames, and feeds this data to synchronization unit 3212. Audio decoder 3208 decodes the audio ES to generate audio frames, and feeds this data to synchronization unit 3212. Optionally, the video frames can be stored in a buffer before being fed to synchronization unit 3212. Figure 9 (Not shown in the image). Similarly, audio frames can be stored in a buffer (not shown in the image) before being sent to the synchronization unit 3212. Figure 9 (Not shown in the text)

[0386] Synchronization unit 3212 synchronizes video and audio frames and provides the video / audio to video / audio display 3214. For example, synchronization unit 3212 synchronizes the presentation of video and audio information. The information can be decoded using a syntax that includes timestamps related to the presentation of the decoded audio and visual data, as well as timestamps related to the transmission of the data stream itself.

[0387] If the stream includes subtitles, the subtitle decoder 3210 decodes the subtitles, synchronizes the subtitles with the video and audio frames, and provides the video / audio / subtitles to the video / audio / subtitle display 3216.

[0388] This invention is not limited to the system described above; the image encoding device or image decoding device in the above embodiments can be included in other systems such as automotive systems.

[0389] Mathematical operators The mathematical operators used in this application are similar to those used in the C programming language. However, this application precisely defines the results of integer division and arithmetic shift operations, and also defines other operations such as exponentiation and real-value division. Numbering and counting conventions typically start from 0; for example, "first" corresponds to the 0th, "second" corresponds to the 1st, and so on.

[0390] Arithmetic operators The arithmetic operators are defined as follows:

[0391] Logical operators Logical operators are defined as follows: Boolean AND operation on x and y Boolean OR operation on x and y Boolean NOT operation If x is true or not equal to 0, then calculate the value of y; otherwise, calculate the value of z.

[0392] Relational operators The relational operators are defined as follows: >greater than >= Greater than or equal to <less than <= Less than or equal to == equals != does not equal When a relational operator is applied to a syntax element or variable that has already been assigned the value "na" (not applicable), the value "na" is treated as a distinct value of that syntax element or variable. The value "na" is considered not equal to any other value.

[0393] bitwise operators The bitwise operators are defined as follows: Bitwise AND operation. When operating on integer parameters, it operates on the two's complement representation of the integer value. When operating on binary parameters, if it includes fewer bits than another parameter, the shorter parameter is extended by adding more valid bits equal to 0.

[0394] Bitwise OR operation. When operating on integer parameters, it operates on the two's complement representation of the integer value. When operating on binary parameters, if it includes fewer bits than another parameter, the shorter parameter is extended by adding more valid bits equal to 0.

[0395] The bitwise XOR operation operates on integer parameters, performing a two's complement representation of the integer value. When operating on binary parameters, if the parameter has fewer bits than another parameter, the shorter parameter is extended by adding more valid 0 bits.

[0396] This function arithmetically shifts x, in its two's complement representation, to the right by y bits. This function is defined only when y is a non-negative integer. The result of the right shift is that the most significant bit (MSB) is shifted in equal to the MSB of x before the shift operation.

[0397] This function arithmetically shifts x, in its two's complement representation, y bits to the left. This function is defined only if y is a non-negative integer. The result of the left shift is that the least significant bit (LSB) is shifted in by 0.

[0398] Assignment operators The arithmetic operators are defined as follows: =Assignment operator The increment operator (++) is equivalent to x = x + 1. When used for array indices, the variable value is evaluated before the increment operation.

[0399] – Decrease, that is, x – equals x = x – 1; when used for array subscripts, the variable value is calculated before the decrement operation.

[0400] += adds a specified value, that is, x += 3 is equivalent to x = x + 3, and x += (–3) is equivalent to x = x + (–3).

[0401] -= decrements by a specified value, that is, x -= 3 is equivalent to x = x - 3, and x -= (–3) is equivalent to x = x - (–3).

[0402] Range representation The following notation is used to specify the range of values: x takes an integer value from y to z (inclusive), where x, y, and z are integers, and z is greater than y.

[0403] Mathematical functions The definition of a mathematical function is as follows:

[0404] The Asin(x) trigonometric arcsine function operates on the parameter x, which is in the range of -1.0 to 1.0 (inclusive). The output value is... to (Including end values) The unit is radians.

[0405] The Atan(x) trigonometric arctangent function is used to operate on the parameter x, and the output value is... to Within the range (including endpoints), the unit is radians.

[0406]

[0407] Ceil(x) is the smallest integer greater than or equal to x.

[0408]

[0409]

[0410]

[0411] Cos(x) is a trigonometric cosine function that operates on the parameter x, with units in radians.

[0412] Floor(x) is the largest integer less than or equal to x.

[0413]

[0414] Ln(x) returns the natural logarithm of x (the logarithm to the base e, where e is the constant of the base of the natural logarithm, 2.718281828...).

[0415] Log2(x) is the base-2 logarithm of x.

[0416] Log10(x) is the base-10 logarithm of x.

[0417]

[0418]

[0419]

[0420]

[0421] Sin(x) is a trigonometric sine function that operates on the parameter x, with units of radians.

[0422]

[0423] Swap(x, y) = (y, x) Tan(x) is the trigonometric tangent function, which operates on the parameter x and is expressed in radians.

[0424] Operation priority order When parentheses are not used to explicitly indicate the order of precedence in an expression, the following rules apply: Higher-priority operations are evaluated before any lower-priority operations.

[0425] Operations of the same priority are calculated from left to right.

[0426] The table below shows the order of operations from highest to lowest priority. The higher the position in the table, the higher the priority.

[0427] For operators also used in the C programming language, the operator precedence order in this specification is the same as that in the C programming language.

[0428] Table: Operation priority sorted from highest (top of table) to lowest (bottom of table)

[0429] Textual description of logical operations In the text, describe the following logical operation statements in mathematical form: if(condition 0) statement 0 else if(condition 1) statement 1 ...

[0430] statement n It can be described in the following ways: ...the following / ...the following content applies: If condition 0, then statement 0. Otherwise, if condition 1 is true, then statement 1... ... Otherwise (a hint regarding the remaining conditions), then statement n Each "if...otherwise, if...otherwise,..." statement in the text is introduced by "...as follows" or "...the following applies," followed by "if...". The last condition of "if...otherwise, if...otherwise,..." always has an "otherwise,...". Statements containing "if...otherwise, if...otherwise" in the middle can be identified by matching "...as follows" or "...the following applies" with the ending "otherwise...".

[0431] In the text, describe the following logical operation statements in mathematical form: if(condition 0a&&condition 0b) statement 0 else if(condition 1a || condition 1b) statement 1 ... else statement n It can be described in the following ways: ...the following / ...the following content applies: Statement 0 is true if all of the following conditions are true: Condition 0a Condition 0b Otherwise, statement 1 is executed if one or more of the following conditions are met: Condition 1a Condition 1b ... Otherwise, statement n In the text, describe the following logical operation statements in mathematical form: if(condition 0) statement 0 if(condition 1) statement 1 It can be described in the following ways: When condition 0, statement 0 When condition 1 is met, statement 1 is executed.

[0432] Although the embodiments of the present invention are described primarily in relation to video decoding, it should be noted that embodiments of the decoding system 10, encoder 20, and decoder 30 (correspondingly, system 10), as well as other embodiments described herein, can also be used for still image processing or decoding, i.e., processing or decoding a single image in video decoding independent of any previous or consecutive images. Generally, if image processing decoding is limited to a single image 17, only the inter-frame prediction units 244 (encoder) and 344 (decoder) are unavailable. All other functions (also referred to as tools or techniques) of the video encoder 20 and video decoder 30 can also be used for still image processing, such as residual calculation 204 / 304, transform 206, quantization 208, inverse quantization 210 / 310, (inverse) transform 212 / 312, segmentation 262 / 362, intra-frame prediction 254 / 354 and / or loop filtering 220 / 320, entropy coding 270, and entropy decoding 304.

[0433] Embodiments of encoder 20 and decoder 30, and the functions described herein with reference to encoder 20 and decoder 30, can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, these functions can be stored as one or more instructions or code in a computer-readable medium or transmitted via a communication medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium (e.g., a data storage medium), or any communication medium that facilitates the transmission of a computer program from one place to another according to a communication protocol, etc. In this way, a computer-readable medium can generally correspond to (1) a non-transitory tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium accessible by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described herein. A computer program product may include a computer-readable medium.

[0434] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store required program code in the form of instructions or data structures and that can be accessed by a computer. Furthermore, any connection may be appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source via coaxial cable, fiber optic cable, twisted pair and DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair and DSL, or wireless technologies such as infrared, radio, and microwave are also included in the above definition of media. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather refer to non-transient tangible storage media. The disks and optical discs used herein include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), and Blu-ray discs, wherein disks typically reproduce data magnetically, while optical discs utilize lasers to reproduce data optically. Combinations of the above items should also be included within the scope of computer-readable media.

[0435] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), one or more general-purpose microprocessors, one or more application-specific integrated circuits (ASICs), one or more field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein can refer to any of the above-described structures or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the various functions described herein can be provided within dedicated hardware and / or software modules for encoding and decoding, or incorporated into combined codecs. Moreover, these techniques can be fully implemented in one or more circuit or logic elements.

[0436] The techniques of this invention can be implemented in a variety of devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or a set of ICs (e.g., chipsets). The various components, modules, or units described herein are intended to emphasize functional aspects of the apparatus for performing the disclosed techniques, but do not necessarily need to be implemented by different hardware units. In fact, as described above, the various units can be combined with suitable software and / or firmware within a codec hardware unit, or provided as a collection of interoperable hardware units including one or more processors as described above.

Claims

1. A decoding method, comprising: Obtain the first syntax element and the third syntax element from the sequence parameter set (SPS) in the video bitstream. The value of the first syntax element is used to indicate whether the decoded picture buffer (DPB) parameter syntax structure appears in the SPS. The SPS contains syntax elements applied to the video sequence. The value of the third syntax element plus 1 indicates the maximum number of temporal sublayers in the video sequence. When the value of the first syntax element indicates that a DPB parameter syntax structure appears in the SPS: When the value of the third syntax element is greater than 0, the second syntax element is obtained from the SPS, or when the second syntax element does not appear, the value of the second syntax element is inferred to be 0, wherein the value of the second syntax element is used to control whether the syntax element in the DPB parameter syntax structure appears; and, the DPB parameter syntax structure with the value of the third syntax element as the first input parameter and the value of the second syntax element as the second input parameter is obtained from the SPS.

2. The method according to claim 1, characterized in that, The value of the first syntax element is used to indicate whether the DPB parameter syntax structure appears in the SPS, including: A value of 1 for the first syntax element indicates that a DPB parameter syntax structure appears in the SPS; or, The value of the first syntax element is 0, indicating that the DPB parameter syntax structure does not appear in the SPS.

3. The method according to claim 1, characterized in that, The value of the second syntax element controls whether syntax elements appear in the DPB parameter syntax structure, including: When the value of the second syntax element is 0, in the DPB parameter syntax structure, only the highest sub-level has indicators for the corresponding syntax elements max_dec_pic_buffering_minus1, max_num_reorder_pics, and max_latency_increase_plus1; or, When the value of the second syntax element is 1, in the DPB parameter syntax structure, each sub-layer has an indicator of the corresponding syntax elements max_dec_pic_buffering_minus1, max_num_reorder_pics, and max_latency_increase_plus1. Where: max_dec_pic_buffering_minus1 plus 1 represents the maximum required size of DPB, in units of image storage buffer; max_num_reorder_pics represents the maximum allowed number of images in the output layer set OLS whose decoding order can be before an image in the OLS and whose output order can be after that image; max_latency_increase_plus1 is used to calculate the value of MaxLatencyPictures when it is not equal to 0, and the value of MaxLatencyPictures represents the maximum number of images in the OLS whose output order can be before an image in the OLS and whose decoding order can be after that image.

4. The method according to claim 1, characterized in that, The first syntax element is represented as sps_ptl_dpb_hrd_params_present_flag in the video stream, the third syntax element is represented as sps_max_sublayers_minus1 in the video stream, and the second syntax element is represented as sps_sublayer_dpb_params_flag in the video stream.

5. An encoding method, comprising: The first syntax element and the third syntax element are encoded into the sequence parameter set (SPS) in the video bitstream. The value of the first syntax element is used to indicate whether the decoded picture buffer (DPB) parameter syntax structure appears in the SPS. The SPS contains syntax elements applied to the video sequence. The value of the third syntax element plus 1 indicates the maximum number of temporal sublayers in the video sequence. When the value of the first syntax element indicates that a DPB parameter syntax structure appears in the SPS: When the value of the third syntax element is greater than 0, the second syntax element is encoded into the SPS; wherein, the value of the second syntax element is used to control whether the syntax element in the DPB syntax structure appears; and, the DPB parameter syntax structure with the value of the third syntax element as the first input parameter and the value of the second syntax element as the second input parameter is encoded into the SPS.

6. The method according to claim 5, characterized in that, The value of the first syntax element is used to indicate whether the DPB parameter syntax structure appears in the SPS, including: A value of 1 for the first syntax element indicates that a DPB parameter syntax structure appears in the SPS; or, The value of the first syntax element is 0, indicating that the DPB parameter syntax structure does not appear in the SPS.

7. The method according to claim 5, characterized in that, The value of the second syntax element controls whether syntax elements appear in the DPB parameter syntax structure, including: When the value of the second syntax element is 0, in the DPB parameter syntax structure, only the highest sub-level has indicators for the corresponding syntax elements max_dec_pic_buffering_minus1, max_num_reorder_pics, and max_latency_increase_plus1; or, When the value of the second syntax element is 1, in the DPB parameter syntax structure, each sub-layer has an indicator of the corresponding syntax elements max_dec_pic_buffering_minus1, max_num_reorder_pics, and max_latency_increase_plus1. Where: max_dec_pic_buffering_minus1 plus 1 represents the maximum required size of DPB, in units of image storage buffer; max_num_reorder_pics represents the maximum allowed number of images in the output layer set OLS whose decoding order can be before an image in the OLS and whose output order can be after that image; max_latency_increase_plus1 is used to calculate the value of MaxLatencyPictures when it is not equal to 0, and the value of MaxLatencyPictures represents the maximum number of images in the OLS whose output order can be before an image in the OLS and whose decoding order can be after that image.

8. The method according to claim 5, characterized in that, The first syntax element is represented as sps_ptl_dpb_hrd_params_present_flag in the video stream, the third syntax element is represented as sps_max_sublayers_minus1 in the video stream, and the second syntax element is represented as sps_sublayer_dpb_params_flag in the video stream.

9. An encoder, characterized in that, Includes processing circuitry for performing the method according to any one of claims 5 to 8.

10. A decoder, characterized in that, Includes processing circuitry for performing the method according to any one of claims 1 to 4.

11. A computer program product, characterized in that, Includes program code, which, when executed by a computer or processor, is used to perform the method according to any one of claims 1 to 8.

12. A decoder, characterized in that, include: One or more processors; as well as A computer-readable storage medium coupled to the one or more processors and storing a program executed by the one or more processors, wherein, when the program is executed by the one or more processors, the decoder is configured to perform the method according to any one of claims 1 to 4.

13. An encoder, characterized in that, The encoder includes: One or more processors; A computer-readable storage medium coupled to and storing a program executable by the one or more processors, wherein, when the program is executed by the one or more processors, the encoder is configured to perform the method according to any one of claims 5 to 8.

14. A computer-readable storage medium including program code, characterized in that, When the program code is executed by a computer device, the computer device performs the method according to any one of claims 1 to 8.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a video stream, which is obtained by the method according to any one of claims 5 to 8, or the video stream includes: A sequence parameter set (SPS); The SPS includes a first syntax element and a third syntax element. The value of the first syntax element is used to indicate whether the decoded picture buffer (DPB) parameter syntax structure appears in the SPS. The SPS includes syntax elements applied to the video sequence. The value of the third syntax element plus 1 indicates the maximum number of temporal sub-layers in the video sequence. When the value of the first syntax element indicates that a DPB parameter syntax structure appears in the SPS: When the value of the third syntax element is greater than 0, the SPS also includes a second syntax element, or when the second syntax element does not appear, the value of the second syntax element should be inferred to be 0, wherein the value of the second syntax element is used to control whether the syntax element in the DPB parameter syntax structure appears; and the SPS also includes a DPB parameter syntax structure with the value of the third syntax element as the first input parameter and the value of the second syntax element as the second input parameter.

16. The storage medium according to claim 15, characterized in that, The value of the first syntax element is used to indicate whether the DPB parameter syntax structure appears in the SPS, including: A value of 1 for the first syntax element indicates that a DPB parameter syntax structure appears in the SPS; or, The value of the first syntax element is 0, indicating that the DPB parameter syntax structure does not appear in the SPS.

17. The storage medium according to claim 15, characterized in that, The value of the second syntax element controls whether syntax elements appear in the DPB parameter syntax structure, including: When the value of the second syntax element is 0, in the DPB parameter syntax structure, only the highest sub-level has indicators for the corresponding syntax elements max_dec_pic_buffering_minus1, max_num_reorder_pics, and max_latency_increase_plus1; or, When the value of the second syntax element is 1, in the DPB parameter syntax structure, each sub-layer has an indicator of the corresponding syntax elements max_dec_pic_buffering_minus1, max_num_reorder_pics, and max_latency_increase_plus1. Where: max_dec_pic_buffering_minus1 plus 1 represents the maximum required size of DPB, in units of image storage buffer; max_num_reorder_pics represents the maximum allowed number of images in the output layer set OLS whose decoding order can be before an image in the OLS and whose output order can be after that image; max_latency_increase_plus1 is used to calculate the value of MaxLatencyPictures when it is not equal to 0, and the value of MaxLatencyPictures represents the maximum number of images in the OLS whose output order can be before an image in the OLS and whose decoding order can be after that image.

18. The storage medium according to claim 15, characterized in that, The first syntax element is represented as sps_ptl_dpb_hrd_params_present_flag in the video stream, the third syntax element is represented as sps_max_sublayers_minus1 in the video stream, and the second syntax element is represented as sps_sublayer_dpb_params_flag in the video stream.

Images