Corresponding methods for encoders, decoders and flexible profile configurations

Abstract

A method for decoding a video stream, implemented by a decoding device, wherein the video stream includes data representing one or more sequence parameter set (SPS) flags, the method comprising: for one of the one or more SPS flags, obtaining two variables of the SPS flag by parsing the video stream, wherein the value of one of the two variables is the minimum allowed value of the SPS flag, and the value of the other variable is the maximum allowed value of the SPS flag; obtaining the SPS flag by parsing the video stream; evaluating whether the value of the SPS flag is valid based on the values ​​of the two variables; and decoding the video stream based on the value of the SPS flag when the value of the SPS flag is valid.

Description

[0001] Cross-referencing related applications

[0002] This patent application claims priority to international patent application PCT / EP2020 / 050077, filed on January 3, 2020. The disclosure of the aforementioned patent application is incorporated herein by reference in its entirety. Technical Field

[0003] The embodiments of this application (invention) generally relate to the field of image processing, and more specifically to high-level syntax and bitstream consistency. Background Technology

[0004] Video decoding (video encoding and decoding) is widely used in digital video applications, such as broadcast digital television, video transmission based on the Internet and mobile networks, real-time conversational applications such as video chat and video conferencing, DVD and Blu-ray discs, video content capture and editing systems, and portable cameras for security applications.

[0005] Even with shorter videos, a large amount of video data needs to be described, which can be challenging when transmitting data over bandwidth-constrained communication networks or otherwise. Therefore, video data is typically compressed before transmission over modern telecommunications networks. Video size can also be an issue when storing video on storage devices due to potentially limited memory resources. Video compression devices typically encode video data using software and / or hardware at the source side before transmission or storage, reducing the amount of data required to represent a digital video image. The compressed data is then received at the destination by a video decompression device used to decode the video data. Given limited network resources and the growing demand for higher video quality, there is a need to improve compression and decompression techniques to increase compression ratios with minimal impact on image quality. Summary of the Invention

[0006] Embodiments of this application provide encoding and decoding apparatus and methods as described in the independent claims.

[0007] The above and other objectives are achieved by the subject matter claimed in the independent claims. Other implementations are apparent from the dependent claims, the specification, and the drawings.

[0008] Specific embodiments are summarized in the appended independent claims, and other embodiments are summarized in the dependent claims.

[0009] This invention provides a method for decoding a video stream implemented by a decoding device, wherein the video stream includes data representing one or more sequence parameter set (SPS) flags. The method includes: for one of the one or more SPS flags, obtaining two variables of the SPS flag by parsing the video stream, wherein the value of one of the two variables is the minimum allowed value of the SPS flag, and the value of the other variable is the maximum allowed value of the SPS flag; obtaining the value of the SPS flag by parsing the video stream; evaluating whether the value of the SPS flag is valid based on the values ​​of the two variables; and decoding the video stream based on the value of the SPS flag when the value of the SPS flag is valid.

[0010] In the following text, the terms sps and SPS, sps flag and SPS flag, sps flag value and SPS flag value should be understood as having the same meaning.

[0011] Therefore, this invention introduces an additional syntax element to SPS that is designed to limit the SPS flag from the lower bound (i.e., the minimum allowed value of the SPS flag).

[0012] The above method may further include: stopping decoding the video stream when the value of the SPS flag is invalid.

[0013] In the above method, the two variables of the SPS flag can be Boolean variables, which can be encapsulated in two syntax elements (e.g., toolset_idc_H and toolset_idc_L), where the two variables can correspond to specific bits within the two syntax elements.

[0014] In the above method, the values ​​of the two variables can come from the set {0,1}.

[0015] In the above method, the value of the SPS flag can be equal to one of the values ​​of the variable.

[0016] In the above method, the values ​​of the two variables can come from the set {0,1}, and can be represented as l. i and h i And can be indicated in the bitstream before the SPS flag, where l i ≤sps i ≤h i SPS i This is the value of the SPS flag.

[0017] In the above method, for all SPS flags of one or more SPS flags, two variables in the set {0,1} can be indicated by two vectors having elements in the set {0,1}: toolset_idc_l = l0l1,…,l i ,…l n-1 And toolset_idc_h = h0Hh1,…,h i ,…h n-1 , where n is the number of one or more SPS flags, and i is an integer.

[0018] It should be noted that in existing technologies, mechanisms such as toolset_idc only support limiting the SPS flag value to 0, and there is no method to limit the SPS flag to 1.

[0019] In view of this, the present invention extends the toolset_idc mechanism to support functionality from both low and high boundaries, thereby overcoming this problem. In other words, the solution provided by the present invention can be used to force 0 and 1 SPS flag values.

[0020] The present invention also provides a method for encoding a video stream implemented by an encoding device, wherein the video stream includes data representing one or more sequence parameter set (SPS) flags, and the method includes:

[0021] For one of the one or more SPS flags, two variables of the SPS flag are set respectively, wherein the value of one of the two variables is the minimum allowed value of the SPS flag, and the value of the other variable is the maximum allowed value of the SPS flag; the value of the SPS flag is determined based on the two variables of the SPS flag; the value of the SPS flag and the two variables of the SPS flag are encoded into the video bitstream.

[0022] The two variables that set the SPS flag are based on the required decoding capability.

[0023] The above method may also include: determining the value of the SPS flag to be 1 when the value of a variable that serves as the minimum permissible value of the SPS flag is equal to 1.

[0024] The above method may also include: determining the value of the SPS flag to be 0 when the value of a variable that is the maximum allowed value of the SPS flag is equal to 0.

[0025] The above method may also include: when the values ​​of both variables of the SPS flag are equal to 1, determining that the value of the SPS flag is 1 respectively.

[0026] The above method may also include: when the values ​​of both variables of the SPS flag are equal to 0, determining that the value of the SPS flag is 0 respectively.

[0027] In the above method, the two variables of the SPS flag can be Boolean variables, which can be encapsulated in two syntax elements respectively, and the two variables can correspond to specific bits within the two syntax elements respectively.

[0028] In the above method, the values ​​of the two variables can come from the set {0,1}.

[0029] In the above method, the value of the SPS flag can be equal to one of the values ​​of the variable.

[0030] In the above method, the values ​​of the two variables can come from the set {0,1}, and can be represented as l. i and h i And can be indicated in the bitstream before the SPS flag, where l i ≤sps i ≤h i SPS i This is the value of the SPS flag.

[0031] In the above method, for all SPS flags of one or more SPS flags, two variables in the set {0,1} can be indicated by two vectors having elements in the set {0,1}: toolset_idc_l = l0l1,…,l i ,…l n-1 And toolset_idc_h = h0Hh1,…,h i ,…h n-1 , where n is the number of one or more SPS flags, and i is an integer.

[0032] The present invention also discloses an encoder, including a processing circuit for performing the above-described method.

[0033] The present invention also discloses a decoder, including a processing circuit for performing the above-described method.

[0034] The present invention also discloses a computer program product including program code, which, when executed in a computer or processor, is used to perform the above-described method.

[0035] The present invention also discloses a decoder, comprising: one or more processors; a non-transitory computer-readable storage medium coupled to the processors and storing a program executed by the processors, wherein the program, when executed by the processors, causes the decoder to perform the methods described above.

[0036] The present invention also discloses an encoder, comprising: one or more processors; a non-transitory computer-readable storage medium coupled to the processors and storing a program executed by the processors, wherein the program, when executed by the processors, causes the encoder to perform the methods described above.

[0037] The present invention also discloses a non-transitory computer-readable medium carrying program code, which, when executed by a computer device, causes the computer device to perform the above-described method.

[0038] The present invention also discloses a non-transient storage medium comprising an encoded bitstream decoded by an image decoding device, the bitstream being generated by dividing frames of a video signal or image signal into multiple blocks and including multiple SPS flags, wherein the bitstream further includes two variables of the SPS flags, one of the variables having a value that is the minimum allowed value of the SPS flag, and the other variable having a value that is the maximum allowed value of the SPS flag.

[0039] The present invention also discloses a decoder for decoding a video stream, wherein the video stream includes data representing one or more sequence parameter set (SPS) flags, and the decoder includes: a first acquisition unit, configured to obtain two variables of one of the one or more SPS flags by parsing the video stream, wherein the value of one of the two variables is the minimum allowed value of the SPS flag, and the value of the other variable is the maximum allowed value of the SPS flag; a second acquisition unit, configured to obtain the value of the SPS flag by parsing the video stream; an evaluation unit, configured to evaluate whether the value of the SPS flag is valid based on the values ​​of the two variables; and a decoding unit, configured to decode the video stream based on the value of the SPS flag when the value of the SPS flag is valid.

[0040] The present invention also discloses an encoder for encoding a video stream, wherein the video stream includes data representing one or more sequence parameter set (SPS) flags, and the encoder includes:

[0041] An instruction unit is configured to set two variables for one of the one or more SPS flags, wherein the value of one of the two variables is the minimum allowed value of the SPS flag, and the value of the other variable is the maximum allowed value of the SPS flag.

[0042] A determining unit is configured to determine the value of the SPS flag based on the two variables of the SPS flag;

[0043] An encoding unit is used to encode the value of the SPS flag and the two variables of the SPS flag into the video stream.

[0044] The following drawings and description illustrate one or more embodiments in detail. Other features, objects, and advantages will be apparent from the description, drawings, and claims. Attached Figure Description

[0045] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings. In the drawings:

[0046] Figure 1A A block diagram illustrating an example of a video decoding system for implementing embodiments of the present invention;

[0047] Figure 1B A block diagram of another example of a video decoding system for implementing embodiments of the present invention;

[0048] Figure 2 A block diagram illustrating an example video encoder used to implement embodiments of the present invention;

[0049] Figure 3 This is a block diagram of an exemplary structure for implementing embodiments of the present invention, specifically a video decoder.

[0050] Figure 4 A block diagram of an example encoding or decoding device;

[0051] Figure 5 A block diagram of another example of an encoding or decoding device;

[0052] Figure 6 A block diagram of an exemplary structure for a content delivery system 3100 used to implement content distribution services;

[0053] Figure 7 A block diagram of an example structure for a terminal device;

[0054] Figure 8 A flowchart of a method for decoding a video stream according to an embodiment of the present invention is shown;

[0055] Figure 9 An embodiment of the present invention is shown, which provides a decoder for decoding a video stream;

[0056] Figure 10 A flowchart of a method for encoding a video stream according to an embodiment of the present invention is shown;

[0057] Figure 11 An encoder for encoding video streams, provided by an embodiment of the present invention, is illustrated.

[0058] In the following text, unless otherwise expressly stated, the same reference numerals refer to the same or at least functionally equivalent features. Detailed Implementation

[0059] In the following description, reference is made to the accompanying drawings, which form part of this invention and illustrate specific aspects of embodiments of the invention or to the drawings in which embodiments of the invention may be used. It should be understood that embodiments of the invention may be used in other aspects and may include structural or logical variations not depicted in the drawings. Therefore, the following detailed description should not be construed in a limiting sense, and the scope of the invention is defined by the appended claims.

[0060] For example, it is understood that the disclosure relating to the described method can also apply to the corresponding device or system for performing the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more units (e.g., functional units) to perform the described one or more method steps (e.g., one unit performs one or more steps, or multiple units each perform one or more of a plurality of steps), even if such one or more units are not explicitly described or illustrated in the drawings. On the other hand, for example, if a specific apparatus is described according to one or more units (e.g., functional units), the corresponding method may include a step to perform the function of one or more units (e.g., one step performs the function of one or more units, or multiple steps each perform the function of one or more of a plurality of units), even if such one or more units are not explicitly described or illustrated in the drawings. Furthermore, it is understood that, unless otherwise explicitly stated, features of the various exemplary embodiments and / or aspects described herein can be combined with each other.

[0061] Video decoding generally refers to the processing of a sequence of images that constitute a video or video sequence. In the field of video decoding, the terms "frame" and "picture / image" can be used synonymously. Video decoding (or generally decoding) consists of two parts: video encoding and video decoding. Video encoding is performed on the source side and typically involves processing (e.g., compressing) the raw video image to reduce the amount of data required to represent the video image (thus enabling more efficient storage and / or transmission). Video decoding is performed on the destination side and typically involves inverse processing relative to the encoder to reconstruct the video image. The "decoding" of the video image (or generally referred to as an image) involved in the embodiments should be understood as involving the "encoding" or "decoding" of the video image or the corresponding video sequence. The encoding and decoding parts are also collectively referred to as codec (encoding and decoding, CODEC).

[0062] In lossless video decoding, the original video image can be reconstructed, meaning the reconstructed video image has the same quality as the original (assuming no transmission loss or other data loss during storage or transmission). In lossy video decoding, further compression, such as quantization, reduces the amount of data representing the video image, and the decoder cannot completely reconstruct the video image; that is, the quality of the reconstructed video image is lower or worse than the quality of the original video image.

[0063] Several video coding standards belong to the "lossy hybrid video codec" group (i.e., combining spatial and temporal prediction in the sample domain with 2D transform decoding in the transform domain for applying quantization). Each image in a video sequence is typically segmented into a set of non-overlapping blocks, usually decoded at the block level. In other words, the encoder typically processes the video at the block (video block) level, i.e., encoding, for example, generating prediction blocks through spatial (intra-frame) prediction and / or temporal (inter-frame) prediction; subtracting the prediction blocks from the current block (the block currently being processed / to be processed) to obtain residual blocks; transforming and quantizing the residual blocks in the transform domain to reduce the amount of data to be sent (compressed), while the decoder applies the inverse processing relative to the encoder to the encoded or compressed blocks to reconstruct the representation of the current block. Furthermore, the processing steps of the encoder and decoder are identical, such that the encoder and decoder generate the same predictions (e.g., intra-frame and inter-frame predictions) and / or reconstructions for processing, i.e., decoding subsequent blocks.

[0064] In the following embodiments, according to Figure 1A , 1B to Figure 3 The video decoding system 10, video encoder 20, and video decoder 30 are described.

[0065] Figure 1A This is a schematic block diagram of an example decoding system 10, such as a video decoding system 10 (or simply decoding system 10) that can utilize the technology of this application. The video encoder 20 (or simply encoder 20) and video decoder 30 (or simply decoder 30) in the video decoding system 10 represent examples of devices that can be used to perform various technologies according to the various examples described in this application.

[0066] like Figure 1A As shown, the decoding system 10 includes a source device 12, for example, the source device 12 is used to provide encoded image data 21 to the destination device 14 for decoding encoded image data 13.

[0067] The source device 12 includes an encoder 20 and may additionally (optionally) include an image source 16, a preprocessor (or preprocessing unit) 18 (e.g., an image preprocessor 18), and a communication interface or communication unit 22.

[0068] Image source 16 may include or may be any type of image capture device, such as a camera for capturing real-world images, and / or any type of image generation device, such as a computer graphics processor for generating computer-animated images, or any other type of device for obtaining and / or providing real-world images, computer-generated images (e.g., screen content, virtual reality (VR) images) and / or any combination thereof (e.g., augmented reality (AR) images). Image source may be any type of memory / storage for storing any of the aforementioned images.

[0069] To distinguish between the processing performed by the preprocessor 18 and the preprocessing unit 18, the image or image data 17 may also be referred to as the raw image or raw image data 17.

[0070] The preprocessor 18 receives (raw) image data 17 and preprocesses the image data 17 to obtain a preprocessed image 19 or preprocessed image data 19. For example, the preprocessing performed by the preprocessor 18 may include trimming, color format conversion (e.g., from RGB to YCbCr), color correction, or noise reduction. It is understood that the preprocessing unit 18 may be an optional component.

[0071] Video encoder 20 is used to receive preprocessed image data 19 and provide encoded image data 21 (e.g., hereinafter referred to as...). Figure 2 (Further detailed description).

[0072] The communication interface 22 of the source device 12 can be used to receive encoded image data 21 and send the encoded image data 21 (or any other processed version thereof) to another device (e.g., destination device 14) or any other device via the communication channel 13 for storage or direct reconstruction.

[0073] Destination device 14 includes decoder 30 (e.g., video decoder 30), and may also (optionally) include communication interface or communication unit 28, post-processor 32 (or post-processing unit 32) and display device 34.

[0074] The communication interface 28 of the destination device 14 is used to receive encoded image data 21 (or any other processed version thereof), for example, directly from the source device 12 or any other source (e.g., a storage device such as an encoded image data storage device), and to provide the encoded image data 21 to the decoder 30.

[0075] Communication interfaces 22 and 28 can be used to send or receive encoded image data 21 or encoded data 13 via a direct communication link (e.g., a direct wired or wireless connection) between source device 12 and destination device 14 or via any type of network (e.g., a wired network, a wireless network or any combination thereof, or any type of private and public network or any type of combination thereof).

[0076] For example, the communication interface 22 can be used to encapsulate the encoded image data 21 into a suitable format (e.g., data packets) and / or process the encoded image data through any type of transmission encoding or processing method for transmission over a communication link or communication network.

[0077] For example, the communication interface 28 corresponding to the communication interface 22 can be used to receive transmitted data and process the transmitted data using any type of corresponding transmission decoding or processing and / or decapsulation to obtain encoded image data 21.

[0078] Both communication interface 22 and communication interface 28 can be configured as Figure 1A The communication channel 13, from source device 12 to destination device 14, is a one-way communication interface, or configured as a two-way communication interface, and can be used to send and receive messages, etc., to establish connections, acknowledge and exchange any other information related to the communication link and / or data transmission (e.g., encoded image data transmission).

[0079] Decoder 30 is used to receive encoded image data 21 and provide decoded image data 31 or decoded image 31 (e.g., according to the following). Figure 3 or Figure 5 (Further detailed description).

[0080] The post-processor 32 of the destination device 14 is used to post-process the decoded image data 31 (also referred to as reconstructed image data) (e.g., decoded image 31) to obtain post-processed image data 33 (e.g., post-processed image 33). For example, the post-processing performed by the post-processing unit 32 may include color format conversion (e.g., from YCbCr to RGB), color correction, trimming or resampling, or any other processing, such as preparing the decoded image data 31 for display by a display device 34, etc.

[0081] The display device 34 of the destination device 14 is used to receive the post-processed image data 33 to display the image to a user or viewer. The display device 34 can be or may include any type of display (e.g., integrated or external display or screen) to represent the reconstructed image. For example, the display may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display.

[0082] although Figure 1A Source device 12 and destination device 14 are described as separate devices; however, device embodiments may also include two devices or two functions, namely source device 12 or its corresponding function and destination device 14 or its corresponding function. In these embodiments, source device 12 or its corresponding function and destination device 14 or its corresponding function may be implemented using the same hardware and / or software or using separate hardware and / or software or any combination thereof.

[0083] According to the description, Figure 1A The presence and (accurate) division of different units or functions in the source device 12 and / or destination device 14 shown may vary depending on the actual device and application, which is obvious to those skilled in the art.

[0084] Encoder 20 (e.g., video encoder 20) or decoder 30 (e.g., video decoder 30), or both encoder 20 and decoder 30, can be transmitted via, for example, Figure 1B The processing circuitry shown can be implemented using, for example, one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, dedicated video decoding processors, or any combination thereof. Encoder 20 can be implemented by processing circuitry 46 to embody the combination... Figure 2 The encoder 20 may be implemented using the various modules described herein and / or any other encoder system or subsystem described herein. The decoder 30 may be implemented by processing circuitry 46 to embody the combination of... Figure 3The various modules described in decoder 30 and / or any other decoder system or subsystem described herein. The processing circuitry can be used to perform the various operations described below. Figure 5 As shown, if the technology is partially implemented in software, the device can store the software instructions in a suitable non-transitory computer-readable storage medium, and can use one or more processors to execute the instructions in the hardware to perform the technology of the present invention. Either the video encoder 20 and the video decoder 30 can be integrated into a single device as part of a combined codec (encoder / decoder, CODEC), such as... Figure 1B As shown.

[0085] Source device 12 and destination device 14 can include any of a variety of devices, including any type of handheld or fixed device, such as a laptop or tablet computer, mobile phone, smartphone, tablet / tabletcomputer, camera, desktop computer, set-top box, television, display device, digital media player, video game console, video streaming device (e.g., content service server or content distribution server), broadcast receiver device, broadcast transmitter device, etc., and may or may not use any type of operating system. In some cases, source device 12 and destination device 14 can be configured for wireless communication. Therefore, source device 12 and destination device 14 can be wireless communication devices.

[0086] In some cases, Figure 1A The video decoding system 10 shown is merely an example, and the technology of this application can be applied to video decoding setups (e.g., video encoding or video decoding) that do not necessarily involve any data communication between the encoding and decoding devices. In other examples, data is retrieved from local memory, streamed over a network, etc. The video encoding device may encode data and store it in memory, and / or the video decoding device may retrieve data from memory and decode it. In some examples, encoding and decoding are performed by devices that do not communicate with each other but simply encode data into memory and / or retrieve data from memory and decode it.

[0087] For ease of description, this document refers to the high-efficiency video coding (HEVC) or versatile video coding (VVC) (next-generation video coding standard) reference software developed by the ITU-T Video Coding Experts Group (VCEG) and the Joint Collaboration Team on Video Coding (JCT-VC) of the ISO / IEC Moving Picture Experts Group (MPEG). Those skilled in the art will understand that the embodiments of this invention are not limited to HEVC or VVC.

[0088] Encoders and Encoding Methods

[0089] Figure 2 This is a schematic block diagram of an exemplary video encoder 20 used to implement the technology of this application. Figure 2 In the example, the video encoder 20 includes an input terminal 201 (or input interface 201), a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a loop filtering unit 220, a decoded picture buffer (DPB) 230, a mode selection unit 260, an entropy coding unit 270, and an output terminal 272 (or output interface 272). The mode selection unit 260 may include an inter-frame prediction unit 244, an intra-frame prediction unit 254, and a segmentation unit 262. The inter-frame prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). Figure 2 The video encoder 20 shown can also be called a hybrid video encoder or a video encoder based on a hybrid video codec.

[0090] The residual calculation unit 204, transform processing unit 206, quantization unit 208, and mode selection unit 260 can form the forward signal path of the encoder 20, while the inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, buffer 216, loop filter 220, decoded picture buffer (DPB) 230, inter-frame prediction unit 244, and intra-frame prediction unit 254 can form the backward signal path of the video encoder 20. The backward signal path of the video encoder 20 corresponds to the decoder (see [link to decoder]). Figure 3The signal path of the video decoder 30 in the video encoder 20. The inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, loop filter 220, decoded picture buffer (DPB) 230, inter-frame prediction unit 244 and intra-frame prediction unit 254 also constitute the "built-in decoder" of the video encoder 20.

[0091] Image and image segmentation (images and patches)

[0092] Encoder 20 can be used to receive image 17 (or image data 17) via input terminal 201, such as images in an image sequence forming a video or video sequence. The received image or image data can also be a pre-processed image 19 (or pre-processed image data 19). For simplicity, it is referred to as image 17 in the following description. Image 17 can also be referred to as the current image or the image to be decoded (especially in video decoding to distinguish the current image from other images (e.g., previously encoded and / or decoded images) in the same video sequence (i.e., a video sequence that also includes the current image).

[0093] A (digital) image is, or can be viewed as, a two-dimensional array or matrix of samples with intensity values. Samples in the array are also called pixels (short for image elements). The number of samples in the horizontal and vertical directions (or axes) of the array or image defines the image size and / or resolution. Color is typically represented using three color components; that is, an image can be represented as or comprise three arrays of samples. In RGB format or color space, an image includes corresponding arrays of red, green, and blue samples. However, in video decoding, each pixel is typically represented by a luminance and chrominance format or in a color space, for example, YCbCr, including the luminance component represented by Y (sometimes also L) and the two chrominance components represented by Cb and Cr. The luminance component Y represents the brightness or grayscale intensity (e.g., in a grayscale image), while the two chrominance components Cb and Cr represent the chrominance or color information components. Accordingly, a YCbCr format image includes a luminance sample array of luminance sample values ​​(Y) and two chrominance sample arrays of chrominance values ​​(Cb and Cr). An RGB format image can be converted or transformed to YCbCr format, and vice versa; this process is also known as color conversion or color transformation. If the image is black and white, it may only include the luminance sample array. Accordingly, for example, an image may be a black and white luminance sample array or a luminance sample array and two corresponding chrominance sample arrays in 4:2:0, 4:2:2, and 4:4:4 color formats.

[0094] Embodiments of the video encoder 20 may include an image segmentation unit ( Figure 2 (Not shown in the image) is used to segment image 17 into multiple (typically non-overlapping) image blocks 203. These blocks may also be referred to as root blocks, macroblocks (H.264 / AVC), coding tree blocks (CTBs), or coding tree units (CTUs) (H.265 / HEVC and VVC). The image segmentation unit can be used to apply the same block size and a corresponding grid with a defined block size to all images in a video sequence, or to vary the block size between images, subsets of images, or groups of images, and segment each image into corresponding blocks.

[0095] In other embodiments, the video encoder may be used to directly receive blocks 203 of image 17, such as one, several, or all of the blocks that make up image 17. Image block 203 may also be referred to as the current image block or the image block to be decoded.

[0096] Although image patch 203 is smaller than image 17, like image 17, image patch 203 is also, or can be considered as, a two-dimensional array or matrix of samples with intensity values ​​(sample values). In other words, depending on the applied color format, patch 203 may include one sample array (e.g., a luminance array in the case of black and white for image 17, or a luminance or chrominance array in the case of color for image 17) or three sample arrays (e.g., one luminance array and two chrominance arrays in the case of color for image 17) or any other number and / or type of array. The number of samples in the horizontal and vertical directions (or axes) of patch 203 defines the size of patch 203. Accordingly, a patch may be an M×N (M columns × N rows) sample array, or an M×N transform coefficient array, etc.

[0097] Figure 2 The embodiment of the video encoder 20 shown can be used to encode the image 17 block by block, for example, to encode and predict in blocks 203.

[0098] Figure 2 The embodiment of the video encoder 20 shown can also be used to segment and / or encode images using slices (also known as video slices), wherein one or more slices (typically non-overlapping) can be used to segment or encode an image. Each slice may include one or more blocks (e.g., CTUs) or one or more groups of blocks (e.g., tiles (H.265 / HEVC and VVC) or bricks (VVC)).

[0099] Figure 2The embodiment of the video encoder 20 shown can also be used to segment and / or encode images using stripe / block groups (also known as video block groups) and / or blocks (also known as video blocks), wherein one or more stripe / block groups (typically non-overlapping) can be used to segment or encode images. Each stripe / block group may include one or more blocks (e.g., CTUs) or one or more blocks, wherein each block may be rectangular or the like, and may include one or more blocks (e.g., CTUs), such as complete or partial blocks.

[0100] Residual calculation

[0101] The residual calculation unit 204 can be used to calculate the residual block 205 (also called residual 205) based on the image block 203 and the prediction block 265 (the prediction block 265 is described in detail below) in the following manner: the residual block 205 in the sample domain is obtained by subtracting the sample value of the prediction block 265 from the sample value of the image block 203 on a sample-by-sample (pixel-by-pixel) basis.

[0102] Transformation

[0103] The transformation processing unit 206 can be used to perform discrete cosine transform (DCT) or discrete sine transform (DST) on the sample values ​​of the residual block 205 to obtain the transformation coefficients 207 in the transform domain. The transformation coefficients 207 can also be called the transformation residual coefficients, representing the residual block 205 in the transform domain.

[0104] Transform processing unit 206 can be used to apply integer approximations of DCT / DST, such as those specified for H.265 / HEVC. Compared to orthogonal DCT transforms, this integer approximation is typically scaled by a certain factor. Other scaling factors are used as part of the transform process to maintain the norm of the residual block after forward and inverse transforms. The scaling factor is usually selected based on certain constraints, such as the scaling factor being a power of 2 used for shift operations, the bit depth of the transform coefficients, and a trade-off between accuracy and implementation cost. For example, a specific scaling factor is specified for the inverse transform (and the corresponding inverse transform on the video decoder 30 side) via inverse transform processing unit 212, etc.; correspondingly, a corresponding scaling factor can be specified for the forward transform on the encoder 20 side via transform processing unit 206, etc.

[0105] An embodiment of the video encoder 20 (specifically the transform processing unit 206) can be used to encode or compress output transform parameters (e.g., one or more types of transforms) directly or through the entropy encoding unit 270, such that, for example, the video decoder 30 can receive and use the transform parameters for decoding.

[0106] Quantification

[0107] The quantization unit 208 can be used to quantize the transform coefficients 207 by applying scalar quantization or vector quantization to obtain quantization coefficients 209. Quantization coefficients 209 can also be called quantization transform coefficients 209 or quantization residual coefficients 209.

[0108] The quantization process can reduce the bit depth associated with some or all of the transform coefficients 207. For example, n-bit transform coefficients can be rounded down to m-bit transform coefficients during quantization, where n is greater than m. The degree of quantization can be modified by adjusting the quantization parameter (QP). For example, for scalar quantization, different degrees of scaling can be used to achieve finer or coarser quantization. Smaller quantization step sizes correspond to finer quantization, while larger quantization step sizes correspond to coarser quantization. The appropriate quantization step size can be represented by the quantization parameter (QP). For example, the quantization parameter can be an index of a predefined set of appropriate quantization step sizes. For example, a small quantization parameter can correspond to fine quantization (small quantization step size), while a large quantization parameter can correspond to coarse quantization (large quantization step size), and vice versa. Quantization may include division by the quantization step size, while corresponding and / or dequantization performed by the dequantization unit 210, etc., may include multiplication by the quantization step size. In some embodiments, the quantization step size can be determined using the quantization parameter according to standards such as HEVC. Typically, the quantization step size can be calculated using a fixed-point approximation of the equations involving division, based on the quantization parameters. Additional scaling factors can be introduced for quantization and dequantization to recover the norm of the residual block, which may have been modified due to scaling used in the fixed-point approximation of the equations for the quantization step size and quantization parameters. In one exemplary implementation, the scaling of the inverse transform and dequantization can be combined. Alternatively, a custom quantization table can be used and signaled to the decoder by the encoder via the bitstream, etc. Quantization is a lossy operation, where the loss increases with the size of the quantization step.

[0109] An embodiment of the video encoder 20 (specifically the quantization unit 208) can be used to encode output quantization parameters (QPs) directly or through the entropy coding unit 270, such that a video decoder 30 can receive and use the quantization parameters for decoding.

[0110] Inverse Quantization

[0111] The dequantization unit 210 is used to apply the dequantization of the quantization coefficients to the quantization coefficients by using the same quantization step size as the quantization unit 208 and the inverse process of the quantization scheme applied by the quantization unit 208, to obtain the dequantization coefficients 211. The dequantization coefficients 211 can also be called the dequantization residual coefficients 211, corresponding to the transform coefficients 207. However, due to the loss caused by quantization, the dequantization coefficients 211 are usually different from the transform coefficients 207.

[0112] Inverse transform

[0113] The inverse transform processing unit 212 performs the inverse transform of the transform processed by the transform processing unit 206, such as the inverse discrete cosine transform (DCT) or the inverse discrete sine transform (DST) or other inverse transforms, to obtain the reconstructed residual block 213 (or the corresponding dequantization coefficients 213) in the sample domain. The reconstructed residual block 213 can also be called the transform block 213.

[0114] reconstruction

[0115] The reconstruction unit 214 (e.g., adder or summer 214) is used to add the transformation block 213 (i.e., the reconstruction residual block 213) to the prediction block 265, for example, by adding the sample values ​​of the reconstruction residual block 213 and the sample values ​​of the prediction block 265 one sample at a time, to obtain the reconstruction block 215 in the sample domain.

[0116] Filtering

[0117] Loop filtering unit 220 (or simply "loop filter" 220) is used to filter the reconstructed block 215 to obtain the filtered block 221, or typically to filter the reconstructed samples to obtain filtered sample values. For example, the loop filtering unit is used to smooth pixel transitions or improve video quality. Loop filtering unit 220 may include one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as an adaptive loop filter (ALF), a noise suppression filter (NSF), or any combination thereof. In one example, loop filtering unit 220 may include a deblocking filter, a SAO filter, and an ALF filter. The filtering process may be performed in the order of deblocking filtering, SAO filtering, and ALF filtering. In another example, a process called luma mapping with chromascaling (LMCS) (i.e., adaptive in-loop reshaping) is added. This process is performed before deblocking filtering. In another example, the deblocking filtering process can also be applied to internal sub-block edges, such as affine sub-block edges, ATMVP sub-block edges, sub-block transform (SBT) edges, and intra sub-partition (ISP) edges. Although the loop filtering unit 220 in... Figure 2 The filter is shown as an in-loop filter, but in other configurations, the loop filter unit 220 can be implemented as a post-loop filter. The filter block 221 can also be called the filter reconstruction block 221.

[0118] An embodiment of the video encoder 20 (specifically the loop filter unit 220) can be used to encode output loop filter parameters (such as SAO filter parameters, ALF filter parameters, or LMCS parameters) directly or through the entropy coding unit 270, so that, for example, the decoder 30 can receive and apply the same loop filter parameters or the corresponding loop filter for decoding.

[0119] Decoding image buffer

[0120] The decoded picture buffer (DPB) 230 can be a memory that stores a reference image or typically stores reference image data for the video encoder 20 to encode video data. The DPB 230 can be formed from any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The decoded picture buffer (DPB) 230 can be used to store one or more filter blocks 221. The decoded picture buffer 230 can also be used to store other previously filtered blocks (e.g., previously reconstructed and filtered blocks 221) of the same current image or different images (e.g., previously reconstructed images), and can provide a complete previously reconstructed (i.e., decoded) image (and corresponding reference blocks and samples) and / or a partially reconstructed current image (and corresponding reference blocks and samples) for inter-frame prediction, etc. For example, when the reconstructed block 215 is not filtered by the loop filter unit 220, the decoded picture buffer (DPB) 230 can also be used to store one or more unfiltered reconstructed blocks 215, or typically store unfiltered reconstructed samples, or any other unprocessed versions of the reconstructed blocks or reconstructed samples.

[0121] Pattern selection (segmentation and prediction)

[0122] The mode selection unit 260 includes a segmentation unit 262, an inter-frame prediction unit 244, and an intra-frame prediction unit 254, for receiving or obtaining raw image data (e.g., raw block 203 (current block 203 of the current image 17)) and reconstructed image data (e.g., filtered and / or unfiltered reconstructed samples or blocks of the same (current) image and / or one or more previously decoded images) from the decoded image buffer 230 or other buffers (e.g., line buffers, not shown)). The reconstructed image data is used as reference image data for predictions such as inter-frame prediction or intra-frame prediction to obtain prediction block 265 or prediction value 265.

[0123] The mode selection unit 260 can be used to determine or select a segmentation mode for the current block prediction mode (including non-segmentation mode) and prediction mode (e.g., intra-frame prediction mode or inter-frame prediction mode), and generate a corresponding prediction block 265, which is used for the calculation of residual block 205 and the reconstruction of reconstruction block 215.

[0124] Embodiments of the mode selection unit 260 can be used to select segmentation and prediction modes (e.g., from prediction modes supported by or available to the mode selection unit 260), which provide the best match or minimum residual (minimum residual implies better compression in transmission or storage), or provide minimum indication overhead (minimum indication overhead implies better compression in transmission or storage), or both. The mode selection unit 260 can be used to determine the segmentation and prediction modes based on rate distortion optimization (RDO), i.e., selecting the prediction mode that provides minimum rate distortion. In this document, the terms "best," "minimum," "optimal," etc., do not necessarily refer to "best," "minimum," "optimal," etc., overall, but can also refer to situations that meet termination or selection criteria. For example, a value exceeding or falling below a threshold or other limitation may lead to a "suboptimal selection," but will reduce complexity and processing time.

[0125] In other words, segmentation unit 262 can be used to segment images of a video sequence into a series of coding tree units (CTUs), and CTU 203 can be further segmented into smaller segments or sub-blocks (forming blocks again), for example, using quad-tree (QT) segmentation, binary-tree (BT) segmentation, or triple-tree (TT) segmentation, or any combination thereof, iteratively, and making predictions for each segment or sub-block, wherein the mode selection includes selecting the tree structure of segmentation block 203 and applying the prediction mode to each segment or sub-block.

[0126] The segmentation (e.g., by segmentation unit 260) and prediction processing (performed by inter-frame prediction unit 244 and intra-frame prediction unit 254) performed by the exemplary video encoder 20 will be described in detail below.

[0127] segmentation

[0128] Segmentation unit 262 can be used to segment an image in a video sequence into a series of coding tree units (CTUs). Segmentation unit 262 can divide (or partition) coding tree units (CTUs) 203 into smaller segments, such as square or rectangular blocks. For an image with three sample arrays, a CTU consists of N×N luma sample blocks and two corresponding chroma sample blocks. The maximum allowed size of the luma block in a CTU is specified as 128×128 in the developing Versatile Video Coding (VVC), but may be specified in the future as a value different from 128×128, such as 256×256. CTUs of an image can be aggregated / grouped into strips / blocks, blocks, or bricks. A block covers a rectangular area of ​​an image, and a block can be divided into one or more bricks. A brick consists of multiple rows of CTUs within a block. A block that is not divided into multiple bricks can be called a brick. However, a brick is a proper subset of a block and therefore is not called a block. VVC supports two tile grouping modes: raster scan strip / tile grouping mode and rectangular strip mode. In raster scan tile grouping mode, a strip / tile group consists of a series of tiles under a tiled raster scan of the image. In rectangular strip mode, a strip consists of multiple tiles of the image, which together form a rectangular region of the image. The individual tiles in a rectangular strip are arranged in the raster scan order of the strip's tiles. These smaller tiles (also called sub-tiles) can be further subdivided into even smaller segments. This is also called tree segmentation or hierarchical tree segmentation, where the root tile at root tree level 0 (layer 0, depth 0), etc., can be recursively subdivided into at least two tiles at the next lower tree level, such as nodes at tree level 1 (layer 1, depth 1). These tiles can then be further subdivided into at least two tiles at the next lower level, such as tree level 2 (layer 2, depth 2), etc., until the segmentation ends due to the fulfillment of a termination criterion, such as reaching the maximum tree depth or the minimum tile size. Tiles that are not further subdivided are also called leaf tiles or leaf nodes of the tree. A tree that is divided into two parts is called a binary tree (BT), a tree that is divided into three parts is called a ternary tree (TT), and a tree that is divided into four parts is called a quad tree (QT).

[0129] For example, a coding tree unit (CTU) can be or include a CTB of luminance samples and two corresponding CTBs of chrominance samples from an image with three sample arrays, or a CTB of samples from a black and white image or an image decoded using three independent color planes and syntax structures for decoding the samples. Correspondingly, a coding tree block (CTB) can be an N×N sample block, where N can be set to a value to divide the components into multiple CTBs; this is called segmentation. A coding unit (CU) can be or include a coding block of luminance samples and two corresponding coding blocks of chrominance samples from an image with three sample arrays, or a coding block consisting of samples from a black and white image or an image decoded using three separate color planes and syntax structures. These syntax structures are used to decode the aforementioned samples. Correspondingly, a coding block (CB) can be an M×N sample block, where M and N can be set to values ​​to divide the CTB into multiple coding blocks; this is called segmentation.

[0130] In an embodiment, for example according to HEVC, a coding tree unit (CTU) can be divided into multiple CUs using a quadtree structure represented as a coding tree. At the leaf CU level, it is determined whether to use inter-frame (temporal) prediction or intra-frame (spatial) prediction to decode the image region. Each leaf CU can be further divided into one, two, or four PUs based on the PU partitioning type. The same prediction process is applied within a PU, and relevant information is sent to the decoder on a PU-by-PU basis. After obtaining residual blocks by applying the prediction process according to the PU partitioning type, the leaf CU can be partitioned into transform units (TUs) according to another quadtree structure similar to that used for the CU.

[0131] For example, in an embodiment, according to the latest video coding standard currently under development (called Versatile Video Coding (VVC)), a combination of quadtrees with nested multi-type trees (e.g., binary and ternary trees) is used to partition the segmentation structure, for example, for segmenting coding tree units. Within the coding tree structure of a coding tree unit, a CU can be a square or a rectangle. For example, the coding tree unit (CTU) is first segmented using a quadtree. Then, the leaf nodes of the quadtree can be further segmented using a multi-type tree structure. In the multi-type tree structure, there are four partitioning types: vertical binary tree partition (SPLIT_BT_VER), horizontal binary tree partition (SPLIT_BT_HOR), vertical ternary tree partition (SPLIT_TT_VER), and horizontal ternary tree partition (SPLIT_TT_HOR). The leaf nodes of the multi-type tree are called coding units (CUs), and unless the CU is greater than the maximum transform length, the segment is used for prediction and transform processing without any further segmentation. This means that, in most cases, the CU, PU, ​​and TU have the same block size in a quadtree-nested multi-type tree coding block structure. This anomaly occurs when the maximum supported transform length is less than the width or height of the CU's color component. VVC defines a unique indication mechanism for partitioning information in a coding tree structure with quadtree-nested multi-type trees. In this mechanism, the coding tree unit (CTU) is processed as the root of the quadtree and is first partitioned through the quadtree structure. Then, each quadtree leaf node (when large enough to be partitioned) is further partitioned through a multi-type tree structure. In the multi-type tree structure, a first flag (mtt_split_cu_flag) indicates whether a node is further partitioned; when a node is further partitioned, a second flag (mtt_split_cu_vertical_flag) indicates the partition direction, and a third flag (mtt_split_cu_binary_flag) indicates whether the partition is a binary or ternary partition. Based on the values ​​of `mtt_split_cu_vertical_flag` and `mtt_split_cu_binary_flag`, the decoder can derive the multi-type tree partitioning mode (MttSplitMode) of the CU based on predefined rules or tables. It should be noted that for certain designs, such as the 64×64 luma block and 32×32 chroma pipeline design in a VVC hardware decoder, TT partitioning is prohibited when the width or height of the luma encoding block is greater than 64. Figure 6As shown. TT partitioning is also prohibited when the width or height of the chroma coding block is greater than 32. Pipeline design divides the image into multiple virtual pipeline data units (VPDUs), defined as non-overlapping units in the image. In the hardware decoder, multiple pipeline stages process consecutive VPDUs simultaneously. In most pipeline stages, the VPDU size is roughly proportional to the buffer size, so it is necessary to keep the VPDUs small. In most hardware decoders, the VPDU size can be set to the maximum transform block (TB) size. However, in VVC, ternary tree (TT) and binary tree (BT) partitioning can increase the VPDU size.

[0132] Additionally, it should be noted that when a part of a tree node block extends beyond the lower or right boundary of the image, the tree node block is forcibly divided until all samples of each decoded CU are within the image boundary.

[0133] For example, the intra sub-partition (ISP) tool can divide the luminance intra-prediction block vertically or horizontally into 2 or 4 sub-partitions based on the block size.

[0134] In one example, the mode selection unit 260 of the video encoder 20 can be used to perform any combination of the segmentation techniques described herein.

[0135] As described above, the video encoder 20 is used to determine or select the best or optimal prediction mode from (e.g., a predetermined) set of prediction modes. For example, the set of prediction modes may include intra-frame prediction modes and / or inter-frame prediction modes.

[0136] Intra-frame prediction

[0137] The intra-prediction mode set can include 35 different intra-prediction modes, such as non-directional modes like DC (or mean) mode and planar mode, or directional modes as defined in HEVC; or it can include 67 different intra-prediction modes, such as non-directional modes like DC (or mean) mode and planar mode, or directional modes as defined in VVC. For example, several conventional angular intra-prediction modes are adaptively replaced with, for example, wide-angle intra-prediction modes for non-square blocks as defined in VVC. As another example, to avoid division operations in DC prediction, only the longer side is used to calculate the average value of non-square blocks. Furthermore, the intra-prediction results of planar mode can be modified using the position-dependent intra-prediction combination (PDPC) method.

[0138] Intra-prediction unit 254 is used to generate intra-prediction block 265 using reconstructed samples of neighboring blocks in the same current image, based on intra-prediction modes in the intra-prediction mode set.

[0139] Intra-prediction unit 254 (or typically mode selection unit 260) is also used to output intra-prediction parameters (or typically information about the selected intra-prediction mode of the indicator block) to entropy coding unit 270 in the form of syntax element 266, so as to be included in the encoded image data 21, such that video decoder 30 can receive and use the prediction parameters for decoding.

[0140] Inter-frame prediction

[0141] The set of possible inter-frame prediction modes depends on the available reference image (i.e., at least a portion of the decoded image stored in the DPB230 as described above) and other inter-frame prediction parameters, such as whether the entire reference image or only a portion of the reference image (e.g., the search window region around the current block) is used to search for the best matching reference block, and / or, for example, whether pixel interpolation is performed, such as half-pixel, quarter-pixel, and / or 1 / 16-pixel interpolation.

[0142] In addition to the prediction modes mentioned above, skip mode, direct mode and / or other inter-frame prediction modes can also be applied.

[0143] For example, extended fusion prediction, this mode's fusion candidate list consists of five candidate types in sequence: spatial MVP of spatially adjacent CUs, temporal MVP of co-located CUs, history-based MVP of the FIFO table, pairwise average MVP, and zero MV. Furthermore, decoder-side motion vector refinement (DMVR) based on bilateral matching can be applied to improve the MV accuracy of the fusion mode. Merge mode with MVD (MMVD) originates from fusion mode with motion vector differences. The MMVD flag is indicated immediately after sending the skip flag and fusion flag to indicate whether the CU uses MMVD mode. Additionally, a CU-level adaptive motion vector resolution (AMVR) scheme can be applied. AMVR supports decoding the CU's MVD with different accuracies. The MVD of the current CU can be adaptively selected based on its prediction mode. When decoding a CU in fusion mode, combined inter / intra prediction (CIIP) mode can be applied to the current CU. The inter-frame and intra-frame prediction signals are weighted and averaged to obtain the CIIP prediction. For affine motion compensation prediction, the affine motion field of the block is described using motion information from two control point motion vectors (4 parameters) or three control point motion vectors (6 parameters). Subblock-based temporal motion vector prediction (SbTMVP) is similar to temporal motion vector prediction (TMVP) in HEVC, but it predicts the motion vectors of sub-CUs within the current CU. Bidirectional optical flow (BDOF), formerly known as BIO, is a simplified version with significantly reduced computational requirements, particularly in terms of the number of multiplications and the size of the multipliers. In the triangular partitioning mode, the CU is uniformly divided into two triangular parts using either diagonal or anti-diagonal partitioning. Furthermore, the bidirectional prediction mode extends the simple averaging to support weighted averaging of the two prediction signals.

[0144] Inter-frame prediction unit 244 may include a motion estimation (ME) unit and a motion compensation (MC) unit (neither of which is in the same frame). Figure 2(As shown in the figure). The motion estimation unit can be used to receive or acquire image block 203 (current image block 203 of current image 17) and decoded image 231, or at least one or more previously reconstructed blocks, such as reconstructed blocks of one or more other / different previously decoded images 231, for motion estimation. For example, the video sequence may include the current image and the previously decoded image 231, or in other words, the current image and the previously decoded image 231 may be part of or constitute an image sequence that makes up a video sequence.

[0145] For example, encoder 20 can be used to select a reference block from multiple reference blocks of the same or different images in multiple other images, and provide the offset (spatial offset) between the position (x-coordinate, y-coordinate) of the reference image (or reference image index) and / or the position of the reference block and the position of the current block as an inter-frame prediction parameter to the motion estimation unit. This offset is also called a motion vector (MV).

[0146] The motion compensation unit is used to obtain (e.g., receive) inter-frame prediction parameters and perform inter-frame prediction based on or using the inter-frame prediction parameters to obtain inter-frame prediction block 265. Motion compensation performed by the motion compensation unit may include extracting or generating prediction blocks based on motion / block vectors determined by motion estimation, and may also include performing interpolation with sub-pixel precision. Interpolation filtering can generate other pixel samples based on known pixel samples, potentially increasing the number of candidate prediction blocks that can be used to decode image blocks. After receiving the motion vector corresponding to the PU of the current image block, the motion compensation unit can locate the prediction block pointed to by the motion vector in one of the reference image lists.

[0147] The motion compensation unit can also generate syntax elements associated with blocks and video stripes for use by the video decoder 30 when decoding image blocks of the video stripe. In addition to stripes and corresponding syntax elements, or as a substitute for stripes and corresponding syntax elements, chunk groups and / or chunks and their corresponding syntax elements can also be generated or used.

[0148] Entropy coding

[0149] Entropy coding unit 270 is used to apply or not apply entropy coding algorithms or schemes (such as variable length coding (VLC), context adaptive VLC (CAVLC) scheme, arithmetic coding scheme, binarization, context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding methods or techniques) to (uncompressed) quantization coefficients 209, inter-frame prediction parameters, intra-frame prediction parameters, loop filter parameters, and / or other syntax elements, to obtain encoded image data 21 that can be output as an encoded bitstream 21 through output terminal 272, so that video decoder 30 and the like can receive and use these parameters for decoding. The encoded bitstream 21 can be sent to video decoder 30, or stored in memory for later transmission or retrieval by video decoder 30.

[0150] Other structural variations of the video encoder 20 can be used to encode video streams. For example, for certain blocks or frames, the non-transform-based encoder 20 can directly quantize the residual signal without the transform processing unit 206. In another implementation, the quantization unit 208 and the inverse quantization unit 210 in the encoder 20 can be combined into a single unit.

[0151] Decoder and Decoding Method

[0152] Figure 3 An example of a video decoder 30 for implementing the techniques of this application is shown. The video decoder 30 is used to receive encoded image data 21 (e.g., an encoded bitstream 21) encoded, for example, by encoder 20, to obtain a decoded image 331. The encoded image data or bitstream includes information for decoding the encoded image data, such as data representing image blocks of encoded video stripes (and / or chunks or blocks) and associated syntax elements.

[0153] exist Figure 3In the example, decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., a summer 314), a loop filter 320, a decoded picture buffer (DPB) 330, a mode application unit 360, an inter-frame prediction unit 344, and an intra-frame prediction unit 354. The inter-frame prediction unit 344 may be or may include a motion compensation unit. In some examples, video decoder 30 may perform substantially the same functions as... Figure 2 The video encoder 100 shown performs the decoding process in the opposite direction to the encoding process.

[0154] As described for encoder 20, the inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, loop filter 220, decoded picture buffer (DPB) 230, inter-frame prediction unit 344, and intra-frame prediction unit 354 also constitute the "built-in decoder" of video encoder 20. Accordingly, the function of inverse quantization unit 310 can be the same as that of inverse quantization unit 110; the function of inverse transform processing unit 312 can be the same as that of inverse transform processing unit 212; the function of reconstruction unit 314 can be the same as that of reconstruction unit 214; the function of loop filter 320 can be the same as that of loop filter 220; and the function of decoded picture buffer 330 can be the same as that of decoded picture buffer 230. Therefore, the description of the corresponding units and functions of video encoder 20 is correspondingly applicable to the corresponding units and functions of video decoder 30.

[0155] Entropy Decoding

[0156] The entropy decoding unit 304 is used to parse the bitstream 21 (or generally the encoded image data 21) and perform entropy decoding on the encoded image data 21 to obtain the quantization coefficients 309 and / or the decoded decoding parameters. Figure 3 (Not shown in the image) , such as inter-frame prediction parameters (e.g., reference image index and motion vector), intra-frame prediction parameters (e.g., intra-frame prediction mode or index), transform parameters, quantization parameters, loop filter parameters, and / or any one or all of other syntax elements. The entropy decoding unit 304 can be used to perform a decoding algorithm or scheme corresponding to the encoding scheme described by the entropy encoding unit 270 of the encoder 20. The entropy decoding unit 304 can also be used to provide inter-frame prediction parameters, intra-frame prediction parameters, and / or other syntax elements to the mode application unit 360, and to provide other parameters to other units of the decoder 30. The video decoder 30 can receive video strip-level and / or video block-level syntax elements. In addition to stripes and corresponding syntax elements, or as a substitute for stripes and corresponding syntax elements, it can also receive and / or use chunk groups and / or chunks and corresponding syntax elements.

[0157] Inverse Quantization

[0158] The dequantization unit 310 can be used to receive quantization parameters (QP) (or generally information related to dequantization) and quantization coefficients from encoded image data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304, etc.), and dequantize the decoded quantization coefficients 309 based on the quantization parameters to obtain dequantized coefficients 311, which may also be referred to as transform coefficients 311. The dequantization process may include using the quantization parameters determined by the video encoder 20 for each video block in the video strip (or block or group of blocks) to determine the degree of quantization, and similarly determine the degree of dequantization to be applied.

[0159] Inverse transform

[0160] The inverse transform processing unit 312 can be used to receive dequantized coefficients 311, also known as transform coefficients 311, and apply a transform to the dequantized coefficients 311 to obtain the reconstructed residual block 213 in the sample domain. The reconstructed residual block 213 can also be referred to as transform block 313. The transform can be an inverse transform, such as inverse DCT, inverse DST, inverse integer transform, or a conceptually similar inverse transform process. The inverse transform processing unit 312 can also be used to receive transform parameters or corresponding information from the encoded image data 21 (e.g., parsed and / or decoded by the entropy decoding unit 304) to determine the transform to be performed on the dequantized coefficients 311.

[0161] reconstruction

[0162] The reconstruction unit 314 (e.g., adder or summer 314) can be used to add the reconstruction residual block 313 to the prediction block 365 by adding the sample values ​​of the reconstruction residual block 313 to the sample values ​​of the prediction block 365, to obtain the reconstruction block 315 in the sample domain.

[0163] Filtering

[0164] Loop filtering unit 320 (in or after the decoding loop) is used to filter the reconstructed block 315 to obtain filtered block 321, to smooth pixel transitions or otherwise improve video quality. Loop filtering unit 320 may include one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as an adaptive loop filter (ALF), a noise suppression filter (NSF), or any combination thereof. In one example, loop filtering unit 320 may include a deblocking filter, a SAO filter, and an ALF filter. The filtering process may be performed in the order of deblocking filtering, SAO filtering, and ALF filtering. In another example, a process called luma mapping with chroma scaling (LMCS) (i.e., adaptive in-loop reshaping) is added. This process is performed before deblocking filtering. In another example, the deblocking filtering process can also be applied to internal sub-block edges, such as affine sub-block edges, ATMVP sub-block edges, sub-block transform (SBT) edges, and intra sub-partition (ISP) edges. Although the loop filtering unit 320 in... Figure 3 The loop filter unit 320 is shown as an in-loop filter, but in other configurations, it can be implemented as a post-loop filter.

[0165] Decoding image buffer

[0166] Then, the decoded video block 321 of the image is stored in the decoded image buffer 330, which stores the decoded image 331 as a reference image. These reference images are used for subsequent motion compensation of other images and / or for output to the display respectively.

[0167] The decoder 30 is used to output the decoded image 311 through the output unit 312, etc., to present to the user or for the user to view.

[0168] predict

[0169] The function of the inter-frame prediction unit 344 can be the same as that of the inter-frame prediction unit 244 (especially the motion compensation unit), and the function of the intra-frame prediction unit 354 can be the same as that of the inter-frame prediction unit 254. It determines whether to segment or divide based on the segmentation and / or prediction parameters or corresponding information received from the encoded image data 21 (e.g., parsing and / or decoding by the entropy decoding unit 304, etc.) and performs prediction. The pattern application unit 360 can be used to perform prediction (intra-frame or inter-frame prediction) on each block based on the reconstructed image, block, or corresponding sample (filtered or unfiltered) to obtain the predicted block 365.

[0170] When a video stripe is decoded into an intra-decoded (I) stripe, the intra-prediction unit 354 of the mode application unit 360 generates prediction blocks 365 for image blocks of the current video stripe based on the indicated intra-prediction mode and data from the previous decoded blocks of the current image. When a video image is decoded into an inter-decoded (i.e., B or P) stripe, the inter-prediction unit 344 (e.g., a motion compensation unit) of the mode application unit 360 generates prediction blocks 365 for video blocks of the current video stripe based on motion vectors and other syntax elements received from the entropy decoding unit 304. For inter-prediction, these prediction blocks can be generated based on one of the reference images in one of the reference image lists. The video decoder 30 can construct reference frame lists: list 0 and list 1, based on the reference images stored in the DPB 330 using a default construction technique. In addition to stripes (e.g., video stripes) or as an alternative to stripes, the same or similar processes can be applied to or by embodiments using chunk groups (e.g., video chunk groups) and / or chunks (e.g., video chunks), for example, video can be decoded using I, P, or B chunk groups and / or chunks.

[0171] The pattern application unit 360 is used to determine the prediction information of video blocks in the current video strip by parsing motion vectors or related information and other syntax elements, and to use the prediction information to generate prediction blocks for the current video block being decoded. For example, the pattern application unit 360 uses some received syntax elements to determine the prediction mode (e.g., intra-frame prediction or inter-frame prediction) for decoding video blocks in the video strip, the inter-frame prediction stripe type (e.g., B stripe, P stripe, or GPB stripe), the construction information of one or more reference image lists of the stripe, the motion vector of each inter-frame encoded video block of the stripe, the inter-frame prediction state of each inter-frame decoded video block of the stripe, and other information to decode video blocks in the current video stripe. In addition to stripes (e.g., video stripes) or as an alternative to stripes, the same or similar process can be applied to or applied by embodiments using chunk groups (e.g., video chunk groups) and / or chunks (e.g., video chunks), for example, video can be decoded using I, P, or B chunk groups and / or chunks.

[0172] Figure 3 The embodiment of the video decoder 30 shown can be used to segment and / or decode images using stripes (also known as video stripes), wherein one or more stripes (typically non-overlapping) can be used to segment or decode an image. Each strip may include one or more blocks (e.g., CTUs) or one or more groups of blocks (e.g., chunks (H.265 / HEVC and VVC) or bricks (VVC)).

[0173] Figure 3 The illustrated embodiment of the video decoder 30 can be used to segment and / or decode an image using stripe / block groups (also known as video block groups) and / or blocks (also known as video blocks). An image can be segmented or decoded using one or more stripe / block groups (typically non-overlapping stripe / block groups). Each stripe / block group may include one or more blocks (such as CTUs) or one or more blocks, and each block may be rectangular or other shapes, and include one or more complete blocks or partial blocks (such as CTUs).

[0174] Other variations of the video decoder 30 can be used to decode the encoded image data 21. For example, the decoder 30 can generate an output video stream without the loop filter unit 320. For example, the non-transform-based decoder 30 can directly dequantize the residual signals of certain blocks or frames without the inverse transform processing unit 312. In another implementation, the dequantization unit 310 and the inverse transform processing unit 312 can be combined into a single unit in the video decoder 30.

[0175] It should be understood that in encoder 20 and decoder 30, the processing result of the current step can be further processed and then output to the next step. For example, after interpolation filtering, motion vector derivation, or loop filtering, the processing result of interpolation filtering, motion vector derivation, or loop filtering can be further calculated, such as clipping or shifting.

[0176] It should be noted that further calculations can be performed on the derived motion vectors of the current block (including but not limited to control point motion vectors in affine mode, sub-block motion vectors in affine mode, planar mode, and ATMVP mode, and time motion vectors). For example, the value of the motion vector can be restricted to a predefined range based on the representation bits of the motion vector. If the representation bit of the motion vector is bitDepth, the value range of the motion vector is –2^(bitDepth–1) to 2^(bitDepth–1)–1, where the “^” symbol represents a power. For example, if bitDepth is set to 16, the range is –32768 to 32767; if bitDepth is set to 18, the range is –131072 to 131071. For example, the derived values ​​of the motion vectors (e.g., the MV of four 4×4 sub-blocks in an 8×8 block) are restricted such that the maximum difference between the integer parts of the MVs of the four 4×4 sub-blocks does not exceed N pixels, such as not exceeding 1 pixel. This section provides two methods for limiting motion vectors based on bitDepth.

[0177] Figure 4 This is a schematic diagram of a video decoding device 400 provided for an embodiment of the present invention. The video decoding device 400 is suitable for implementing the disclosed embodiments described herein. In one embodiment, the video decoding device 400 may be a decoder (such as...) Figure 1A Video decoder 30) or encoder (such as Figure 1A (video encoder 20).

[0178] The video decoding device 400 includes: an input port 410 and a receiving unit (Rx) 420 for receiving data; a processor, logic unit, or central processing unit (CPU) 430 for processing data; a transmitting unit (Tx) 440 and an output port 450 for transmitting data; and a memory 460 for storing data. The video decoding device 400 may also include optical-to-electrical (OE) components and electro-optical (EO) components coupled to the input port 410, receiving unit 420, transmitting unit 440, and output port 450, serving as input or output points for optical or electrical signals.

[0179] Processor 430 is implemented through hardware and software. Processor 430 can be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. Processor 430 communicates with ingress port 410, receiver unit 420, transmitter unit 440, egress port 450, and memory 460. Processor 430 includes a decoding module 470. Decoding module 470 implements the disclosed embodiments described above. For example, decoding module 470 performs, processes, prepares, or provides various decoding operations. Therefore, including decoding module 470 significantly improves the functionality of video decoding device 400, enabling transitions between different states of video decoding device 400. Alternatively, decoding module 470 can be implemented with instructions stored in memory 460 and executed by processor 430.

[0180] Memory 460 may include one or more disks, tape drives, or solid-state drives, and may be used as an overflow data storage device to store programs as selected for execution, as well as instructions and data read during program execution. For example, memory 460 may be volatile and / or non-volatile, and may be read-only memory (ROM), random-access memory (RAM), ternary content-addressable memory (TCAM), and / or static random-access memory (SRAM).

[0181] Figure 5 A simplified block diagram of the apparatus 500 provided for an exemplary embodiment. The apparatus 500 can be used as... Figure 1A The source device 12 and the destination device 14, either or both.

[0182] The processor 502 in device 500 may be a central processing unit. Alternatively, the processor 502 may be any other type of device or multiple devices, existing or to be developed in the future, capable of manipulating or processing information. While the disclosed implementation may be implemented using a single processor, such as the processor 502 shown in the figure, using more than one processor can improve speed and efficiency.

[0183] In one implementation, the memory 504 in device 500 may be a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may be used as memory 504. Memory 504 may include code and data 506 accessed by processor 502 via bus 512. Memory 504 may also include an operating system 508 and an application program 510, wherein application program 510 includes at least one program that enables processor 502 to perform the methods described herein. For example, application program 510 may include application 1 to application N, and also includes a video decoding application that performs the methods described herein.

[0184] The device 500 may also include one or more output devices, such as a display 518. In one example, the display 518 may be a touch-sensitive display that combines a display with a touch-sensitive element capable of sensing touch input. The display 518 may be coupled to the processor 502 via a bus 512.

[0185] Although bus 512 of device 500 is described herein as a single bus, bus 512 may include multiple buses. Furthermore, auxiliary memory 514 may be directly coupled to other components of device 500 or accessible via a network, and may include a single integrated unit (e.g., a memory card) or multiple units (e.g., multiple memory cards). Therefore, device 500 can be implemented in a variety of configurations.

[0186] The use of sequence parameter set (SPS) flags is a well-known technique (e.g., from AVC, HEVC, VVC) to control the decoding algorithm used to reconstruct an image from the decoded representation in the bitstream. For example, if the sps_sao_enabled flag is 1, the SAO algorithm is used to obtain the reconstructed image; otherwise, if the sps_sao_enabled flag is 0, SAO is not used, and it behaves similarly to any existing SPS control flags.

[0187] Tier System

[0188] A crucial task in video codecs is profile definition, which ensures compatibility between the video decoder and the bitstream it receives. For this purpose, video coding standards typically define several sets of functions, called profiles, for specific categories of applications. These profiles are declared using profile codes (profile_idc), and sometimes also using a set of additional constraints applied in the encoder. The profile codes and the constraints of the indications support the decoder's ability to identify the requirements for decoding a particular bitstream. (In many system environments, only one or two profiles are supported, so decoders in these environments do not need to focus on identifying less commonly used profiles). In MPEG-5 / EVC, in addition to the profile_idc mechanism, there is an extension to the decoding tool restrictions called the toolset_idc method. This method assumes an indication of an additional array of {0,1} elements, where each bit corresponds to a specific codec feature, such as the decoding algorithm affecting the decoded image, controlled by the SPS control flag, and imposes additional restrictions on the use of this tool in such a way that if a specific bit in the toolset_idc is equal to 1, the corresponding SPS flag can be 1 or 0. Otherwise, the corresponding SPS flag should be equal to 0. More generally, any particular SPS flag should be less than or equal to the corresponding bit in the toolset_idc array.

[0189] The following syntax table provides the methods indicated by the EVC toolset_idc directive.

[0190]

[0191] The table below provides the SPS flag restrictions in EVC based on the toolset_idc value.

[0192]

[0193]

[0194] It should be noted that in the existing technology, the toolset_idc mechanism only supports restricting the SPS flag value to 0, and there is no method to restrict the SPS flag to 1.

[0195] In view of this, the present invention extends the toolset_idc mechanism to support functionality from both low and high boundaries, thereby overcoming this problem. In other words, the solution provided by the present invention can be used to force 0 and 1 SPS flag values.

[0196] To achieve this, the present invention introduces an additional syntax element to SPS called tools_idc_L, which is used to limit the SPS flag from the lower boundary (i.e. the minimum allowed value of the corresponding SPS flag).

[0197] It should be noted that the additional syntax element can also be represented as tools_idc_l, as shown below, with the same purpose: to limit the SPS flag from the lower boundary (i.e., the minimum allowed value of the corresponding SPS flag).

[0198] In the first embodiment of the present invention, in addition to the tools_idc (tools_idc_H) SPS flag, another SPS flag, tools_idc_L, is introduced. The following table provides examples of the SPS indications provided in this embodiment.

[0199]

[0200] The usage of the toolset_idc_H and toolset_idc_L syntax elements is given below.

[0201]

[0202]

[0203] sps_btt_flag equal to 1 indicates that binary and ternary trees are used for partitioning. sps_btt_flag equal to 0 indicates that binary and ternary trees are not used for partitioning, and only quadtrees are used.

[0204] sps_suco_flag equal to 1 indicates that the decoding order of the partition unit is used. sps_suco_flag equal to 0 indicates that the decoding order of the partition unit is not used.

[0205] `sps_affine_flag` equal to 1 indicates that affine model-based motion compensation can be used for inter-frame prediction. `sps_affine_flag` equal to 0 indicates that the syntax should be constrained so that affine model-based motion compensation is not used in CVS, and that `affine_flag`, `affine_merge_idx`, and `affine_mode_flag` do not exist in the CVS coding unit syntax. When `sps_affine_flag` does not exist, it is inferred to be equal to 0.

[0206] A value of 1 for sps_amvr_flag indicates that adaptive motion vector precision is used. A value of 0 for sps_amvr_flag indicates that adaptive motion vector precision is not used. If sps_amvr_flag does not exist, it is inferred that it is equal to 0.

[0207] A value of 1 for sps_dmvr_flag indicates that decoder-side motion vector correction can be used. A value of 0 for sps_dmvr_flag indicates that decoder-side motion vector correction is not used. If sps_dmvr_flag does not exist, it is inferred that it is equal to 0.

[0208] A value of 1 for sps_mmvd_flag indicates that MMVD can be used. A value of 0 for sps_mmvd_flag indicates that MMVD is not used. If sps_mmvd_flag does not exist, it is inferred that it is equal to 0.

[0209] A value of 1 for sps_alf_flag indicates that an adaptive loop filter can be applied. A value of 0 for sps_alf_flag indicates that an adaptive loop filter cannot be applied.

[0210] A value of 1 for sps_admvp_flag indicates that advanced motion vector prediction, indication, and interpolation are used. A value of 0 for sps_admvp_flag indicates that advanced motion vector prediction, indication, and interpolation are not used.

[0211] sps_eipd_flag equal to 1 indicates that extended intra-prediction mode is used. sps_eipd_flag equal to 0 indicates that extended intra-prediction mode is not used.

[0212] A value of 1 for `sps_ibc_flag` indicates that intra-block copying can be used. A value of 0 for `sps_ibc_flag` indicates that intra-block copying is not used. If `sps_ibc_flag` does not exist, it is inferred that it is equal to 0.

[0213] A value of 1 for sps_cm_init_flag indicates that the context modeling and initialization process is used. A value of 0 for sps_cm_init_flag indicates that the context modeling and initialization process is not used.

[0214] A value of 1 for sps_adcc_flag indicates the use of advanced residual decoding. A value of 0 for sps_adcc_flag indicates the use of run-length residual decoding. If sps_adcc_flag does not exist, it is inferred to be 0.

[0215] sps_iqt_flag equal to 1 indicates that improved quantization and transformation are used. sps_iqt_flag equal to 0 indicates that improved quantization and transformation are not used.

[0216] `sps_ats_flag` equal to 1 indicates that `ats_cu_intra_flag` and `ats_cu_inter_flag` exist in the CVS residual decoding syntax. `sps_ats_flag` equal to 0 indicates that `ats_cu_intra_flag` and `ats_cu_inter_flag` do not exist in the CVS residual decoding syntax. When `sps_ats_flag` does not exist, its value is inferred to be 0.

[0217] A value of 1 for sps_addb_flag indicates that an advanced deblocking filter can be applied. A value of 0 for sps_addb_flag indicates that a deblocking filter can be applied.

[0218] A value of 1 for sps_alf_flag indicates that an adaptive loop filter can be applied. A value of 0 for sps_alf_flag indicates that an adaptive loop filter cannot be applied.

[0219] A value of 1 for sps_htdf_flag indicates that a Hadamard transform domain filter can be applied. A value of 0 for sps_htdf_flag indicates that a Hadamard transform domain filter cannot be applied.

[0220] sps_rpl_flag equal to 1 indicates that the syntax associated with the reference image list exists. sps_rpl_flag equal to 0 indicates that the syntax associated with the reference image list does not exist.

[0221] sps_pocs_flag equal to 1 indicates that syntax related to image sequence numbering exists. sps_pocs_flag equal to 0 indicates that syntax related to image sequence numbering does not exist.

[0222] A value of 1 for `sps_dquant_flag` indicates that the improved incremental QP indicator procedure is used. A value of 0 for `sps_dquant_flag` indicates that the improved incremental QP indicator procedure is not used. When both `sps_btt_flag` and `sps_admvp_flag` are equal to 0, `sps_dquant_flag` should be equal to 0.

[0223] A value of 1 for sps_cm_init_flag indicates that the context modeling and initialization process is used. A value of 0 for sps_cm_init_flag indicates that the context modeling and initialization process is not used.

[0224] A value of 1 for sps_addb_flag indicates that an advanced deblocking filter can be applied. A value of 0 for sps_addb_flag indicates that a deblocking filter can be applied.

[0225] In other words, all specific SPS flags should be within the scope defined by the toolset_idc_H and toolset_idc_L syntax elements.

[0226] It should be noted that the table above only provides an example of the use of this technique, which is only applicable to other SPS flag sets and / or other mappings between SPS flags and {0,1} element arrays.

[0227] In a modification of the first embodiment of the present invention, in addition to the tools_idc(tools_idc_h) SPS flag, another SPS flag, tools_idc_h, is introduced. The following table provides examples of SPS indications provided by this modified embodiment.

[0228]

[0229]

[0230] The following text describes the usage of the toolset_idc_h and toolset_idc_l syntax elements.

[0231] toolset_idc_h represents the set of constraints that CVS conforms to, as indicated in the table below.

[0232] If the feature in the syntax element tools_idc_h is equal to 0, then according to the table below, the corresponding SPS flag should be equal to 0.

[0233] Otherwise, when the bit value in the syntax element tools_idc_h is equal to 1, the corresponding SPS flag value is unrestricted according to the table below.

[0234] toolset_idc_l represents the set of constraints that CVS conforms to, as indicated in the table below.

[0235] If the feature in the syntax element toolset_idc_l is equal to 1, then according to the table below, the corresponding SPS flag should be equal to 1.

[0236] Otherwise, when the bit value in the syntax element toolset_idc_l is equal to 0, the corresponding SPS flag value is unconstrained according to the table below.

[0237] The value of toolset_idc_l should satisfy the following constraints:

[0238] The value of —(toolset_idc_l|toolset_idc_h) should be equal to toolset_idc_h.

[0239]

[0240]

[0241] In other words, all specific SPS flags should be within the scope defined by the toolset_idc_h and toolset_idc_l syntax elements.

[0242] It should be noted that, in addition, the table of the first modified embodiment shown above only provides an example of the use of this technology, and it is only applicable to other SPS flag sets and / or other mappings between SPS flags and {0,1} element arrays.

[0243] Figure 8 A flowchart of a method for decoding a video stream provided by an embodiment of the present invention is shown. Figure 8 The method shown is a method for decoding a video stream implemented by a decoding device, wherein the video stream includes data representing one or more sequence parameter set (SPS) flags, the method comprising: for one of the one or more SPS flags, (1601) obtaining two variables of the SPS flag by parsing the video stream, wherein the value of one of the two variables is the minimum allowed value of the SPS flag, and the value of the other variable is the maximum allowed value of the SPS flag; (1603) obtaining the value of the SPS flag by parsing the video stream; (1605) evaluating whether the value of the SPS flag is valid based on the values ​​of the two variables; (1607) when the value of the SPS flag is valid, decoding the video stream based on the value of the SPS flag.

[0244] Figure 9 An embodiment of the present invention illustrates a decoder 30 for decoding video bitstreams, which can be corresponding to... Figure 8 The method. Figure 9 The decoder 30 shown is used to decode a video stream, wherein the video stream includes data representing one or more sequence parameter set (SPS) flags. The decoder includes: a first acquisition unit 3001, used to obtain two variables of one of the one or more SPS flags by parsing the video stream, wherein the value of one of the two variables is the minimum allowed value of the SPS flag, and the value of the other variable is the maximum allowed value of the SPS flag; a second acquisition unit 3003, used to obtain the value of the SPS flag by parsing the video stream; an evaluation unit 3005, used to evaluate whether the value of the SPS flag is valid based on the values ​​of the two variables; and a decoding unit 3007, used to decode the video stream based on the value of the SPS flag when the value of the SPS flag is valid.

[0245] The unit may be a software module executed by a processor or processing circuitry.

[0246] Both the first acquisition unit 3001 and the second acquisition unit 3003 can be entropy decoding units 304. Both the evaluation unit 3005 and the decoding unit 3007 can be... Figure 3 The unit associated with the SPS flag in the diagram. Decoder 30 can be destination device 14, device 500, video decoder 3206, or terminal device 3106.

[0247] Figure 10 A flowchart of a method for encoding a video stream according to an embodiment of the present invention is shown. Figure 10 The method shown is a method for encoding a video stream implemented by an encoding device, wherein the video stream includes data representing one or more sequence parameter set (SPS) flags, the method comprising: (1701) for one of the one or more SPS flags, setting two variables of the SPS flag respectively, wherein the value of one of the two variables is the minimum allowed value of the SPS flag, and the value of the other variable is the maximum allowed value of the SPS flag; (1703) determining the value of the SPS flag based on the two variables of the SPS flag; and (1705) encoding the value of the SPS flag and the two variables of the SPS flag into the video stream.

[0248] Figure 11 An encoder 20 for encoding video streams, provided in an embodiment of the present invention, is illustrated. The decoder can correspond to... Figure 10 The method. Figure 11 The encoder shown is an encoder 20 for encoding a video stream, wherein the video stream includes data representing one or more sequence parameter set (SPS) flags. The encoder includes: a setting unit 2001, configured to set two variables of one of the one or more SPS flags, wherein the value of one of the two variables is the minimum allowed value of the SPS flag, and the value of the other variable is the maximum allowed value of the SPS flag; a determining unit 2003, configured to determine the value of the SPS flag based on the two variables of the SPS flag; and an encoding unit 2005, configured to encode the value of the SPS flag and the two variables of the SPS flag into the video stream.

[0249] The unit may be a software module executed by a processor or processing circuitry.

[0250] Encoding unit 2005 can be entropy coding unit 270. Setting unit 2001 and determining unit 2003 can both be... Figure 2 The unit associated with the SPS flag in the code. Encoder 20 can be source device 12 or device 500.

[0251] The following section describes the application of the encoding and decoding methods shown in the above embodiments, as well as the systems using these methods.

[0252] Figure 6 This is a block diagram of a content delivery system 3100 used to implement content distribution services. The content delivery system 3100 includes a capture device 3102, a terminal device 3106, and optionally a display 3126. The capture device 3102 communicates with the terminal device 3106 via a communication link 3104. The communication link may include the communication channel 13 described above. The communication link 3104 includes, but is not limited to, WIFI, Ethernet, cable, wireless (3G / 4G / 5G), USB, or any combination thereof.

[0253] The capture device 3102 is used to generate data and can encode the data using the encoding method shown in the above embodiments. Alternatively, the capture device 3102 can distribute the data to a streaming media server (not shown), which encodes the data and sends the encoded data to the terminal device 3106. The capture device 3102 includes, but is not limited to, cameras, smartphones or tablets, computers or laptops, video conferencing systems, PDAs, in-vehicle devices, or any combination thereof. For example, the capture device 3102 may include the source device 12 described above. When the data includes video, the video encoder 20 included in the capture device 3102 can actually perform video encoding processing. When the data includes audio (i.e., sound), the audio encoder included in the capture device 3102 can actually perform audio encoding processing. In some practical scenarios, the capture device 3102 distributes the encoded video data and encoded audio data by multiplexing them together. In other practical scenarios, such as in a video conferencing system, the encoded audio data and encoded video data are not multiplexed. The capture device 3102 distributes the encoded audio data and the encoded video data to the terminal device 3106, respectively.

[0254] In the content delivery system 3100, the terminal device 310 receives and regenerates the encoded data. The terminal device 3106 can be a device with data reception and recovery capabilities, such as a smartphone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a television 3114, a set-top box (STB) 3116, a video conferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, an in-vehicle device 3124, or a combination of any of the above devices capable of decoding the encoded data. For example, the terminal device 3106 may include the destination device 14 as described above. When the encoded data includes video, the video decoder 30 included in the terminal device prioritizes video decoding. When the encoded data includes audio, the audio decoder included in the terminal device prioritizes audio decoding.

[0255] For terminal devices with displays, such as smartphones or tablets 3108, computers or laptops 3110, network video recorders (NVRs) / digital video recorders (DVRs) 3112, televisions 3114, personal digital assistants (PDAs) 3122, or in-vehicle devices 3124, the terminal device can feed the decoded data to its display. For terminal devices without displays, such as STBs 3116, video conferencing systems 3118, or video surveillance systems 3120, an external display 3126 is connected to receive and display the decoded data.

[0256] When each device in the system performs encoding or decoding, an image encoding device or an image decoding device as shown in the above embodiments can be used.

[0257] Figure 7This is a schematic diagram of an example structure of terminal device 3106. After terminal device 3106 receives a stream from capture device 3102, protocol processing unit 3202 analyzes the transport protocol of the stream. The protocol includes, but is not limited to, real-time streaming protocol (RTSP), hypertext transfer protocol (HTTP), HTTP live streaming protocol (HLS), MPEG-DASH, real-time transport protocol (RTP), real-time messaging protocol (RTMP), or any combination thereof.

[0258] After processing the stream, the protocol processing unit 3202 generates a stream file. The file is output to the demultiplexing unit 3204. The demultiplexing unit 3204 can separate the multiplexed data into encoded audio data and encoded video data. As mentioned above, in some practical scenarios, such as in a video conferencing system, the encoded audio data and encoded video data are not multiplexed. In this case, the encoded data is sent to the video decoder 3206 and audio decoder 3208 without going through the demultiplexing unit 3204.

[0259] Through demultiplexing, a video elementary stream (ES), an audio ES, and optional subtitles are generated. Video decoder 3206 includes video decoder 30 as described in the above embodiments, decodes the video ES using the decoding method shown in the above embodiments to generate video frames, and feeds this data to synchronization unit 3212. Audio decoder 3208 decodes the audio ES to generate audio frames and sends this data to synchronization unit 3212. Alternatively, the audio frames can be stored in a buffer before being sent to synchronization unit 3212. Figure 7 (Not shown in the image). Similarly, audio frames can be stored in a buffer (not shown in the image) before being sent to the synchronization unit 3212. Figure 7 (Not shown in the text)

[0260] Synchronization unit 3212 synchronizes video and audio frames and provides the video / audio to video / audio display 3214. For example, synchronization unit 3212 synchronizes the presentation of video and audio information. The information can be decoded using timestamps related to the presentation of the decoded audio and visual data, as well as timestamps related to the transmission of the data stream itself.

[0261] If the stream includes subtitles, the subtitle decoder 3210 decodes the subtitles, synchronizes the subtitles with the video and audio frames, and provides the video / audio / subtitles to the video / audio / subtitle display 3216.

[0262] The present invention is not limited to the above-described system. The image encoding device or image decoding device in the above embodiments can be integrated into other systems, such as automotive systems.

[0263] Mathematical operators

[0264] The mathematical operators used in this application are similar to those used in the C programming language. However, the results of integer division and arithmetic shift operations are defined more precisely, and other operations such as exponentiation and real-value division are also defined. Numbering and counting conventions typically start from 0; for example, "first" corresponds to the 0th, "second" corresponds to the 1st, and so on.

[0265] Arithmetic operators

[0266] Arithmetic operators are defined as follows:

[0267] Addition

[0268] – Subtraction (used as a two-parameter operator) or NOT (used as a unary prefix operator).

[0269] Multiplication, including matrix multiplication.

[0270]

[0271] ÷ is used to represent division in mathematical equations, but without truncation or rounding.

[0272] It is used to represent division operations in mathematical equations, but without truncation or rounding.

[0273] Used to calculate the sum of f(i), where i takes all integers between x and y, inclusive.

[0274] The modulo operation x%y returns the remainder when x is divided by y, where x and y are integers and x ≥ 0 and y > 0.

[0275] Logical operators

[0276] Logical operators are defined as follows:

[0277] x&&y Boolean logical AND operation between x and y

[0278] Boolean logical OR operation between x and y

[0279] ! Boolean NOT operator

[0280] x? y:z If x is true or not equal to 0, then calculate the value of y; otherwise, calculate the value of z.

[0281] Relational operators

[0282] Relational operators are defined as follows:

[0283] > greater than

[0284] >= Greater than or equal to

[0285] < less than

[0286] <= Less than or equal to

[0287] == equals

[0288] ! = Not equal to

[0289] When a relational operator is applied to a syntax element or variable that has already been assigned the value "na" (not applicable), the value "na" is treated as a distinct value for that syntax element or variable. The value "na" is not equal to any other value.

[0290] Bitwise operators

[0291] The bitwise operators are defined as follows:

[0292] & is a bitwise AND operation. When used on integer variables, it operates on the two's complement representation of the integer value. When used on binary arguments, if the argument has fewer bits than another argument, the shorter argument is extended by adding more significant bits equal to 0.

[0293] | Bitwise OR operation. When operating on integer variables, it operates on the two's complement representation of the integer value. When operating on binary parameters, if it includes fewer bits than another parameter, the shorter parameter is extended by adding more significant bits equal to 0.

[0294] ^ Bitwise XOR operation. When operating on integer variables, it operates on the two's complement representation of the integer value. When operating on binary parameters, if it includes fewer bits than another parameter, the shorter parameter is extended by adding more significant bits equal to 0.

[0295] The two's complement representation of x >> y is an arithmetic right shift of y bits. This function is defined only for non-negative integer values ​​of y. The result of the right shift is that the value of the most significant bit (MSB) shifted in is equal to the MSB of x before the shift operation.

[0296] x << y The two's complement integer representation of x is arithmetically left-shifted by y binary digits. This function is defined only for non-negative integer values of y. The result of the left shift is that the value of the bit shifted into the least significant bit (LSB) is 0.

[0297] Assignment operator

[0298] The arithmetic operators are defined as follows:

[0299] = Assignment operator

[0300] ++ Increment, i.e., x++ is equivalent to x = x + 1; when used in an array index, it is equal to the value of the variable before the increment operation.

[0301] –– Decrement, i.e., x–– is equivalent to x = x – 1; when used in an array index, it is equal to the value of the variable before the decrement operation.

[0302] += Increase by a specified amount, i.e., x += 3 is equivalent to x = x + 3, and x += (–3) is equivalent to x = x + (–3).

[0303] –= Decrease by a specified amount, i.e., x –= 3 is equivalent to x = x – 3, and x –= (–3) is equivalent to x = x – (–3).

[0304] Range notation

[0305] The following symbols are used to denote the range of values:

[0306] x = y..z x takes integer values from y to z (including the end values), where x, y, and z are integers and z is greater than y.

[0307] Mathematical functions

[0308] The following mathematical functions are defined:

[0309]

[0310] Asin(x) The trigonometric arcsine function, operating on the parameter x, where x is in the range from –1.0 to 1.0 (including the end values), and the output value is in the range from –π÷2 to π÷2 (including the end values), in radians.

[0311] Atan(x) The trigonometric arctangent function, operating on the parameter x, and the output value is in the range from –π÷2 to π÷2 (including the end values), in radians.

[0312]

[0313] Ceil(x) represents the smallest integer value greater than or equal to x.

[0314] Clip1Y (x) = Clip3(0, (1< <BitDepth Y )–1,x)

[0315] Clip1 C (x) = Clip3(0, (1< <BitDepth C )–1,x)

[0316]

[0317] Cos(x) is a trigonometric cosine function that operates on the parameter x, with units of radians.

[0318] Floor(x) is the largest integer value less than or equal to x.

[0319]

[0320] The natural logarithm of Ln(x)x ​​(the logarithm to the base e, where e is the constant of the base of the natural logarithm, 2.718281828...).

[0321] Log2(x) returns the base-2 logarithm of x.

[0322] Log10(x) returns the base-10 logarithm of x.

[0323]

[0324]

[0325] Round(x)=Sign(x)*Floor(Abs(x)+0.5)

[0326]

[0327] Sin(x) is a trigonometric sine function that performs calculations on the parameter x, with the unit being radians.

[0328]

[0329] Swap(x,y) = (y,x)

[0330] Tan(x) is the trigonometric tangent function that performs operations on the parameter x in radians.

[0331] Operation priority order

[0332] When parentheses are not used to explicitly indicate the order of precedence in an expression, the following rules apply:

[0333] - Higher priority operations are evaluated before any lower priority operations.

[0334] Operations of the same priority are evaluated from left to right.

[0335] The table below shows the order of operations from highest to lowest priority. The higher the position in the table, the higher the priority.

[0336] For operators also used in the C programming language, the operator precedence order in this specification is the same as that in the C programming language.

[0337] Table: Operation priority sorted from highest (top of table) to lowest (bottom of table).

[0338] Textual description of logical operations

[0339] In the text, logical operations are described in mathematical form as follows:

[0340]

[0341] It can be described in the following way:

[0342] ...as follows / ...the following shall prevail:

[0343] - If condition 0, then statement 0.

[0344] Otherwise, if condition 1 is true, then statement 1.

[0345] -……

[0346] – Otherwise (a hint about the remaining conditions), then statement n

[0347] Each "if...otherwise, if...otherwise,..." statement in the text begins with "...as follows" or "...the following applies," followed by "if...". The last condition of "if..., otherwise, if..., otherwise,..." is always "otherwise,...". The middle "if...otherwise, if...otherwise,..." statements can be identified by matching "...as follows" or "...the following applies" with the ending "otherwise,...".

[0348] In the text, logical operations are described in mathematical form as follows:

[0349]

[0350] It can be described in the following way:

[0351] ...as follows / ...the following shall prevail:

[0352] - Statement 0 is valid if all of the following conditions are met:

[0353] –Condition 0a

[0354] –Condition 0b

[0355] Otherwise, if one or more of the following conditions are met, then statement 1:

[0356] –Condition 1a

[0357] –Condition 1b

[0358] -……

[0359] Otherwise, statement n

[0360] In the text, logical operations are described in mathematical form as follows:

[0361]

[0362]

[0363] It can be described in the following way:

[0364] If condition 0, then statement 0.

[0365] If condition 1 is true, then statement 1

[0366] While embodiments of the present invention have been described primarily in relation to video decoding, it should be noted that embodiments of the decoding system 10, encoder 20, and decoder 30 (correspondingly, system 10), as well as other embodiments described herein, can also be used for still image processing or decoding, i.e., processing or decoding a single image in video decoding independent of any previous or consecutive images. Typically, if image processing decoding is limited to a single image 17, only the inter-frame prediction units 244 (encoder) and 344 (decoder) are unavailable. All other functions (also referred to as tools or techniques) of the video encoder 20 and video decoder 30 can also be used for still image processing, such as residual calculation 204 / 304, transform 206, quantization 208, inverse quantization 210 / 310, (inverse) transform 212 / 312, segmentation 262 / 362, intra-frame prediction 254 / 354 and / or loop filtering 220 / 320, entropy coding 270, and entropy decoding 304.

[0367] Embodiments of encoder 20 and decoder 30, and the functions described herein with reference to encoder 20 and decoder 30, can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, these functions can be stored as one or more instructions or code in a computer-readable medium or transmitted via a communication medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium (e.g., a data storage medium), or any communication medium that facilitates the transfer of a computer program from one place to another according to a communication protocol, etc. In this way, a computer-readable medium can generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium accessible by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described herein. A computer program product may include a computer-readable medium.

[0368] By way of example and not limitation, these computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Furthermore, any connection may be appropriately referred to as a computer-readable medium. For example, the definition of medium includes coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technology (e.g., infrared, radio, microwave, etc.) if instructions are transmitted from a website, server, or other remote source via coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology (e.g., infrared, radio, and microwave, etc.). However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather refer to non-transient tangible storage media. The disks and optical discs used herein include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), and Blu-ray discs, wherein disks typically reproduce data magnetically, while optical discs utilize lasers to reproduce data optically. Combinations of the above items should also be included within the scope of computer-readable media.

[0369] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein can refer to any of the above-described structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the various functions described herein can be provided within dedicated hardware and / or software modules for encoding and decoding, or incorporated into combined codecs. Furthermore, these techniques can be fully implemented in one or more circuit or logic elements.

[0370] The techniques of this invention can be implemented in a variety of devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or a set of ICs (e.g., chipsets). This invention describes various components, modules, or units to emphasize functional aspects of the apparatus used to perform the disclosed techniques, but these do not necessarily need to be implemented by different hardware units. Rather, as described above, the various units can be combined with suitable software and / or firmware within a codec hardware unit, or provided as a collection of interoperable hardware units including one or more processors as described above.

[0371] The present invention discloses the following other aspects.

[0372] 1. One aspect of a method for processing image or video data, the method comprising:

[0373] Based on the indication attributes of the bitstream, which includes the encoded representation of the data, it is determined whether the decoder is capable of decoding the bitstream.

[0374] The bitstream conforms to the encoding specification and includes multiple sequence-level (SPS) control flags;

[0375] For each element of the non-empty subset of the SPS control flag, two variables in the {0,1} set are indicated;

[0376] Wherein, each element of the non-empty subset of the SPS control flag is equal to one of the indicator variables.

[0377] 2. According to one aspect of the method described in aspect 1, wherein two variables L in the set {0,1} i and H iIn the bitstream, it is indicated before the i-th SPS flag of the non-empty subset of the SPS control flags, where L i ≤sps i ≤H i SPS i It is the i-th SPS flag.

[0378] 3. According to one aspect of the method of aspect 2, wherein two variables in the {0,1} set of all SPS flags corresponding to the non-empty subset of the SPS control flags are indicated as two vectors having elements in the {0,1} set: toolset_idc_L = L0L1,…,L i ,…L n-1 and toolset_idc_H = H0H1,…,H i ,…H n-1 , where n is the number of elements in the subset of the SPS control flag.

[0379] 4. One aspect of a method for decoding a video stream implemented by a decoding device, wherein the video stream includes data representing a plurality of sequence-level (SPS) control flags, the decoding method comprising: obtaining two variables of the SPS control flags by parsing the video stream, wherein the value of one of the two variables is a lower boundary of the value of the SPS control flags, and the value of the other of the two variables is a maximum permissible value of the value of the SPS control flags;

[0380] The encoder image of the reference SPS is decoded based on the two variables.

[0381] 5. According to one aspect of the method of aspect 4, the method further includes: when both variables are equal to 1, determining the value of the SPS control flag to be 1 without parsing the video stream.

[0382] 6. According to one aspect of the method of aspect 4, the method further includes: when both variables are equal to 0, determining the value of the SPS control flag to be 0 without parsing the video stream.

[0383] 7. An aspect of an encoder (20) comprising processing circuitry for performing the method according to any one of aspects 1 to 6.

[0384] 8. An aspect of a decoder (30), the decoder comprising processing circuitry for performing the method according to any one of aspects 1 to 6.

[0385] 9. An aspect of a computer program product comprising program code, said program code, when executed in a computer or processor, for performing the method according to any of the foregoing aspects.

[0386] 10. One aspect of a decoder, the decoder comprising: one or more processors; a non-transitory computer-readable storage medium coupled to the processors and storing a program executed by the processors, wherein, when executed by the processors, the program causes the decoder to perform the method according to any of the preceding aspects.

[0387] 11. One aspect of an encoder, the encoder comprising: one or more processors; a non-transitory computer-readable storage medium coupled to the processors and storing a program executed by the processors, wherein, when executed by the processors, the program causes the encoder to perform the method according to any of the foregoing aspects.

[0388] 12. One aspect of a non-transitory computer-readable storage medium carrying program code, which, when executed by a computer device, causes the computer device to perform the method according to any of the preceding aspects.

Claims

1. A method for decoding a video bitstream, implemented by a decoding device, characterized in that, The video stream includes data representing one or more sequence parameter sets (SPS) identifiers, and the method includes: For one of the one or more SPS flags, by parsing the video stream, two variables of the SPS flag are obtained respectively, wherein the value of one of the two variables is the minimum allowed value of the SPS flag, and the value of the other variable is the maximum allowed value of the SPS flag; The value of the SPS flag is obtained by parsing the video stream; Evaluate the validity of the value of the SPS flag based on the values ​​of the two variables; When the value of the SPS flag is valid, the video stream is decoded according to the value of the SPS flag.

2. The method according to claim 1, characterized in that, The method further includes: stopping decoding the video stream when the value of the SPS flag is invalid.

3. The method according to claim 1 or 2, characterized in that, The two variables of the SPS flag are Boolean variables, which are encapsulated in two syntax elements and correspond to specific bits within the two syntax elements.

4. The method according to claim 1, characterized in that, The values ​​of the two variables come from the set {0, 1}.

5. The method according to claim 1, characterized in that, The value of the SPS flag is equal to one of the values ​​of the two variables.

6. The method according to claim 5, characterized in that, The values ​​of the two variables come from the set {0, 1}, denoted as l. i and h i And indicated in the bitstream before the SPS flag, wherein, l i ≤sps i ≤h i SPS i The value of the SPS flag is i, where i is an integer.

7. The method according to claim 6, characterized in that, For all SPS flags of the one or more SPS flags, the two variables in the set {0, 1} are indicated by two vectors having elements in the set {0, 1}: toolset_idc_l = l0, l1, ..., l i , ..., l n-1 And toolset_idc_h = h0, h1, ..., h i , ..., h n-1 , where n is the number of the one or more SPS flags, and i is an integer.

8. A method for encoding a video stream using an encoding device, characterized in that, The video stream includes data representing one or more sequence parameter sets (SPS) identifiers, and the method includes: For one of the one or more SPS flags, two variables of the SPS flag are set respectively, wherein the value of one of the two variables is the minimum allowed value of the SPS flag, and the value of the other variable is the maximum allowed value of the SPS flag; The value of the SPS flag is determined based on the two variables of the SPS flag; The value of the SPS flag and the two variables of the SPS flag are encoded into the video stream.

9. The method according to claim 8, characterized in that, The value of the SPS flag is determined to be 1 when the value of the variable that is the minimum allowed value of the SPS flag is equal to 1.

10. The method according to claim 8, characterized in that, The value of the SPS flag is determined to be 0 when the value of the variable that is the maximum allowed value of the SPS flag is equal to 0.

11. The method according to claim 8, characterized in that, The method further includes: When the values ​​of the two variables of the SPS flag are both equal to 1, the values ​​of the SPS flag are determined to be 1 respectively.

12. The method according to claim 8, characterized in that, The method further includes: When the values ​​of the two variables of the SPS flag are both equal to 0, the values ​​of the SPS flag are determined to be 0.

13. The method according to any one of claims 8 to 12, characterized in that, The two variables of the SPS flag are Boolean variables, which are encapsulated in two syntax elements and correspond to specific bits within the two syntax elements.

14. The method according to claim 8, characterized in that, The values ​​of the two variables come from the set {0, 1}.

15. The method according to claim 8, characterized in that, The value of the SPS flag is equal to one of the values ​​of the variable.

16. The method according to claim 15, characterized in that, The values ​​of the two variables come from the set {0, 1}, denoted as l. i and h i And indicated in the bitstream before the SPS flag, wherein, l i ≤sps i ≤h i SPS i The value of the SPS flag is i, where i is an integer.

17. The method according to claim 16, characterized in that, For all SPS flags of the one or more SPS flags, the two variables in the set {0, 1} are indicated by two vectors having elements in the set {0, 1}: toolset_idc_l = l0, l1, ..., l i , ..., l n-1 And toolset_idc_h = h0, h1, ..., h i , ..., h n-1 , where n is the number of the one or more SPS flags, and i is an integer.

18. An encoder (20), characterized in that, Includes processing circuitry for performing the method according to any one of claims 8 to 17.

19. A decoder (30), characterized in that, Includes processing circuitry for performing the method according to any one of claims 1 to 7.

20. A computer program product including program code, characterized in that, When the program code is executed in a computer or processor, it is used to perform the method according to any one of claims 1 to 17.

21. A decoder, characterized in that, include: one or more processors; A non-transitory computer-readable storage medium coupled to the processor and storing a program executed by the processor, wherein, when executed by the processor, the program causes the decoder to perform the method according to any one of claims 1 to 7.

22. An encoder, characterized in that, include: one or more processors; A non-transitory computer-readable storage medium coupled to the processor and storing a program executed by the processor, wherein, when executed by the processor, the program causes the encoder to perform the method according to any one of claims 8 to 17.

23. A non-transitory computer-readable medium, characterized in that, The non-transitory computer-readable medium carries program code that, when executed by a computer device, causes the computer device to perform the method according to any one of claims 1 to 17.

24. A decoder (30) for decoding video bitstreams, characterized in that, The video stream includes data representing one or more sequence parameter sets (SPS) flags, and the decoder includes: The first acquisition unit is configured to obtain two variables of one of the one or more SPS flags by parsing the video bitstream, wherein the value of one of the two variables is the minimum allowed value of the SPS flag, and the value of the other variable is the maximum allowed value of the SPS flag. The second acquisition unit is used to obtain the value of the SPS flag by parsing the video bitstream; An evaluation unit is used to evaluate whether the value of the SPS flag is valid based on the values ​​of the two variables; A decoding unit is configured to decode the video stream according to the value of the SPS flag when the value of the SPS flag is valid.

25. An encoder (20) for encoding video bitstreams, characterized in that, The video stream includes data representing one or more sequence parameter sets (SPS) flags, and the encoder includes: An instruction unit is configured to set two variables for one of the one or more SPS flags, wherein the value of one of the two variables is the minimum allowed value of the SPS flag, and the value of the other variable is the maximum allowed value of the SPS flag. A determining unit is configured to determine the value of the SPS flag based on the two variables of the SPS flag; An encoding unit is used to encode the value of the SPS flag and the two variables of the SPS flag into the video stream.

Images